Jiarui's Blog.

Tensorflow and Python Learning Notes

2018/05/02 Share

glob – Filename pattern matching

tf.gfile.Glob

tf.gfile.Glob(filename)

Returns a list of files that match the given pattern(s).

Args:

  • filename: string or iterable of strings. The glob pattern(s).

Returns:

  • A list of strings containing filenames that match the given pattern(s).

Reference:

Example Data

The examples below assume the following test files are present in the current working directory:

1
2
3
4
5
6
7
8
9
10
$ python glob_maketestdata.py

dir
dir/file.txt
dir/file1.txt
dir/file2.txt
dir/filea.txt
dir/fileb.txt
dir/subdir
dir/subdir/subfile.txt

Note Use glob_maketestdata.py in the sample code to create these files if you want to run the examples.

Wildcards

An asterisk (*) matches zero or more characters in a segment of a name. For example, dir/*.

1
2
3
import glob
for name in glob.glob('dir/*'):
print name

The pattern matches every pathname (file or directory) in the directory dir, without recursing further into subdirectories.

1
2
3
4
5
6
7
8
$ python glob_asterisk.py

dir/file.txt
dir/file1.txt
dir/file2.txt
dir/filea.txt
dir/fileb.txt
dir/subdir

To list files in a subdirectory, you must include the subdirectory in the pattern:

1
2
3
4
5
6
7
8
9
import glob

print 'Named explicitly:'
for name in glob.glob('dir/subdir/*'):
print '\t', name

print 'Named with wildcard:'
for name in glob.glob('dir/*/*'):
print '\t', name

The first case above lists the subdirectory name explicitly, while the second case depends on a wildcard to find the directory.

1
2
3
4
5
6
$ python glob_subdir.py

Named explicitly:
dir/subdir/subfile.txt
Named with wildcard:
dir/subdir/subfile.txt

The results, in this case, are the same. If there was another subdirectory, the wildcard would match both subdirectories and include the filenames from both.

Single Character Wildcard

The other wildcard character supported is the question mark (?). It matches any single character in that position in the name. For example,

1
2
3
4
import glob

for name in glob.glob('dir/file?.txt'):
print name

Matches all of the filenames which begin with “file”, have one more character of any type, then end with ”.txt”.

1
2
3
4
5
6
$ python glob_question.py

dir/file1.txt
dir/file2.txt
dir/filea.txt
dir/fileb.txt

Reference

Tensorflow slim (TF-Slim)

Six 库

six是一个专门用来兼容 Python 2 和 Python 3 的库,用来使得代码同时在 Python 2 和 Python 3 上兼容。只需要在代码前加一句就可以了。

1
import six

tf.app.flags.FLAGS

tf.app.flags.FLAGS 可用来传递参数,

1
2
3
4
5
6
7
import tensorflow as tf  

FLAGS = tf.app.flags.FLAGS

tf.app.flags.DEFINE_string('train_data_path', '/home/user/data', 'training data dir')
tf.app.flags.DEFINE_integer('max_sentence_len', 80, 'max num of tokens per query')
tf.app.flags.DEFINE_float('learning_rate', 0.001, 'learning rate')

或者

1
2
3
4
5
6
7
8
9
import tensorflow as tf

flags = tf.app.flags

FLAGS = flags.FLAGS

flags.DEFINE_string('train_logdir', None, 'Where the checkpoint and logs are stored')
flags.DEFINE_boolean(‘fake_data’, False, ‘If true, uses fake data ‘
‘for unit testing.’)

tf.cast(x, dtype, name=None)

此函数是类型转换函数

参数:

  • x:输入
  • dtype:转换目标类型
  • name:名称
  • 返回:Tensor

例:

1
2
3
4
5
# tensor `a` is [1.8, 2.2], dtype=tf.float  
tf.cast(a, tf.int32) ==> [1, 2] # dtype=tf.int32
```

## tf.squeeze

tf.squeeze(
input,
axis=None,
name=None,
squeeze_dims=None
)

1
2
3
4

去掉维数为1的维度。

例:

‘t’ is a tensor of shape [1, 2, 1, 3, 1, 1]

tf.shape(tf.squeeze(t)) # [2, 3]

1
2

也可以指定去掉哪个维度:

‘t’ is a tensor of shape [1, 2, 1, 3, 1, 1]

tf.shape(tf.squeeze(t, [2, 4])) # [1, 2, 3, 1]

1
2
3
4
5
6
7
8
9
10
11
12

## tf.where(a,b,c)

tf.where(a,b,c)函数:

功能:

当a输出结果为true时,tf.where(a,b,c)函数会选择b值输出。

当a输出结果为false时,tf.where(a,b,c)函数会选择c值输出。

例子:

import tensorflow as tf
v1=tf.constant([1.0,2.0,3.0,4.0])
v2=tf.constant([4.0,3.0,2.0,1.0])

with tf.Session() as sess:
init=tf.global_variables_initializer()
sess.run(init)
print(sess.run(tf.greater(v1,v2)))
print(sess.run(tf.where(tf.greater(v1,v2),v1,v2)))

1
2

结果:

[False False True True]
[ 4. 3. 3. 4.]

1
2
3
4
5
6

**转载自**

[https://blog.csdn.net/william_hehe/article/details/78636624](https://blog.csdn.net/william_hehe/article/details/78636624)

## tf.gather

gather(
params,
indices,
validate_indices=None,
name=None
)

1
2
3
4
5
6
7
8
9
10
11
12
13
14

tf.gather(等待被取元素的张量,索引)

tf.gather根据索引,从输入张量中依次取元素,构成一个新的张量。索引的维度可以小于张量的维度。这时,取张量元素时,会把相应的低维当作一个整体取出来。

例如

假设输入张量 [[1,2,3],[4,5,6],[7,8,9]] 是个二维的。如果只给一个一维索引0. 它就把 [1,2,3] 整体取出。如果给两个一维索引,0和1,它就形成 [[1,2,3],[4,5,6]]

**转载自**

[https://zhuanlan.zhihu.com/p/34578921](https://zhuanlan.zhihu.com/p/34578921)

## tf.zeros_like

zeros_like(
tensor,
dtype=None,
name=None,
optimize=True
)

1
2
3
4

Creates a tensor with all elements set to zero.

For example:

tensor = tf.constant([[1, 2, 3], [4, 5, 6]])
tf.zeros_like(tensor) # [[0, 0, 0], [0, 0, 0]]

1
2
3
4
5
6
7

## tf.reshape
tf.reshape(tensor,shape, name=None)

函数的作用是将tensor变换为参数shape的形式。

其中shape为一个列表形式,特殊的一点是列表中可以存在-1。-1代表的含义是不用我们自己指定这一维的大小,函数会自动计算,但列表中只能存在一个-1。(当然如果存在多个-1,就是一个存在多解的方程了)

x = [[1 2 3]

[5 6 7]]

x_f = tf.reshape(x,(1,-1)) # x_f = [[1 2 3 5 6 7]]

1
2

## tf.tile

tf.tile(
input,
multiples,
name=None
)

1
2
3
4

This operation creates a new tensor by replicating input multiples times. The output tensor's i'th dimension has `input.dims(i) * multiples[i]` elements, and the values of input are replicated `multiples[i]` times along the 'i'th dimension. For example, tiling `[a b c d]` by `[2]` produces `[a b c d a b c d]`.

## tf.Session

import tensorflow as tf

Build a graph.

a = tf.constant(5.0)
b = tf.constant(6.0)
c = a * b

Launch the graph in a session.

sess = tf.Session()

Evaluate the tensor c.

print(sess.run(c))

import tensorflow as tf
a = tf.constant(5.0)
b = tf.constant(6.0)
c = a * b
with tf.Session():

We can also use ‘c.eval()’ here.

print(c.eval())

1
2
3
4
5

## `tf.clip_by_value`
`tf.clip_by_value(A, min, max)`:输入一个张量A,把A中的每一个元素的值都压缩在min和max之间。小于min的让它等于min,大于max的元素的值等于max。

例如:

[python] view plain copy
import tensorflow as tf;
import numpy as np;

A = np.array([[1,1,2,4], [3,4,8,5]])

with tf.Session() as sess:
print sess.run(tf.clip_by_value(A, 2, 5))

1
2

输出:

[[2 2 2 4]
[3 4 5 5]]
`

CATALOG
  1. 1. glob – Filename pattern matching
    1. 1.1. tf.gfile.Glob
    2. 1.2. Example Data
    3. 1.3. Wildcards
    4. 1.4. Single Character Wildcard
    5. 1.5. Reference
  2. 2. Tensorflow slim (TF-Slim)
  3. 3. Six 库
  4. 4. tf.app.flags.FLAGS
  5. 5. tf.cast(x, dtype, name=None)
  • ‘t’ is a tensor of shape [1, 2, 1, 3, 1, 1]
  • ‘t’ is a tensor of shape [1, 2, 1, 3, 1, 1]
  • x = [[1 2 3]
  • Build a graph.
  • Launch the graph in a session.
  • Evaluate the tensor c.
  • We can also use ‘c.eval()’ here.