当前位置： &
1,878 次阅读 -
作者：武维 来源：InfoQ
深度学习及TensorFlow简介
深度学习目前已经被应用到图像识别，语音识别，自然语言处理，机器翻译等场景并取得了很好的行业应用效果。至今已有数种深度学习框架，如TensorFlow、Caffe、Theano、Torch、MXNet，这些框架都能够支持深度神经网络、卷积神经网络、深度信念网络和递归神经网络等模型。TensorFlow最初由Google Brain团队的研究员和工程师研发，目前已成为GitHub上最受欢迎的机器学习项目。
TensorFlow开源一周年以来，已有500+contributors，以及11000+个commits。目前采用TensorFlow平台，在生产环境下进行深度学习的公司有ARM、Google、UBER、DeepMind、京东等公司。目前谷歌已把TensorFlow应用到很多内部项目，如谷歌语音识别，GMail，谷歌图片搜索等。TensorFlow主要特性有：
使用灵活：TensorFlow是一个灵活的神经网络学习平台，采用图计算模型，支持High-Level的API，支持Python、C++、Go、Java接口。
跨平台：TensorFlow支持CPU和GPU的运算，支持台式机、服务器、移动平台的计算。并从r0.12版本支持Windows平台。
产品化：TensorFlow支持从研究团队快速迁移学习模型到生产团队。实现了研究团队发布模型，生产团队验证模型，构建起了模型研究到生产实践的桥梁。
高性能：TensorFlow中采用了多线程，队列技术以及分布式训练模型，实现了在多CPU、多GPU的环境下分布式训练模型。
本文主要介绍TensorFlow一些关键技术的使用实践，包括TensorFlow变量、TensorFlow应用架构、TensorFlow可视化技术、GPU使用，以及HDFS集成使用。
TensorFlow变量
TensorFlow中的变量在使用前需要被初始化，在模型训练中或训练完成后可以保存或恢复这些变量值。下面介绍如何创建变量，初始化变量，保存变量，恢复变量以及共享变量。
&span class="token comment" spellcheck="true"&#创建模型的权重及偏置&/span&
weights &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Variable&span class="token punctuation"&(&/span&tf&span class="token punctuation"&.&/span&random_normal&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&&span class="token number"&784&/span&&span class="token punctuation"&,&/span& &span class="token number"&200&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& stddev&span class="token operator"&=&/span&&span class="token number"&0.35&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&"weights"&/span&&span class="token punctuation"&)&/span&
biases &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Variable&span class="token punctuation"&(&/span&tf&span class="token punctuation"&.&/span&zeros&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&&span class="token number"&200&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&"biases"&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#指定变量所在设备为CPU:0&/span&
&span class="token keyword"&with&/span& tf&span class="token punctuation"&.&/span&device&span class="token punctuation"&(&/span&&span class="token string"&"/cpu:0"&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&
v &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Variable&span class="token punctuation"&(&/span&&span class="token punctuation"&.&/span&&span class="token punctuation"&.&/span&&span class="token punctuation"&.&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#初始化模型变量&/span&
init_op &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&global_variables_initializer&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&
sess&span class="token operator"&=&/span&tf&span class="token punctuation"&.&/span&Session&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&
sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&init_op&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#保存模型变量，由三个文件组成model.data，model.index，model.meta&/span&
saver &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&Saver&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&
saver&span class="token punctuation"&.&/span&restore&span class="token punctuation"&(&/span&sess&span class="token punctuation"&,&/span& &span class="token string"&"/tmp/model"&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#恢复模型变量&/span&
saver&span class="token punctuation"&.&/span&restore&span class="token punctuation"&(&/span&sess&span class="token punctuation"&,&/span& &span class="token string"&"/tmp/model"&/span&&span class="token punctuation"&)&/span&
12345678910111213141516171819202122232425262728293031323334353637
&span class="token comment" spellcheck="true"&#创建模型的权重及偏置&/span&&weights &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Variable&span class="token punctuation"&(&/span&tf&span class="token punctuation"&.&/span&random_normal&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&&span class="token number"&784&/span&&span class="token punctuation"&,&/span& &span class="token number"&200&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& stddev&span class="token operator"&=&/span&&span class="token number"&0.35&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&"weights"&/span&&span class="token punctuation"&)&/span&&biases &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Variable&span class="token punctuation"&(&/span&tf&span class="token punctuation"&.&/span&zeros&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&&span class="token number"&200&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&"biases"&/span&&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#指定变量所在设备为CPU:0&/span&&&span class="token keyword"&with&/span& tf&span class="token punctuation"&.&/span&device&span class="token punctuation"&(&/span&&span class="token string"&"/cpu:0"&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&&&&v &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Variable&span class="token punctuation"&(&/span&&span class="token punctuation"&.&/span&&span class="token punctuation"&.&/span&&span class="token punctuation"&.&/span&&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#初始化模型变量&/span&&init_op &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&global_variables_initializer&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&&sess&span class="token operator"&=&/span&tf&span class="token punctuation"&.&/span&Session&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&&sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&init_op&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#保存模型变量，由三个文件组成model.data，model.index，model.meta&/span&&saver &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&Saver&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&&saver&span class="token punctuation"&.&/span&restore&span class="token punctuation"&(&/span&sess&span class="token punctuation"&,&/span& &span class="token string"&"/tmp/model"&/span&&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#恢复模型变量&/span&&saver&span class="token punctuation"&.&/span&restore&span class="token punctuation"&(&/span&sess&span class="token punctuation"&,&/span& &span class="token string"&"/tmp/model"&/span&&span class="token punctuation"&)&/span&
在复杂的深度学习模型中，存在大量的模型变量，并且期望能够一次性地初始化这些变量。TensorFlow提供了tf.variable_scope和tf.get_variable两个API，实现了共享模型变量。tf.get_variable(&name&, &shape&, &initializer&)：表示创建或返回指定名称的模型变量，其中name表示变量名称，shape表示变量的维度信息，initializer表示变量的初始化方法。tf.variable_scope(&scope_name&)：表示变量所在的命名空间，其中scope_name表示命名空间的名称。共享模型变量使用示例如下：
&span class="token comment" spellcheck="true"&#定义卷积神经网络运算规则，其中weights和biases为共享变量&/span&
&span class="token keyword"&def&/span& &span class="token function"&conv_relu&/span&&span class="token punctuation"&(&/span&input&span class="token punctuation"&,&/span& kernel_shape&span class="token punctuation"&,&/span& bias_shape&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&
&span class="token comment" spellcheck="true"&# 创建变量"weights".&/span&
weights &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&get_variable&span class="token punctuation"&(&/span&&span class="token string"&"weights"&/span&&span class="token punctuation"&,&/span& kernel_shape&span class="token punctuation"&,&/span& initializer&span class="token operator"&=&/span&tf&span class="token punctuation"&.&/span&random_normal_initializer&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&# 创建变量 "biases".&/span&
biases &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&get_variable&span class="token punctuation"&(&/span&&span class="token string"&"biases"&/span&&span class="token punctuation"&,&/span& bias_shape&span class="token punctuation"&,&/span& initializer&span class="token operator"&=&/span&tf&span class="token punctuation"&.&/span&constant_initializer&span class="token punctuation"&(&/span&&span class="token number"&0.0&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&)&/span&
conv &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&nn&span class="token punctuation"&.&/span&conv2d&span class="token punctuation"&(&/span&input&span class="token punctuation"&,&/span& weights&span class="token punctuation"&,&/span& strides&span class="token operator"&=&/span&&span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&,&/span& &span class="token number"&1&/span&&span class="token punctuation"&,&/span& &span class="token number"&1&/span&&span class="token punctuation"&,&/span& &span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& padding&span class="token operator"&=&/span&&span class="token string"&'SAME'&/span&&span class="token punctuation"&)&/span&
&span class="token keyword"&return&/span& tf&span class="token punctuation"&.&/span&nn&span class="token punctuation"&.&/span&relu&span class="token punctuation"&(&/span&conv &span class="token operator"&+&/span& biases&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#定义卷积层，conv1和conv2为变量命名空间&/span&
&span class="token keyword"&with&/span& tf&span class="token punctuation"&.&/span&variable_scope&span class="token punctuation"&(&/span&&span class="token string"&"conv1"&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&
&span class="token comment" spellcheck="true"&# 创建变量 "conv1/weights", "conv1/biases".&/span&
relu1 &span class="token operator"&=&/span& conv_relu&span class="token punctuation"&(&/span&input_images&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&5&/span&&span class="token punctuation"&,&/span& &span class="token number"&5&/span&&span class="token punctuation"&,&/span& &span class="token number"&32&/span&&span class="token punctuation"&,&/span& &span class="token number"&32&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&32&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&
&span class="token keyword"&with&/span& tf&span class="token punctuation"&.&/span&variable_scope&span class="token punctuation"&(&/span&&span class="token string"&"conv2"&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&
&span class="token comment" spellcheck="true"&# 创建变量 "conv2/weights", "conv2/biases".&/span&
relu1 &span class="token operator"&=&/span& conv_relu&span class="token punctuation"&(&/span&relu1&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&5&/span&&span class="token punctuation"&,&/span& &span class="token number"&5&/span&&span class="token punctuation"&,&/span& &span class="token number"&32&/span&&span class="token punctuation"&,&/span& &span class="token number"&32&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&32&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&
12345678910111213141516171819202122232425262728293031
&span class="token comment" spellcheck="true"&#定义卷积神经网络运算规则，其中weights和biases为共享变量&/span&&&span class="token keyword"&def&/span& &span class="token function"&conv_relu&/span&&span class="token punctuation"&(&/span&input&span class="token punctuation"&,&/span& kernel_shape&span class="token punctuation"&,&/span& bias_shape&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&&&&&&&span class="token comment" spellcheck="true"&# 创建变量"weights".&/span&&&&&&weights &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&get_variable&span class="token punctuation"&(&/span&&span class="token string"&"weights"&/span&&span class="token punctuation"&,&/span& kernel_shape&span class="token punctuation"&,&/span& initializer&span class="token operator"&=&/span&tf&span class="token punctuation"&.&/span&random_normal_initializer&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&)&/span&&&&&&&span class="token comment" spellcheck="true"&# 创建变量 "biases".&/span&&&&&&biases &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&get_variable&span class="token punctuation"&(&/span&&span class="token string"&"biases"&/span&&span class="token punctuation"&,&/span& bias_shape&span class="token punctuation"&,&/span& initializer&span class="token operator"&=&/span&tf&span class="token punctuation"&.&/span&constant_initializer&span class="token punctuation"&(&/span&&span class="token number"&0.0&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&)&/span&&&&&&conv &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&nn&span class="token punctuation"&.&/span&conv2d&span class="token punctuation"&(&/span&input&span class="token punctuation"&,&/span& weights&span class="token punctuation"&,&/span& strides&span class="token operator"&=&/span&&span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&,&/span& &span class="token number"&1&/span&&span class="token punctuation"&,&/span& &span class="token number"&1&/span&&span class="token punctuation"&,&/span& &span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& padding&span class="token operator"&=&/span&&span class="token string"&'SAME'&/span&&span class="token punctuation"&)&/span&&&&&&&span class="token keyword"&return&/span& tf&span class="token punctuation"&.&/span&nn&span class="token punctuation"&.&/span&relu&span class="token punctuation"&(&/span&conv &span class="token operator"&+&/span& biases&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#定义卷积层，conv1和conv2为变量命名空间&/span&&&span class="token keyword"&with&/span& tf&span class="token punctuation"&.&/span&variable_scope&span class="token punctuation"&(&/span&&span class="token string"&"conv1"&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&&&&&&&span class="token comment" spellcheck="true"&# 创建变量 "conv1/weights", "conv1/biases".&/span&&&&&&relu1 &span class="token operator"&=&/span& conv_relu&span class="token punctuation"&(&/span&input_images&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&5&/span&&span class="token punctuation"&,&/span& &span class="token number"&5&/span&&span class="token punctuation"&,&/span& &span class="token number"&32&/span&&span class="token punctuation"&,&/span& &span class="token number"&32&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&32&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&&&span class="token keyword"&with&/span& tf&span class="token punctuation"&.&/span&variable_scope&span class="token punctuation"&(&/span&&span class="token string"&"conv2"&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&&&&&&&span class="token comment" spellcheck="true"&# 创建变量 "conv2/weights", "conv2/biases".&/span&&&&&&relu1 &span class="token operator"&=&/span& conv_relu&span class="token punctuation"&(&/span&relu1&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&5&/span&&span class="token punctuation"&,&/span& &span class="token number"&5&/span&&span class="token punctuation"&,&/span& &span class="token number"&32&/span&&span class="token punctuation"&,&/span& &span class="token number"&32&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&32&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&
TensorFlow应用架构
TensorFlow的应用架构主要包括模型构建，模型训练，及模型评估三个方面。模型构建主要指构建深度学习神经网络，模型训练主要指在TensorFlow会话中对训练数据执行神经网络运算，模型评估主要指根据测试数据评估模型精确度。如下图所示：
网络模型，损失方程，模型训练操作定义示例如下：
&span class="token comment" spellcheck="true"&#两个隐藏层，一个logits输出层&/span&
hidden1 &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&nn&span class="token punctuation"&.&/span&relu&span class="token punctuation"&(&/span&tf&span class="token punctuation"&.&/span&matmul&span class="token punctuation"&(&/span&images&span class="token punctuation"&,&/span& weights&span class="token punctuation"&)&/span& &span class="token operator"&+&/span& biases&span class="token punctuation"&)&/span&
hidden2 &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&nn&span class="token punctuation"&.&/span&relu&span class="token punctuation"&(&/span&tf&span class="token punctuation"&.&/span&matmul&span class="token punctuation"&(&/span&hidden1&span class="token punctuation"&,&/span& weights&span class="token punctuation"&)&/span& &span class="token operator"&+&/span& biases&span class="token punctuation"&)&/span&
logits &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&matmul&span class="token punctuation"&(&/span&hidden2&span class="token punctuation"&,&/span& weights&span class="token punctuation"&)&/span& &span class="token operator"&+&/span& biases
&span class="token comment" spellcheck="true"&#损失方程，采用softmax交叉熵算法&/span&
cross_entropy &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&nn&span class="token punctuation"&.&/span&sparse_softmax_cross_entropy_with_logits&span class="token punctuation"&(&/span& logits&span class="token punctuation"&,&/span& labels&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&'xentropy'&/span&&span class="token punctuation"&)&/span&
loss &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&reduce_mean&span class="token punctuation"&(&/span&cross_entropy&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&'xentropy_mean'&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#选定优化算法及定义训练操作&/span&
optimizer &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&GradientDescentOptimizer&span class="token punctuation"&(&/span&learning_rate&span class="token punctuation"&)&/span&
global_step &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Variable&span class="token punctuation"&(&/span&&span class="token number"&0&/span&&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&'global_step'&/span&&span class="token punctuation"&,&/span& trainable&span class="token operator"&=&/span&&span class="token boolean"&False&/span&&span class="token punctuation"&)&/span&
train_op &span class="token operator"&=&/span& optimizer&span class="token punctuation"&.&/span&minimize&span class="token punctuation"&(&/span&loss&span class="token punctuation"&,&/span& global_step&span class="token operator"&=&/span&global_step&span class="token punctuation"&)&/span&
模型训练及模型验证示例如下：
&span class="token comment" spellcheck="true"&#加载训练数据，并执行网络训练&/span&
&span class="token keyword"&for&/span& step &span class="token keyword"&in&/span& xrange&span class="token punctuation"&(&/span&FLAGS&span class="token punctuation"&.&/span&max_steps&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&
feed_dict &span class="token operator"&=&/span& fill_feed_dict&span class="token punctuation"&(&/span&data_sets&span class="token punctuation"&.&/span&train&span class="token punctuation"&,&/span& images_placeholder&span class="token punctuation"&,&/span& labels_placeholder&span class="token punctuation"&)&/span&
_&span class="token punctuation"&,&/span& loss_value &span class="token operator"&=&/span& sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&train_op&span class="token punctuation"&,&/span& loss&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& feed_dict&span class="token operator"&=&/span&feed_dict&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#加载测试数据，计算模型精确度&/span&
&span class="token keyword"&for&/span& step &span class="token keyword"&in&/span& xrange&span class="token punctuation"&(&/span&steps_per_epoch&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&
feed_dict &span class="token operator"&=&/span& fill_feed_dict&span class="token punctuation"&(&/span&data_set&span class="token punctuation"&,&/span& images_placeholder&span class="token punctuation"&,&/span& labels_placeholder&span class="token punctuation"&)&/span&
true_count &span class="token operator"&+=&/span& sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&eval_correct&span class="token punctuation"&,&/span& feed_dict&span class="token operator"&=&/span&feed_dict&span class="token punctuation"&)&/span&
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647
&span class="token comment" spellcheck="true"&#两个隐藏层，一个logits输出层&/span&&hidden1 &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&nn&span class="token punctuation"&.&/span&relu&span class="token punctuation"&(&/span&tf&span class="token punctuation"&.&/span&matmul&span class="token punctuation"&(&/span&images&span class="token punctuation"&,&/span& weights&span class="token punctuation"&)&/span& &span class="token operator"&+&/span& biases&span class="token punctuation"&)&/span&&hidden2 &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&nn&span class="token punctuation"&.&/span&relu&span class="token punctuation"&(&/span&tf&span class="token punctuation"&.&/span&matmul&span class="token punctuation"&(&/span&hidden1&span class="token punctuation"&,&/span& weights&span class="token punctuation"&)&/span& &span class="token operator"&+&/span& biases&span class="token punctuation"&)&/span&&logits &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&matmul&span class="token punctuation"&(&/span&hidden2&span class="token punctuation"&,&/span& weights&span class="token punctuation"&)&/span& &span class="token operator"&+&/span& biases&&&&span class="token comment" spellcheck="true"&#损失方程，采用softmax交叉熵算法&/span&&cross_entropy &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&nn&span class="token punctuation"&.&/span&sparse_softmax_cross_entropy_with_logits&span class="token punctuation"&(&/span& logits&span class="token punctuation"&,&/span& labels&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&'xentropy'&/span&&span class="token punctuation"&)&/span&&loss &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&reduce_mean&span class="token punctuation"&(&/span&cross_entropy&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&'xentropy_mean'&/span&&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#选定优化算法及定义训练操作&/span&&optimizer &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&GradientDescentOptimizer&span class="token punctuation"&(&/span&learning_rate&span class="token punctuation"&)&/span&&global_step &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Variable&span class="token punctuation"&(&/span&&span class="token number"&0&/span&&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&'global_step'&/span&&span class="token punctuation"&,&/span& trainable&span class="token operator"&=&/span&&span class="token boolean"&False&/span&&span class="token punctuation"&)&/span&&train_op &span class="token operator"&=&/span& optimizer&span class="token punctuation"&.&/span&minimize&span class="token punctuation"&(&/span&loss&span class="token punctuation"&,&/span& global_step&span class="token operator"&=&/span&global_step&span class="token punctuation"&)&/span&&&&模型训练及模型验证示例如下：&&span class="token comment" spellcheck="true"&#加载训练数据，并执行网络训练&/span&&&span class="token keyword"&for&/span& step &span class="token keyword"&in&/span& xrange&span class="token punctuation"&(&/span&FLAGS&span class="token punctuation"&.&/span&max_steps&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&&&&&&feed_dict &span class="token operator"&=&/span& fill_feed_dict&span class="token punctuation"&(&/span&data_sets&span class="token punctuation"&.&/span&train&span class="token punctuation"&,&/span& images_placeholder&span class="token punctuation"&,&/span& labels_placeholder&span class="token punctuation"&)&/span&&&&&&_&span class="token punctuation"&,&/span& loss_value &span class="token operator"&=&/span& sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&train_op&span class="token punctuation"&,&/span& loss&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& feed_dict&span class="token operator"&=&/span&feed_dict&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#加载测试数据，计算模型精确度&/span&&&span class="token keyword"&for&/span& step &span class="token keyword"&in&/span& xrange&span class="token punctuation"&(&/span&steps_per_epoch&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&&&&&&feed_dict &span class="token operator"&=&/span& fill_feed_dict&span class="token punctuation"&(&/span&data_set&span class="token punctuation"&,&/span& images_placeholder&span class="token punctuation"&,&/span& labels_placeholder&span class="token punctuation"&)&/span&&&&&&true_count &span class="token operator"&+=&/span& sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&eval_correct&span class="token punctuation"&,&/span& feed_dict&span class="token operator"&=&/span&feed_dict&span class="token punctuation"&)&/span&
TensorFlow可视化技术
大规模的深度神经网络运算模型是非常复杂的，并且不容易理解运算过程。为了易于理解、调试及优化神经网络运算模型，数据科学家及应用开发人员可以使用TensorFlow可视化组件：TensorBoard。TensorBoard主要支持TensorFlow模型可视化展示及统计信息的图表展示。TensorBoard应用架构如下：
TensorFlow可视化技术主要分为两部分：TensorFlow摘要模型及TensorBoard可视化组件。在摘要模型中，需要把模型变量或样本数据转换为TensorFlow summary操作，然后合并summary操作，最后通过Summary Writer操作写入TensorFlow的事件日志。TensorBoard通过读取事件日志，进行相关摘要信息的可视化展示，主要包括：Scalar图、图片数据可视化、声音数据展示、图模型可视化，以及变量数据的直方图和概率分布图。TensorFlow可视化技术的关键流程如下所示：
&span class="token comment" spellcheck="true"&#定义变量及训练数据的摘要操作&/span&
tf&span class="token punctuation"&.&/span&summary&span class="token punctuation"&.&/span&scalar&span class="token punctuation"&(&/span&&span class="token string"&'max'&/span&&span class="token punctuation"&,&/span& tf&span class="token punctuation"&.&/span&reduce_max&span class="token punctuation"&(&/span&var&span class="token punctuation"&)&/span&&span class="token punctuation"&)&/span&
tf&span class="token punctuation"&.&/span&summary&span class="token punctuation"&.&/span&histogram&span class="token punctuation"&(&/span&&span class="token string"&'histogram'&/span&&span class="token punctuation"&,&/span& var&span class="token punctuation"&)&/span&
tf&span class="token punctuation"&.&/span&summary&span class="token punctuation"&.&/span&image&span class="token punctuation"&(&/span&&span class="token string"&'input'&/span&&span class="token punctuation"&,&/span& image_shaped_input&span class="token punctuation"&,&/span& &span class="token number"&10&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#定义合并变量操作，一次性生成所有摘要数据&/span&
merged &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&summary&span class="token punctuation"&.&/span&merge_all&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#定义写入摘要数据到事件日志的操作&/span&
train_writer &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&SummaryWriter&span class="token punctuation"&(&/span&FLAGS&span class="token punctuation"&.&/span&log_dir &span class="token operator"&+&/span& &span class="token string"&'/train'&/span&&span class="token punctuation"&,&/span& sess&span class="token punctuation"&.&/span&graph&span class="token punctuation"&)&/span&
test_writer &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&SummaryWriter&span class="token punctuation"&(&/span&FLAGS&span class="token punctuation"&.&/span&log_dir &span class="token operator"&+&/span& &span class="token string"&'/test'&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#执行训练操作，并把摘要信息写入到事件日志&/span&
summary&span class="token punctuation"&,&/span& _ &span class="token operator"&=&/span& sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&merged&span class="token punctuation"&,&/span& train_step&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& feed_dict&span class="token operator"&=&/span&feed_dict&span class="token punctuation"&(&/span&&span class="token boolean"&True&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&)&/span&
train_writer&span class="token punctuation"&.&/span&add_summary&span class="token punctuation"&(&/span&summary&span class="token punctuation"&,&/span& i&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#下载示例code，并执行模型训练&/span&
python mnist_with_summaries&span class="token punctuation"&.&/span&py
&span class="token comment" spellcheck="true"&#启动TensorBoard，TensorBoard的UI地址为http://ip_address:6006&/span&
tensorboard &span class="token operator"&-&/span&&span class="token operator"&-&/span&logdir&span class="token operator"&=&/span&&span class="token operator"&/&/span&path&span class="token operator"&/&/span&to&span class="token operator"&/&/span&log&span class="token operator"&-&/span&directory
1234567891011121314151617181920212223242526272829303132333435363738394041
&span class="token comment" spellcheck="true"&#定义变量及训练数据的摘要操作&/span&&tf&span class="token punctuation"&.&/span&summary&span class="token punctuation"&.&/span&scalar&span class="token punctuation"&(&/span&&span class="token string"&'max'&/span&&span class="token punctuation"&,&/span& tf&span class="token punctuation"&.&/span&reduce_max&span class="token punctuation"&(&/span&var&span class="token punctuation"&)&/span&&span class="token punctuation"&)&/span&&tf&span class="token punctuation"&.&/span&summary&span class="token punctuation"&.&/span&histogram&span class="token punctuation"&(&/span&&span class="token string"&'histogram'&/span&&span class="token punctuation"&,&/span& var&span class="token punctuation"&)&/span&&tf&span class="token punctuation"&.&/span&summary&span class="token punctuation"&.&/span&image&span class="token punctuation"&(&/span&&span class="token string"&'input'&/span&&span class="token punctuation"&,&/span& image_shaped_input&span class="token punctuation"&,&/span& &span class="token number"&10&/span&&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#定义合并变量操作，一次性生成所有摘要数据&/span&&merged &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&summary&span class="token punctuation"&.&/span&merge_all&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#定义写入摘要数据到事件日志的操作&/span&&train_writer &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&SummaryWriter&span class="token punctuation"&(&/span&FLAGS&span class="token punctuation"&.&/span&log_dir &span class="token operator"&+&/span& &span class="token string"&'/train'&/span&&span class="token punctuation"&,&/span& sess&span class="token punctuation"&.&/span&graph&span class="token punctuation"&)&/span&&test_writer &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&SummaryWriter&span class="token punctuation"&(&/span&FLAGS&span class="token punctuation"&.&/span&log_dir &span class="token operator"&+&/span& &span class="token string"&'/test'&/span&&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#执行训练操作，并把摘要信息写入到事件日志&/span&&summary&span class="token punctuation"&,&/span& _ &span class="token operator"&=&/span& sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&merged&span class="token punctuation"&,&/span& train_step&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& feed_dict&span class="token operator"&=&/span&feed_dict&span class="token punctuation"&(&/span&&span class="token boolean"&True&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&)&/span&&train_writer&span class="token punctuation"&.&/span&add_summary&span class="token punctuation"&(&/span&summary&span class="token punctuation"&,&/span& i&span class="token punctuation"&)&/span&&&&&&&span class="token comment" spellcheck="true"&#下载示例code，并执行模型训练&/span&&&&python mnist_with_summaries&span class="token punctuation"&.&/span&py&&&&span class="token comment" spellcheck="true"&#启动TensorBoard，TensorBoard的UI地址为http://ip_address:6006&/span&&tensorboard &span class="token operator"&-&/span&&span class="token operator"&-&/span&logdir&span class="token operator"&=&/span&&span class="token operator"&/&/span&path&span class="token operator"&/&/span&to&span class="token operator"&/&/span&log&span class="token operator"&-&/span&directory
TensorBoard Scalar图如下所示，其中横坐标表示模型训练的迭代次数，纵坐标表示该标量值，例如模型精确度，熵值等。TensorBoard支持这些统计值的下载。
TensorFlow Image摘要信息如下图所示，该示例中显示了测试数据和训练数据中的手写数字图片。
TensorFlow图模型如下图所示，可清晰地展示模型的训练流程，其中的每个方框表示变量所在的命名空间。包含的命名空间有input（输入数据），input_reshape（矩阵变换，用于图形化手写数字）, layer1（隐含层1）, layer2（隐含层2）, dropout（丢弃一些神经元，防止过拟合）, accuracy（模型精确度）, cross_entropy（目标函数值，交叉熵）, train（训练模型）。例如，input命名空间操作后的tensor数据会传递给input_reshape，train，accuracy，layer1，cross_entropy命名空间中的操作。
TensorFlow变量的概率分布如下图所示，其中横坐标为迭代次数，纵坐标为变量取值范围。图表中的线表示概率百分比，从高到底为[maximum, 93%, 84%, 69%, 50%, 31%, 16%, 7%, minimum]。例如，图表中从高到底的第二条线为93%，对应该迭代下有93%的变量权重值小于该线对应的目标值。
上述TensorFlow变量概率分布对应的直方图如下图所示：
TensorFlow GPU使用
GPU设备已经广泛地应用于图像分类，语音识别，自然语言处理，机器翻译等深度学习领域，并实现了开创性的性能改进。与单纯使用CPU相比，GPU 具有数以千计的计算核心，可实现 10-100 倍的性能提升。TensorFlow支持GPU运算的版本为tensorflow-gpu，并且需要先安装相关软件：GPU运算平台CUDA和用于深度神经网络运算的GPU加速库CuDNN。在TensorFlow中，CPU或GPU的表示方式如下所示：
“/cpu:0″：表示机器中第一个CPU。
“/gpu:0″：表示机器中第一个GPU卡。
“/gpu:1″：表示机器中第二个GPU卡。
TensorFlow中所有操作都有CPU和GPU运算的实现，默认情况下GPU运算的优先级比CPU高。如果TensorFlow操作没有指定在哪个设备上进行运算，默认会优选采用GPU进行运算。下面介绍如何在TensorFlow使用GPU：
&span class="token comment" spellcheck="true"&# 定义使用gpu0执行a*b的矩阵运算，其中a，b，c都在gpu0上执行&/span&
&span class="token keyword"&with&/span& tf&span class="token punctuation"&.&/span&device&span class="token punctuation"&(&/span&&span class="token string"&'/gpu:0'&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&
a &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&constant&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&&span class="token number"&1.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&2.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&3.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&4.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&5.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&6.0&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& shape&span class="token operator"&=&/span&&span class="token punctuation"&[&/span&&span class="token number"&2&/span&&span class="token punctuation"&,&/span& &span class="token number"&3&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&'a'&/span&&span class="token punctuation"&)&/span&
b &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&constant&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&&span class="token number"&1.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&2.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&3.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&4.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&5.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&6.0&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& shape&span class="token operator"&=&/span&&span class="token punctuation"&[&/span&&span class="token number"&3&/span&&span class="token punctuation"&,&/span& &span class="token number"&2&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&'b'&/span&&span class="token punctuation"&)&/span&
c &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&matmul&span class="token punctuation"&(&/span&a&span class="token punctuation"&,&/span& b&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&# 通过log_device_placement指定在日志中输出变量和操作所在的设备&/span&
sess &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Session&span class="token punctuation"&(&/span&config&span class="token operator"&=&/span&tf&span class="token punctuation"&.&/span&ConfigProto&span class="token punctuation"&(&/span&log_device_placement&span class="token operator"&=&/span&&span class="token boolean"&True&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&)&/span&
&span class="token keyword"&print&/span& sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&c&span class="token punctuation"&)&/span&
1234567891011121314151617
&span class="token comment" spellcheck="true"&# 定义使用gpu0执行a*b的矩阵运算，其中a，b，c都在gpu0上执行&/span&&&span class="token keyword"&with&/span& tf&span class="token punctuation"&.&/span&device&span class="token punctuation"&(&/span&&span class="token string"&'/gpu:0'&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&&&&a &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&constant&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&&span class="token number"&1.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&2.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&3.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&4.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&5.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&6.0&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& shape&span class="token operator"&=&/span&&span class="token punctuation"&[&/span&&span class="token number"&2&/span&&span class="token punctuation"&,&/span& &span class="token number"&3&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&'a'&/span&&span class="token punctuation"&)&/span&&&&b &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&constant&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&&span class="token number"&1.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&2.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&3.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&4.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&5.0&/span&&span class="token punctuation"&,&/span& &span class="token number"&6.0&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& shape&span class="token operator"&=&/span&&span class="token punctuation"&[&/span&&span class="token number"&3&/span&&span class="token punctuation"&,&/span& &span class="token number"&2&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& name&span class="token operator"&=&/span&&span class="token string"&'b'&/span&&span class="token punctuation"&)&/span&&&&c &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&matmul&span class="token punctuation"&(&/span&a&span class="token punctuation"&,&/span& b&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&# 通过log_device_placement指定在日志中输出变量和操作所在的设备&/span&&sess &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Session&span class="token punctuation"&(&/span&config&span class="token operator"&=&/span&tf&span class="token punctuation"&.&/span&ConfigProto&span class="token punctuation"&(&/span&log_device_placement&span class="token operator"&=&/span&&span class="token boolean"&True&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&)&/span&&&span class="token keyword"&print&/span& sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&c&span class="token punctuation"&)&/span&
本实验环境下只有一个GPU卡，设备的Device Mapping及变量操作所在设备位置如下：
&span class="token comment" spellcheck="true"&#设备的Device Mapping&/span&
&span class="token operator"&/&/span&job&span class="token punctuation"&:&/span&localhost&span class="token operator"&/&/span&replica&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&task&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&gpu&span class="token punctuation"&:&/span&&span class="token number"&0&/span& &span class="token operator"&-&/span&&span class="token operator"&&&/span& device&span class="token punctuation"&:&/span& &span class="token number"&0&/span&&span class="token punctuation"&,&/span& name&span class="token punctuation"&:&/span& Tesla K20c&span class="token punctuation"&,&/span& pci bus id&span class="token punctuation"&:&/span& &span class="token number"&0000&/span&&span class="token punctuation"&:&/span&&span class="token number"&81&/span&&span class="token punctuation"&:&/span&&span class="token number"&00.0&/span&
&span class="token comment" spellcheck="true"&#变量操作所在设备位置&/span&
a&span class="token punctuation"&:&/span& &span class="token punctuation"&(&/span&Const&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span& &span class="token operator"&/&/span&job&span class="token punctuation"&:&/span&localhost&span class="token operator"&/&/span&replica&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&task&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&gpu&span class="token punctuation"&:&/span&&span class="token number"&0&/span&
b&span class="token punctuation"&:&/span& &span class="token punctuation"&(&/span&Const&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span& &span class="token operator"&/&/span&job&span class="token punctuation"&:&/span&localhost&span class="token operator"&/&/span&replica&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&task&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&gpu&span class="token punctuation"&:&/span&&span class="token number"&0&/span&
&span class="token punctuation"&(&/span&MatMul&span class="token punctuation"&)&/span&&span class="token operator"&/&/span&job&span class="token punctuation"&:&/span&localhost&span class="token operator"&/&/span&replica&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&task&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&gpu&span class="token punctuation"&:&/span&&span class="token number"&0&/span&
12345678910111213
&span class="token comment" spellcheck="true"&#设备的Device Mapping&/span&&&span class="token operator"&/&/span&job&span class="token punctuation"&:&/span&localhost&span class="token operator"&/&/span&replica&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&task&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&gpu&span class="token punctuation"&:&/span&&span class="token number"&0&/span& &span class="token operator"&-&/span&&span class="token operator"&>&/span& device&span class="token punctuation"&:&/span& &span class="token number"&0&/span&&span class="token punctuation"&,&/span& name&span class="token punctuation"&:&/span& Tesla K20c&span class="token punctuation"&,&/span& pci bus id&span class="token punctuation"&:&/span& &span class="token number"&0000&/span&&span class="token punctuation"&:&/span&&span class="token number"&81&/span&&span class="token punctuation"&:&/span&&span class="token number"&00.0&/span&&&&&span class="token comment" spellcheck="true"&#变量操作所在设备位置&/span&&a&span class="token punctuation"&:&/span& &span class="token punctuation"&(&/span&Const&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span& &span class="token operator"&/&/span&job&span class="token punctuation"&:&/span&localhost&span class="token operator"&/&/span&replica&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&task&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&gpu&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&b&span class="token punctuation"&:&/span& &span class="token punctuation"&(&/span&Const&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span& &span class="token operator"&/&/span&job&span class="token punctuation"&:&/span&localhost&span class="token operator"&/&/span&replica&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&task&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&gpu&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&&span class="token punctuation"&(&/span&MatMul&span class="token punctuation"&)&/span&&span class="token operator"&/&/span&job&span class="token punctuation"&:&/span&localhost&span class="token operator"&/&/span&replica&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&task&span class="token punctuation"&:&/span&&span class="token number"&0&/span&&span class="token operator"&/&/span&gpu&span class="token punctuation"&:&/span&&span class="token number"&0&/span&
默认配置下，TensorFlow Session会占用GPU卡上所有内存。但TesnorFlow提供了两个GPU内存优化配置选项。config.gpu_options.allow_growth：根据程序运行情况，分配GPU内存。程序开始的时候分配比较少的内存，随着程序的运行，增加内存的分配，但不会释放已经分配的内存。config.gpu_options.per_process_gpu_memory_fraction：表示按照百分比分配GPU内存，例如0.4表示分配40%的GPU内存。示例代码如下：
&span class="token comment" spellcheck="true"&#定义TensorFlow配置&/span&
config &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&ConfigProto&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#配置GPU内存分配方式&/span&
&span class="token comment" spellcheck="true"&#config.gpu_options.allow_growth = True&/span&
&span class="token comment" spellcheck="true"&#config.gpu_options.per_process_gpu_memory_fraction = 0.4&/span&
session &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Session&span class="token punctuation"&(&/span&config&span class="token operator"&=&/span&config&span class="token punctuation"&,&/span& &span class="token punctuation"&.&/span&&span class="token punctuation"&.&/span&&span class="token punctuation"&.&/span&&span class="token punctuation"&)&/span&
1234567891011
&span class="token comment" spellcheck="true"&#定义TensorFlow配置&/span&&config &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&ConfigProto&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&&&span class="token comment" spellcheck="true"&#配置GPU内存分配方式&/span&&&span class="token comment" spellcheck="true"&#config.gpu_options.allow_growth = True&/span&&&span class="token comment" spellcheck="true"&#config.gpu_options.per_process_gpu_memory_fraction = 0.4&/span&&session &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&Session&span class="token punctuation"&(&/span&config&span class="token operator"&=&/span&config&span class="token punctuation"&,&/span& &span class="token punctuation"&.&/span&&span class="token punctuation"&.&/span&&span class="token punctuation"&.&/span&&span class="token punctuation"&)&/span&
TensorFlow与HDFS集成使用
HDFS是一个高度容错性的分布式系统，能提供高吞吐量的数据访问，非常适合大规模数据集上的应用。TensorFlow与HDFS集成示例如下：
&span class="token comment" spellcheck="true"&#配置JAVA和HADOOP环境变量&/span&
source $HADOOP_HOME&span class="token operator"&/&/span&libexec&span class="token operator"&/&/span&hadoop&span class="token operator"&-&/span&config&span class="token punctuation"&.&/span&sh
export LD_LIBRARY_PATH&span class="token operator"&=&/span&$LD_LIBRARY_PATH&span class="token punctuation"&:&/span&$JAVA_HOME&span class="token operator"&/&/span&jre&span class="token operator"&/&/span&lib&span class="token operator"&/&/span&amd64&span class="token operator"&/&/span&server
&span class="token comment" spellcheck="true"&#执行TensorFlow运行模型&/span&
CLASSPATH&span class="token operator"&=&/span&$&span class="token punctuation"&(&/span&$HADOOP_HDFS_HOME&span class="token operator"&/&/span&bin&span class="token operator"&/&/span&hadoop classpath &span class="token operator"&-&/span&&span class="token operator"&-&/span&glob&span class="token punctuation"&)&/span& python tensorflow_model&span class="token punctuation"&.&/span&py
&span class="token comment" spellcheck="true"&#在TensorFlow模型中定义文件的读取队列&/span&
filename_queue &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&string_input_producer&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&&span class="token string"&"hdfs://namenode:8020/path/to/file1.csv"&/span&&span class="token punctuation"&,&/span& &span class="token string"&"hdfs://namenode:8020/path/to/file2.csv"&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#从文件中读取一行数据，value为所对应的行数据&/span&
reader &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&TextLineReader&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&
key&span class="token punctuation"&,&/span& value &span class="token operator"&=&/span& reader&span class="token punctuation"&.&/span&read&span class="token punctuation"&(&/span&filename_queue&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&# 把读取到的value值解码成特征向量，record_defaults定义解码格式及对应的数据类型&/span&
record_defaults &span class="token operator"&=&/span& &span class="token punctuation"&[&/span&&span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&]&/span&
col1&span class="token punctuation"&,&/span& col2&span class="token punctuation"&,&/span& col3&span class="token punctuation"&,&/span& col4&span class="token punctuation"&,&/span& col5 &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&decode_csv&span class="token punctuation"&(&/span&value&span class="token punctuation"&,&/span& record_defaults&span class="token operator"&=&/span&record_defaults&span class="token punctuation"&)&/span&
features &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&pack&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&col1&span class="token punctuation"&,&/span& col2&span class="token punctuation"&,&/span& col3&span class="token punctuation"&,&/span& col4&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&
&span class="token keyword"&with&/span& tf&span class="token punctuation"&.&/span&Session&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span& &span class="token keyword"&as&/span& sess&span class="token punctuation"&:&/span&
&span class="token comment" spellcheck="true"&# 定义同步对象，并启动相应线程把HDFS文件名插入到队列&/span&
coord &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&Coordinator&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&
threads &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&start_queue_runners&span class="token punctuation"&(&/span&coord&span class="token operator"&=&/span&coord&span class="token punctuation"&)&/span&
&span class="token keyword"&for&/span& i &span class="token keyword"&in&/span& range&span class="token punctuation"&(&/span&&span class="token number"&1200&/span&&span class="token punctuation"&)&/span&&span class="token punctuation"&:&/span&
&span class="token comment" spellcheck="true"&# 从文件队列中读取一行数据&/span&
example&span class="token punctuation"&,&/span& label &span class="token operator"&=&/span& sess&span class="token punctuation"&.&/span&run&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&features&span class="token punctuation"&,&/span& col5&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#请求停止队列的相关线程（包括进队及出队线程）&/span&
coord&span class="token punctuation"&.&/span&request_stop&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&
&span class="token comment" spellcheck="true"&#等待队列中相关线程结束（包括进队及出队线程）&/span&
coord&span class="token punctuation"&.&/span&join&span class="token punctuation"&(&/span&threads&span class="token punctuation"&)&/span&
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162
&span class="token comment" spellcheck="true"&#配置JAVA和HADOOP环境变量&/span&&source $HADOOP_HOME&span class="token operator"&/&/span&libexec&span class="token operator"&/&/span&hadoop&span class="token operator"&-&/span&config&span class="token punctuation"&.&/span&sh&export LD_LIBRARY_PATH&span class="token operator"&=&/span&$LD_LIBRARY_PATH&span class="token punctuation"&:&/span&$JAVA_HOME&span class="token operator"&/&/span&jre&span class="token operator"&/&/span&lib&span class="token operator"&/&/span&amd64&span class="token operator"&/&/span&server&&span class="token comment" spellcheck="true"&#执行TensorFlow运行模型&/span&&CLASSPATH&span class="token operator"&=&/span&$&span class="token punctuation"&(&/span&$HADOOP_HDFS_HOME&span class="token operator"&/&/span&bin&span class="token operator"&/&/span&hadoop classpath &span class="token operator"&-&/span&&span class="token operator"&-&/span&glob&span class="token punctuation"&)&/span& python tensorflow_model&span class="token punctuation"&.&/span&py&&&&span class="token comment" spellcheck="true"&#在TensorFlow模型中定义文件的读取队列&/span&&filename_queue &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&string_input_producer&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&&span class="token string"&"hdfs://namenode:8020/path/to/file1.csv"&/span&&span class="token punctuation"&,&/span& &span class="token string"&"hdfs://namenode:8020/path/to/file2.csv"&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&#从文件中读取一行数据，value为所对应的行数据&/span&&reader &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&TextLineReader&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&&key&span class="token punctuation"&,&/span& value &span class="token operator"&=&/span& reader&span class="token punctuation"&.&/span&read&span class="token punctuation"&(&/span&filename_queue&span class="token punctuation"&)&/span&&&&&span class="token comment" spellcheck="true"&# 把读取到的value值解码成特征向量，record_defaults定义解码格式及对应的数据类型&/span&&record_defaults &span class="token operator"&=&/span& &span class="token punctuation"&[&/span&&span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&,&/span& &span class="token punctuation"&[&/span&&span class="token number"&1&/span&&span class="token punctuation"&]&/span&&span class="token punctuation"&]&/span&&col1&span class="token punctuation"&,&/span& col2&span class="token punctuation"&,&/span& col3&span class="token punctuation"&,&/span& col4&span class="token punctuation"&,&/span& col5 &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&decode_csv&span class="token punctuation"&(&/span&value&span class="token punctuation"&,&/span& record_defaults&span class="token operator"&=&/span&record_defaults&span class="token punctuation"&)&/span&&features &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&pack&span class="token punctuation"&(&/span&&span class="token punctuation"&[&/span&col1&span class="token punctuation"&,&/span& col2&span class="token punctuation"&,&/span& col3&span class="token punctuation"&,&/span& col4&span class="token punctuation"&]&/span&&span class="token punctuation"&)&/span&&&&&span class="token keyword"&with&/span& tf&span class="token punctuation"&.&/span&Session&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span& &span class="token keyword"&as&/span& sess&span class="token punctuation"&:&/span&&&&&span class="token comment" spellcheck="true"&# 定义同步对象，并启动相应线程把HDFS文件名插入到队列&/span&&&&coord &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&Coordinator&span class="token punctuation"&(&/span&&span class="token punctuation"&)&/span&&&&threads &span class="token operator"&=&/span& tf&span class="token punctuation"&.&/span&train&span class="token punctuation"&.&/span&start_queue_runners&span class="token punctuation"&(&/span&coord&span class="token operator"&=&/span&coord&span class="token punctuation"&)&/span&&&&&&&span }