tensorflow源碼安裝

主要參考：https://www.tensorflow.org/install/install_sources#ConfigureInstallation


卸載tensorflow
?? ?sudo pip uninstall tensorflow

安裝git

??? 安裝git時記得先安裝，后更新系統
??? sudo apt install git



安裝jdk8:
?? ?my@ubuntu:~$ java

?? ?my@ubuntu:~$ sudo add-apt-repository ppa:openjdk/ppa

?? ?my@ubuntu:~$ sudo add-apt-repository ppa:openjdk-r/ppa
?
?? ?my@ubuntu:~$ sudo apt-get update

?? ?my@ubuntu:~$ sudo apt-get install openjdk-8-jdk

?? ?my@ubuntu:~$ sudo apt-get install openjdk-8-jre openjdk-8-jdk-headlesss
?? ?
?? ?my@ubuntu:~$ java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

Install Bazel：
?? ?sudo apt-get update && sudo apt-get install bazel
?? ?sudo apt-get upgrade bazel
?? ?bazel version

一、安裝virtualenv:
?? ?my@ubuntu:~$ sudo apt-get install python-virtualenv

??? 在virtualenv中創建tensorfloe環境

?? ?my@ubuntu:~$ virtualenv --system-site-packages ~/tensorflow

??? 激活tensorflow的virtualenv環境

?? ?my@ubuntu:~$ source ~/tensorflow/bin/activate

??? 為虛擬環境創建快捷方式：
?? ?sudo printf '\nalias tensorflow="source ~/tensorflow/bin/activate"'>>~/.bashrc
??? 讓其生效：
?? ?source ~/.bashrc

二、在env環境中
?? ?
?? ?git clone https://github.com/tensorflow/tensorflow

?? ?sudo apt-get install python-numpy python-dev python-pip python-wheel

因為我的vm中沒有gpu，省去gpu的配置環節：

?? ?sudo apt-get install libcupti-dev

?? ? sudo pip install six numpy wheel


下面這段摘自tensorflow的官方文檔：
?? ?$cd tensorflow? # cd to the top-level directory created
?? ?$ ./configure
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python2.7
Found possible Python library paths:
? /usr/local/lib/python2.7/dist-packages
? /usr/lib/python2.7/dist-packages
Please input the desired Python library path to use.? Default is [/usr/lib/python2.7/dist-packages]

Using python library path: /usr/local/lib/python2.7/dist-packages
Do you wish to build TensorFlow with MKL support? [y/N]
No MKL support will be enabled for TensorFlow
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
Do you wish to use jemalloc as the malloc implementation? [Y/n]
jemalloc enabled
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N]
No Hadoop File System support will be enabled for TensorFlow
Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
No XLA support will be enabled for TensorFlow
Do you wish to build TensorFlow with VERBS support? [y/N]
No VERBS support will be enabled for TensorFlow
Do you wish to build TensorFlow with OpenCL support? [y/N]
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with CUDA support? [y/N] Y
CUDA support will be enabled for TensorFlow
Do you want to use clang as CUDA compiler? [y/N]
nvcc will be used as CUDA compiler
Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 8.0]: 8.0
Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 6.0]: 6
Please specify the location where cuDNN 6 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size.
[Default is: "3.5,5.2"]: 3.0
Do you wish to build TensorFlow with MPI support? [y/N]
MPI support will not be enabled for TensorFlow
Configuration finished
這里是非常重要的編譯配置：可以參考http://blog.csdn.net/nicholas_wong/article/details/70215127，但還要結合自己的機器報的警告來配置，
我的機器報如下警告：
/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. ?
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. ?
W tensorflow/core/platform，那我安裝的時候就配置sse4.1和sse4.2的選項，剛開始直接把人家的復制過來用，安裝完后報錯，后來自己配置了一下，安裝成功。而且上面的警告有時候是不會出現的，
我在安裝SyntaxNet wrapper 地址及安裝：https://github.com/short-edition/syntaxnet-wrapper時報錯，導致測試走不下去，只能消除警告，
?? ?sudo bazel build -c opt? --copt=-msse4.1 --copt=-msse4.2? //tensorflow/tools/pip_package:build_pip_package
這個編譯時間超長，要有耐心。

上述命令會生成一個叫做build_pip_package的腳本，按照如下命令運行這個腳本，在/tmp/tensorflow_pkg文件夾中創建pip的安裝包：

?? ?bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
?? ?cd? /tmp/tensorflow_pkg
?? ?ls
會發現一個whl的文件：tensorflow-1.3.0-cp27-cp27mu-linux_x86_64.whl，這是編譯生成的。
下面進行安裝：

?? ?sudo? pip install /tmp/tensorflow_pkg/tensorflow-1.3.0-cp27-cp27mu-linux_x86_64.whl
安裝完成后進行測試：注意測試的時候不要到安裝目錄下測試。
$ python

Enter the following short program inside the python interactive shell:

# Python
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))

If the system outputs the following, then you are ready to begin writing TensorFlow programs:

Hello, TensorFlow!

如果不報sse4.1和sse4.2錯誤，有可能安裝成功。

安裝syntaxnet:
?? ?前提是tensorflow已經安裝好，
?? ?git clone --recursive https://github.com/tensorflow/models.git

?? ?cd models/syntaxnet/tensorflow
配置選項：
?? ?$ ./configure

(tensorflow) my@ubuntu:~/models/syntaxnet/tensorflow$ cd ..

?? ?(tensorflow) my@ubuntu:~/models/syntaxnet$ bazel test syntaxnet/... util/utf8/...

?? ?(tensorflow) my@ubuntu:~/models/syntaxnet$ chmod +x? syntaxnet/demo.sh

7、pip安裝 asciitree、mock

?? ?(tensorflow) my@ubuntu:~/models/syntaxnet$? pip install asciitree

?? ?(tensorflow) my@ubuntu:~/models/syntaxnet$ pip install mock


??? 測試一個句子：

?? ?(tensorflow) my@ubuntu:~/models/syntaxnet$ sudo? echo 'Bob brought the pizza to Alice.' | syntaxnet/demo.sh
?? ?

測試中文：?? ?

MODEL_DIRECTORY為解壓的中文模型的位置，一種方式是從http://download.tensorflow.org/models/parsey_universal/Chinese.zip下載
然后解壓到/home/py/models/syntaxnet/syntaxnet/models/parsey_universal，
另外一種方式是安裝 A lightweight SyntaxNet wrapper 地址及安裝：https://github.com/short-edition/syntaxnet-wrapper，它會自動安裝語言模型

測試：
py@ubuntu:~/models/syntaxnet$ MODEL_DIRECTORY=/home/py/models/syntaxnet/syntaxnet/models/parsey_universal/Chinese

py@ubuntu:~/models/syntaxnet$ echo '然而,中國經歷了30多年的改革開放' | syntaxnet/models/parsey_universal/tokenize_zh.sh $MODEL_DIRECTORY | syntaxnet/models/parsey_universal/parse.sh $MODEL_DIRECTORY


安裝SyntaxNet Wrapper

直接復制文檔了：

A lightweight SyntaxNet wrapper

The wrapper allows generic use of SyntaxNet in python. It provides interfaces for morphological analyse, pos tagging and dependency resolution along with optional formatting tool.

The wrapper does not intend to make any assumptions on the use of SyntaxNet, that's why it provides a simple interface and the raw output as default.

Disclaimer : Has been inspired from another wrapper but we did not want a server based wrapper.
Installation

We assume here that you have SyntaxNet installed and working properly on your workstation. If not, please refer to SyntaxNet official page

The wrapper has been tester on Ubuntu 14.04 Trusty and 16.04 Xenial.

(virtualenv)$ git clone https://github.com/short-edition/syntaxnet-wrapper.git
(virtualenv)$ cd syntaxnet-wrapper
(virtualenv)/syntaxnet-wrapper$ pip install -r requirements.txt
(virtualenv)/syntaxnet-wrapper$ vim syntaxnet_wrapper/config.yml
syntaxnet:
? ROOT_DIR: /home/user/workspace/syntactic_parser/tensorflow_models/syntaxnet
? PARSER_EVAL: bazel-bin/syntaxnet/parser_eval
? CONTEXT: syntaxnet/models/parsey_universal/context.pbtxt
? MODEL: syntaxnet/models/parsey_universal

(virtualenv)/syntaxnet-wrapper$ python -m unittest discover syntaxnet_wrapper
(virtualenv)/syntaxnet-wrapper$ pip install .

這一句比較重要：創建 /usr/share/syntaxnet_wrapper_config.yml，內容就是
syntaxnet:
? ROOT_DIR: /home/py/models/syntaxnet
? PARSER_EVAL: bazel-bin/syntaxnet/parser_eval
? CONTEXT: syntaxnet/models/parsey_universal/context.pbtxt
? MODEL: syntaxnet/models/parsey_universal


You should be able to use the wrapper from now. Instead of creating config.yml in the current folder, you can also expose it though the environment variable SYNTAXNET_WRAPPER_CONFIG or copying it in /usr/share/syntaxnet_wrapper_config.yml

You can also use the Dockerfile provided that will install SyntaxNet itself and the wrapper
How to use this wrapper

Two mode

You can use the wrapper in two modes, embodied in two different classes with the same interface

??? SyntaxNetWrapperSubprocess, a python implementation of demo.sh shell script provided in SyntaxNet. It starts new subprocesses at each call.
??? SyntaxNetWrapper, using wrapper's syntaxnet python implementation. Have the advantage to be faster and more memory efficient than the version with subprocesses. However, we are experience some trouble with it. See Well-known issues

The interface

The wrapper is expecting unicode text compatible with utf-8 format. The interface is the same for both classes :

??? morpho_sentence, make morphological analyse for a single sentence

??? morpho_sentences, make morphological analyse for a sentences list

??? tag_sentence, make pos tagging for a single sentence

??? tag_sentences, make pos tagging for a sentences list

??? parse_sentence, make dependency parsing for a single sentence

??? parse_sentences, make dependency parsing for a sentences list

??? transform_morpho, transform_tag and transform_dependency format the outputs in a more readable form. Deleting unfilled field.

例子：

>>> from syntaxnet_wrapper.wrapper import SyntaxNetWrapper
>>> sn_wrapper = SyntaxNetWrapper()
>>> dependency_output = sn_wrapper.parse_sentence(u"Bob brought a pizza to Alice")
>>> print dependency_output
u'1\tBob\t_\tPROPN\tNNP\tNumber=Sing|fPOS=PROPN++NNP\t2\tnsubj\t_\t_\n2\tbrought\t_\tVERB\tVBD\tMood=Ind|Tense=Past|VerbForm=Fin|fPOS=VERB++VBD\t0\tROOT\t_\t_\n3\ta\t_\tDET\tDT\tDefinite=Ind|PronType=Art|fPOS=DET++DT\t4\tdet\t_\t_\n4\tpizza\t_\tNOUN\tNN\tNumber=Sing|fPOS=NOUN++NN\t2\tdobj\t_\t_\n5\tto\t_\tADP\tIN\tfPOS=ADP++IN\t6\tcase\t_\t_\n6\tAlice\t_\tPROPN\tNNP\tNumber=Sing|fPOS=PROPN++NNP\t4\tnmod\t_\t_\n\n'

Use of different language

The wrapper use by default english model but you can use every "Parsey Universal" released by Google. You just need to pass the name of the model as a constructor's argument. The wrapper will then automatically download the model and use it.

>>> sn_wrapper = SyntaxNetWrapper(language='French')