Compiling Tensorflow with custom Clang + LibC ++ (instead of stdlibC ++)
I am trying to compile tensorflow using the clang / llvm custom binding and use clang native libc ++ (instead of borrowing Gcc stdlibc ++).
It looks like bazel plain assumes every clang will use the Gcc libraries because I am getting these errors:
$ bazel build --cxxopt=-std=c++11 --cxxopt=-stdlib=libc++ tensorflow:libtensorflow.so
INFO: Found 1 target...
INFO: From Compiling
external/protobuf/src/google/protobuf/compiler/js/embed.cc [for host]:
external/protobuf/src/google/protobuf/compiler/js/embed.cc:37:12:
warning: unused variable 'output_file' [-Wunused-const-variable]
const char output_file[] = "well_known_types_embed.cc";
^
1 warning generated.
ERROR: /home/hbucher/.cache/bazel/_bazel_hbucher/ad427c7fddd5b68de5e1cfaa7cd8c8cc/external/com_googlesource_code_re2/BUILD:11:1: undeclared inclusion(s) in rule '@com_googlesource_code_re2//:re2':
this rule is missing dependency declarations for the following files included by 'external/com_googlesource_code_re2/re2/bitstate.cc':
'/home/hbucher/install/include/c++/v1/stddef.h'
'/home/hbucher/install/include/c++/v1/__config'
I tried to hack tools / cpp / CROSSTOOL inside bazel as some posts suggested adding the line
cxx_builtin_include_directory: "/home/hbucher/install/include/c++/v1"
but to no avail, it doesn't seem to make any difference.
Then I tried to follow the Basel tutorial to create a custom toolchain. The text doesn't help much because they actually write a cross tool, and what I'm trying to do is tweak the existing host rules and somehow the basel seems to cancel every attempt, I'm trying to tweak its settings.
I've gotten to what is currently in my github repository https://github.com/HFTrader/BazelCustomToolchain
However, it doesn't compile and I can't even figure out how to start debugging this post.
$ bazel build --crosstool_top=@hbclang//:toolchain tensorflow:libtensorflow.so
.....................
ERROR: The crosstool_top you specified was resolved to
'@hbclang//:toolchain', which does not contain a CROSSTOOL file. You can
use a crosstool from the depot by specifying its label.
INFO: Elapsed time: 2.216s
I have attached these lines to my tensorflow / WORKSPACE
new_local_repository(
name="hbclang",
path="/home/hbucher/BazelCustomToolchain",
build_file = "/home/hbucher/BazelCustomToolchain/BUILD",
)
I asked this question on bazel google groups but redirected me to stackoverflow. At this point, I'm going to give up.
Has anyone tried this or am I breaking here?
Thank.
solvable. Not as intended, but it works for me.
export INSTALL_DIR="$HOME/install"
export CC=$INSTALL_DIR/bin/clang
export CXX=$INSTALL_DIR/bin/clang++
export CXXFLAGS="-stdlib=libc++ -L$INSTALL_DIR/lib"
export LDFLAGS="-L$INSTALL_DIR/lib -lm -lrt"
export LD_LIBRARY_PATH="/usr/lib:/lib/x86_64-linux-gnu/:$INSTALL_DIR/lib"
git clone https://github.com/tensorflow/tensorflow.git tensorflow-github
cd tensorflow-github
mkdir build-tmp && cd build-tmp
cmake ../tensorflow/contrib/cmake/
make -j4
Easy as 1-2-3 with cmake
So there are too many small questions and questions in this question, I will give short answers to all of them and please ask more about the ones you want to know more about.
Bazel doesn't assume libstdc ++, it just does what Crosstool says. As you already figured out, writing Crosstool leaves even outstanding engineers looking at the screen in despair, Bazel tries to auto-generate Crosstool for you. This autogeneration limits it, and so far nobody has asked for libC ++ instead of libstdC ++, so we haven't taught it to detect it. The corresponding line in cc_configure.bzl .
Reg. tools / cpp / CROSSTOOL, do you mean the file in the Bazel sources? This file is only used to download Bazel when a new version is released. Usually Bazel uses the autogenerated crossover device mentioned above.
I need more information on what you exactly mean by "bazel seems to cancel every attempt I try to tweak its settings".
Your github repo is a good starting point, you are on the right track there.
And for a local repository, you need to use the local_repository rule instead of new_repository_rule. The former is depending on existing Bazel projects, the latter is for projects that do not have BUILD files (for example, depending on the build of the make project, etc.). Relevant section in the documentation .
source to share