Work around for "boringssl.googlesource.com/boringssl" ? about serving HOT 34 CLOSED

@doubler @kirilg Thanks very much, using doubler's repo of boringssl I can now build tensorflow serving successfully. Closing this issue as problem solved.

from serving.

@liujiabing I solve this by using bazel build -c opt

from serving.

Thanks for bringing this to our attention.
Currently our only dependency on gRPC (which in turn depends on BoringSSL) is in our examples/tutorials. If you don't care about the examples/tutorials and just want to compile the core code, you should still be able to compile the rest of Tensorflow Serving with:
bazel build -- tensorflow_serving/... -tensorflow_serving/example/...

If you do want to use the tutorials or wish to build your own frontend using gRPC, you can substitute with another SSL library in our WORKSPACE file. Unfortunately I'm not aware of other SSL libraries that have bazel support built in.

I'll follow up with the BoringSSL team to see if they have a workaround for this and see if we can come up with solution.

from serving.

I have the same problem.
I forked boringssl from googlesource to github, and replaced url from https://boringssl.googlesource.com/boringssl to https://github.com/doubler/boringssl.git in .gitmodules
and changed the WORKSPACE file from

git_repository(
    name = "boringssl_git",
    commit = "436432d849b83ab90f18773e4ae1c7a8f148f48d",
    init_submodules = True,
    remote = "https://github.com/mdsteele/boringssl-bazel.git",
)

to

git_repository(
    name = "boringssl_git",
    commit = "8e371d413fe178b088587d4dc303d542c3857e20",
    init_submodules = True,
    remote = "https://github.com/doubler/boringssl-bazel.git",
)

but I still have build errors.

The output log is:

INFO: Found 110 targets...
ERROR: missing input file '@boringssl_git//:src/crypto/x509/charmap.h'.
ERROR: /Users/Double/Work/ml/serving/tensorflow_serving/session_bundle/example/BUILD:11:1: Creating runfiles tree bazel-out/local_darwin-fastbuild/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles failed: build-runfiles failed: error executing command /private/var/tmp/_bazel_Double/472342228f4d91901cb07f2a57ad5f6d/serving/_bin/build-runfiles ... (remaining 2 argument(s) skipped): com.google.devtools.build.lib.shell.AbnormalTerminationException: Process terminated by signal 15.
ERROR: /private/var/tmp/_bazel_Double/472342228f4d91901cb07f2a57ad5f6d/external/boringssl_git/BUILD:32:1: @boringssl_git//:crypto: missing input file '@boringssl_git//:src/crypto/x509/charmap.h'.
ERROR: /private/var/tmp/_bazel_Double/472342228f4d91901cb07f2a57ad5f6d/external/boringssl_git/BUILD:32:1 1 input file(s) do not exist.
INFO: Elapsed time: 6.824s, Critical Path: 1.04s

I don't know what is wrong.

from serving.

Your boringssl-bazel repo (https://github.com/doubler/boringssl-bazel.git) should include your regular boringssl clone (https://github.com/doubler/boringssl.git) as a submodule so it actually picks up the source. See "src @ 82aa28f" from https://github.com/mdsteele/boringssl-bazel.

from serving.

@kirilg Thanks for your help.
Now I can use the boringssl dependency, but still have compile error.

bazel build tensorflow_serving/...
INFO: Found 110 targets...
ERROR: /private/var/tmp/_bazel_Double/472342228f4d91901cb07f2a57ad5f6d/external/boringssl_git/BUILD:32:1: C++ compilation of rule '@boringssl_git//:crypto' failed: osx_gcc_wrapper.sh failed: error executing command external/bazel_tools/tools/cpp/osx_gcc_wrapper.sh '-D_FORTIFY_SOURCE=1' -fstack-protector -fcolor-diagnostics -Wall -Wthread-safety -Wself-assign -fno-omit-frame-pointer -iquote external/boringssl_git ... (remaining 26 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:5:1: error: unknown directive
.hidden aesni_gcm_encrypt
^
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:6:1: error: unknown directive
.type aesni_gcm_encrypt,@function
^
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:10:1: error: unknown directive
.size aesni_gcm_encrypt,.-aesni_gcm_encrypt
^
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:13:1: error: unknown directive
.hidden aesni_gcm_decrypt
^
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:14:1: error: unknown directive
.type aesni_gcm_decrypt,@function
^
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:18:1: error: unknown directive
.size aesni_gcm_decrypt,.-aesni_gcm_decrypt
^
INFO: Elapsed time: 13.608s, Critical Path: 9.07s

My compiler is

gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.2.0
Thread model: posix

from serving.

Yeah, we currently fully support Linux but don't have full support for Mac yet, see #1

There's a workaround github repo for BoringSSL that has the Mac generated files - https://github.com/ctiller/boringssl-bazel that you can try cloning (in your boringssl-bazel repo). Don't forget to relink the submodule to your github clone of boringssl and not the default one that it uses pointing to googlesource.

We're working on a longer-term solution for this which will be more robust, but this should be good enough to get the tutorials running.

from serving.

Well, actually I cloned mdsteele/boringssl-bazel, and modified WORKSPACE file
new_http_archive(
name = "gmock_archive",
#url = "https://googlemock.googlecode.com/files/gmock-1.7.0.zip",
url = "https://github.com/peter-wangxu/gmock/archive/1.7.0.zip",
#sha256 = "26fcbb5925b74ad5fc8c26b0495dfc96353f4d553492eb97e85a8a6d2f43095b",
sha256 = "9b0018413f4222b8ee5454a431918c324d010062eecb078677b6897d0c76bc42",
build_file = "tensorflow/google/protobuf/gmock.BUILD",
)
#gmock also need modification, and here is ok
git_repository(
name = "boringssl_git",
commit = "436432d849b83ab90f18773e4ae1c7a8f148f48d",
init_submodules = True,
#remote = "https://github.com/mdsteele/boringssl-bazel.git",
remote = "https://github.com/chenghuige/boringssl-bazel.git",
)

#but for boringssl seems modify here is not enough

bazel build tensorflow_serving/...
ERROR: /home/gezi2/other/serving.local/WORKSPACE:34:1: no such package '@boringssl_git//': /home/gezi2/.cache/bazel/_bazel_gezi/d155f30bfec905b1a4509ecabf4184e5/external/boringssl_git/.git/objects/pack (Directory not empty) and referenced by '//external:libssl'.
ERROR: Loading failed; build aborted.
INFO: Elapsed time: 3.375s

from serving.

@kirilg OK, I changed the platform to CentOS 7 and then successed to build with pre-install zlib-devel which not mentioned in the setup page.
@chenghuige maybe you should change the commit from "436432d849b83ab90f18773e4ae1c7a8f148f48d" to "b8082f0bd36870804a434dcaddd0c8fd54cb8463". Or you can just use my repo(https://github.com/doubler/boringssl-bazel.git db0729054d5964feab9e60089ba2d06a181e78b1)

from serving.

@doubler Thanks, but seems too difficult for me, I find some ohter blocks, like
name = "jpeg_archive",
url = "http://www.ijg.org/files/jpegsrc.v9a.tar.gz",
Can not download this, I can download from ie using vpn, but bazel need to download it from terminal command line...

It would be appreciated if bazel can build without network need, like pre download all the needed packages.

from serving.

@doubler the setup page mentions zlib1g-dev which I guess was not found and thus not installed on CentOS, so you needed zlib-devel. Glad it now compiles!

@chenghuige by pre-download, do you mean list them as submodules in the project and download with git clone --recurse-submodules? I think there could be many such git_repository deps in different workspace files (ours and our dependencies going forward). git_repository should behave in the same way as listing that as a submodule and pulling in the code when cloning, but maintains a cleaner workspace. If bazel can't reach that url, it's likely that git clone also wouldn't be able to.

from serving.

@chenghuige Just like the gmock, you can change the jpeg_archive url in WORKSPACE. Or use the new_local_repository instead of new_http_archive. My WORKSPACE conf:

#new_http_archive(
#  name = "jpeg_archive",
#  url = "http://www.ijg.org/files/jpegsrc.v9a.tar.gz",
#  sha256 = "3a753ea48d917945dd54a2d97de388aa06ca2eb1066cbfdc6652036349fe05a7",
#  build_file = "tensorflow/jpeg.BUILD",
#)
new_local_repository(
    name = "jpeg_archive",
    path = "/export/liujia/Download/jpeg",
    build_file = "tensorflow/jpeg.BUILD",
)

from serving.

@doubler Thanks! I tried new_local_repository like you posted, but I got this error

bazel build -- tensorflow_serving/... -tensorflow_serving/example/...
INFO: Waiting for response from Bazel server (pid 41489)...
ERROR: /home/gezi2/.cache/bazel/_bazel_gezi/16e2e58775bb9b26aa372e99a83ccfe4/external/jpeg_archive/BUILD:67:1: in genrule rule @jpeg_archive//:configure: cycle in dependency graph:
//tensorflow_serving/batching/test_util:puppet_batch_scheduler
//tensorflow_serving/batching:batch_scheduler
@tf//tensorflow/core:tensorflow
@tf//tensorflow/core:tensorflow_opensource
@tf//tensorflow/core:gpu_runtime
@tf//tensorflow/core:framework_internal
@tf//tensorflow/core:lib_internal
@tf//tensorflow/core/platform/default/build_config:platformlib
@jpeg_archive//:jpeg

• @jpeg_archive//:configure [self-edge].
  ERROR: Loading of target '//tensorflow_serving/batching/test_util:puppet_batch_scheduler' failed; build aborted.
  ERROR: Loading failed; build aborted.
  INFO: Elapsed time: 8.582s

from serving.

@doubler where did you find "boringssl_git"? I cannot find this word in my WORKSPACE file. I am using the latest tensorflow. Thanks

from serving.

It's in Tensorflow Serving's WORKSPACE file here: https://github.com/tensorflow/serving/blob/master/WORKSPACE#L17

from serving.

@kirilg OMG...I built the latest tensorflow instead of this "server" repo and I came across this issue. It seems this issue cannot solve the problem from the latest tensorflow...bazel deletes grpc.git in ~/.cache every time I modified the path where it downloads boringssl in grpc's .gitmodule.

from serving.

This installation is very painful. Anyone run this on ubuntu 15.04? Get the same error with borringssl.

from serving.

This issue is not resolved, why was it closed?

external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:5:1: error: unknown directive
.hidden aesni_gcm_encrypt
^
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:6:1: error: unknown directive
.type aesni_gcm_encrypt,@function
^
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:10:1: error: unknown directive
.size aesni_gcm_encrypt,.-aesni_gcm_encrypt
^
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:13:1: error: unknown directive
.hidden aesni_gcm_decrypt
^
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:14:1: error: unknown directive
.type aesni_gcm_decrypt,@function
^
external/boringssl_git/linux-x86_64/crypto/modes/aesni-gcm-x86_64.S:18:1: error: unknown directive
.size aesni_gcm_decrypt,.-aesni_gcm_decrypt
^
INFO: Elapsed time: 165.311s, Critical Path: 23.39s

from serving.

@Cospel Sorry for the pain. What was the exact error you got with Boringssl?

@b I think your problem is a different one. This issue was about googlesource not being accessible (and subsequent workarounds to get a copy of boringssl). I've never encountered this, could you please open a new issue with more information? what command were you using, what platform, is this using the latest TensorFlow serving with no local changes?

By the way, if interested, you can disable grpc encryption entirely such that compilation doesn't require SSL at all by replacing the following BUILD file deps in tensorflow_serving/example/BUILD:
"@grpc//:grpc++" --> "@grpc//:grpc++_unsecure"
"@grpc//:grpc" --> "@grpc//:grpc_unsecure"
(and optionally remove all entries below the "gRPC dependencies" line in the WORKSPACE file).

from serving.

I'm still getting an error. I'm on OSX El Capitan and have this change in my WORKSPACE file:

git_repository(
    name = "boringssl_git",
    commit = "32bfe16a53ad13523eadce08dd2f835bc2b0b52d",
    init_submodules = True,
    remote = "https://github.com/ctiller/boringssl-bazel",
)

Here's the error:

$ bazel build //tensorflow_serving/example:mnist_inference
INFO: Found 1 target...
ERROR: /Users/robertgrzesik/Documents/Development/TensorFlow/serving/tensorflow_serving/example/BUILD:53:1: Linking of rule '//tensorflow_serving/example:mnist_inference' failed: osx_gcc_wrapper.sh failed: error executing command external/bazel_tools/tools/cpp/osx_gcc_wrapper.sh -o bazel-out/local_darwin-fastbuild/bin/tensorflow_serving/example/mnist_inference ... (remaining 512 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
clang: warning: argument unused during compilation: '-pthread'
ld: warning: option -noall_load is obsolete and being ignored
ld: warning: option -noall_load is obsolete and being ignored
ld: warning: option -noall_load is obsolete and being ignored
ld: warning: option -noall_load is obsolete and being ignored
ld: warning: option -noall_load is obsolete and being ignored
ld: warning: option -noall_load is obsolete and being ignored

from serving.

Hi Robert, this seems to be a different issue related to compiling on OSX which is not yet supported, see #1. I see you opened another issue related to this problem, so we can continue the investigation there.

from serving.

@kirilg The issue is mentioned earlier in the thread, but I'll open another thread on OSX compilation problems.

from serving.

i use https://github.com/doubler/boringssl-bazel.git, but get this error:::

WARNING: /root/.cache/bazel/_bazel_root/8446906f776ab5df44471a95fb6148b5/external/grpc/BUILD:917:14: in includes attribute of cc_library rule @grpc//:grpc++: '.' resolves to 'external/grpc' not in 'third_party'. This will be an error in the future.
INFO: Found 155 targets...
ERROR: /root/.cache/bazel/_bazel_root/8446906f776ab5df44471a95fb6148b5/external/tf/tensorflow/contrib/linear_optimizer/BUILD:16:1: C++ compilation of rule '@tf//tensorflow/contrib/linear_optimizer:python/ops/_sdca_ops.so' failed: gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wl,-z,-relro,-z,now -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -iquote ... (remaining 39 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
In file included from external/tf/tensorflow/core/lib/gtl/array_slice.h:101:0,
from external/tf/tensorflow/core/lib/strings/str_util.h:22,
from external/tf/tensorflow/core/framework/op.h:28,
from external/tf/tensorflow/contrib/linear_optimizer/ops/sdca_ops.cc:16:
external/tf/tensorflow/core/lib/gtl/array_slice_internal.h:232:38: error: 'tensorflow::gtl::array_slice_internal::ArraySliceImplBase::ArraySliceImplBase' names constructor
external/tf/tensorflow/core/lib/gtl/array_slice_internal.h:252:32: error: 'tensorflow::gtl::array_slice_internal::ArraySliceImplBase::ArraySliceImplBase' names constructor
In file included from external/tf/tensorflow/core/lib/gtl/array_slice.h:102:0,
from external/tf/tensorflow/core/lib/strings/str_util.h:22,
from external/tf/tensorflow/core/framework/op.h:28,
from external/tf/tensorflow/contrib/linear_optimizer/ops/sdca_ops.cc:16:
external/tf/tensorflow/core/lib/gtl/inlined_vector.h: In member function 'void tensorflow::gtl::InlinedVector<T, N>::Destroy(T*, int)':
external/tf/tensorflow/core/lib/gtl/inlined_vector.h:383:10: error: 'is_trivially_destructible' is not a member of 'std'
external/tf/tensorflow/core/lib/gtl/inlined_vector.h:383:42: error: expected primary-expression before '>' token
external/tf/tensorflow/core/lib/gtl/inlined_vector.h:383:43: error: '::value' has not been declared
INFO: Elapsed time: 732.777s, Critical Path: 3.33s

from serving.

+1

from serving.

I encounter similar problem getting the https://archive.openswitch.net/gmock-1.7.0.zip, I resolved this problem by changing to a local path. But I got another ERROR saying:
@gmock_archive//:gtest: missing input file '@gmock_archive//:gmock-1.7.0/src/gmock-all.cc'.

from serving.

Hi Guys,
Trying to compile serving on mac, compilation issues, did anybody face following error.
tensorflow_serving/core/test_util/BUILD:52:1: C++ compilation of rule '//tensorflow_serving/core/test_util:fake_loader' failed: osx_cc_wrapper.sh failed: error executing command external/bazel_tools/tools/cpp/osx_cc_wrapper.sh '-D_FORTIFY_SOURCE=1' -fstack-protector -fcolor-diagnostics -Wall -Wthread-safety -Wself-assign -fno-omit-frame-pointer '-std=c++0x' -iquote . -iquote ... (remaining 101 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
In file included from tensorflow_serving/core/test_util/fake_loader.cc:16:
./tensorflow_serving/core/test_util/fake_loader.h:59:10: error: thread-local storage is not supported for the current target
static thread_local bool was_deleted_in_this_thread_;
^
tensorflow_serving/core/test_util/fake_loader.cc:24:1: error: thread-local storage is not supported for the current target
thread_local bool FakeLoader::was_deleted_in_this_thread_;
^
2 errors generated.
INFO: Elapsed time: 1583.096s, Critical Path: 1409.25s

from serving.

I also get the "boringssl" errors during bazel build of server.

C++ compilation of rule '@boringssl//:crypto' failed

Here's the detailed error log:

ERROR: ~/cache/def2095ca8f472068ce1fdce87d2ee7d/external/boringssl/BUILD:104:1: C++ compilation of rule '@boringssl//:ssl' failed: gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wl,-z,-relro,-z,now -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -MD ... (remaining 40 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
In file included from /usr/include/stdint.h:25:0,
                 from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/stdint.h:9,
                 from external/boringssl/src/include/openssl/base.h:60,
                 from external/boringssl/src/include/openssl/ssl.h:145,
                 from external/boringssl/src/ssl/tls_method.c:57:

I'm using CentOS Linux 7.
Hope to hear from people who have similar problems. Any solutions?

from serving.

@mogumogu2333 I have the same problem with you with same environment. Did you get some solutions?

from serving.

I found a workaround to the issue:
C++ compilation of rule '@boringssl//:crypto' failed`

Just adding '-O' to boringssl BUILD file will make it pass.

from serving.

@mogumogu2333 Thanks. It's a better idea.

from serving.

@liujiabing previously I post an issue about how to deploy tensorflow serving on centos 7. And I got some good solutions. You may have a look at it if you encounter other build issues issue

from serving.

@mogumogu2333 Thank you and I will.

from serving.

Hi, I encounter this problem by using bazel build -c opt tensorflow_serving/... command to build the tensorflow serving.
ERROR: /root/.cache/bazel/_bazel_root/53bfa30163750db6ab7cbcd6cc66ed01/external/boringssl/BUILD:91:1: C++ compilation of rule '@boringssl//:crypto' failed: gcc failed: error executing command /usr/local/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wl,-z,-relro,-z,now -B/usr/local/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object ... (remaining 47 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S: Assembler messages:
external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1002: Error: no such instruction: vbroadcasti128 .Lsigma(%rip),%ymm11' external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1003: Error: no such instruction:vbroadcasti128 (%rcx),%ymm3'
external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1004: Error: no such instruction: vbroadcasti128 16(%rcx),%ymm15' external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1005: Error: no such instruction:vbroadcasti128 (%r8),%ymm7'
external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1011: Error: suffix or operands invalid for vpshufd' external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1012: Error: suffix or operands invalid forvpshufd'
external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1014: Error: suffix or operands invalid for vpshufd' external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1016: Error: suffix or operands invalid forvpshufd'
external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1020: Error: suffix or operands invalid for vpshufd' external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1021: Error: suffix or operands invalid forvpshufd'
external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1023: Error: suffix or operands invalid for vpshufd' external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1025: Error: suffix or operands invalid forvpshufd'
external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1029: Error: suffix or operands invalid for vpshufd' external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1030: Error: suffix or operands invalid forvpshufd'
external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1032: Error: suffix or operands invalid for vpshufd' external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1034: Error: suffix or operands invalid forvpshufd'
external/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1038: Error: suffix or operands invalid for vpshuf

from serving.

@kirilg Thanks for all your response. Could you please let me know what version of TLS protocol version (i.e. 1.1 to 1.3) are we using in Tensorflow serving.

from serving.