lmb-freiburg / lmbspecialops Goto Github PK

when I compile lmbspecialops, there is always an error occurs
decode_flo_op.cc:65:15: error: ‘const class tensorflow::StringPiece’ has no member named ‘starts_with’ if (!data.starts_with("PIEH")) {
I try to trace back, and found that there is a 'starts_with' function in namespace protobuf/google but not in namespace tensorflow.
how to fix it?

My build is successful, and got this error when importing lmbspecialops:

In [2]: import lmbspecialops
---------------------------------------------------------------------------
NotFoundError                             Traceback (most recent call last)
<ipython-input-2-19097e982614> in <module>
----> 1 import lmbspecialops

/data/xiuye_data/lmbspecialops/python/lmbspecialops.py in <module>
     28     raise ValueError(
     29         'Cannot find lmbspecialops.so . Set the environment variable LMBSPECIALOPS_LIB to the path to lmbspecialops.so file')
---> 30 lmbspecialopslib = tf.load_op_library(_lib_path)
     31 print('Using {0}'.format(_lib_path), flush=True)
     32

~/.pyenv/versions/3.5.2/lib/python3.5/site-packages/tensorflow/python/framework/load_library.py in load_op_library(library_filename)
     54   """
     55   with errors_impl.raise_exception_on_not_ok_status() as status:
---> 56     lib_handle = py_tf.TF_LoadLibrary(library_filename, status)
     57
     58   op_list_str = py_tf.TF_GetOpList(lib_handle)

~/.pyenv/versions/3.5.2/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py in __exit__(self, type_arg, value_arg, traceback_arg)
    471             None, None,
    472             compat.as_text(c_api.TF_Message(self.status.status)),
--> 473             c_api.TF_GetCode(self.status.status))
    474     # Delete the underlying status object from memory otherwise it stays alive
    475     # as there is a reference to status from this from the traceback due to

NotFoundError: /data/xiuye_data/lmbspecialops/build/lib/lmbspecialops.so: undefined symbol: _ZTIN10tensorflow8OpKernelE

I use Python 3.5.2, install TensorFlow 1.4.0 with CUDA 8.0 and CUDNN 6.0.
The cmake version I use is 3.11.4.

I intend to run the FlowNet3.0, so I use the eccv18 branch.

What should I do?
Thank you!

Is someone ever tries to build on MacOS, I had some OpenMP issues, I was able to solve these with:

brew install llvm --with-clang
CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ LDFLAGS=-L/usr/local/opt/llvm/lib/ cmake -DBUILD_WITH_CUDA=OFF ..

Refernece for a siimliar issue in another project:
src-d/kmcuda#6
https://github.com/src-d/kmcuda#macos

Hello, thanks for you work. Please, can you help me, I need to get function, which compute rigid motion from optical flow and depth information. Maybe you have some of examples of this function, thank you.

There is already a special handling for rotations about 0° but for a rotation about 180° there is no.
If trace of the rotation matrix is -1, theta becomes pi and any division by sin(theta) gives infinite values.

T tmp = R.trace()-1;
T theta = std::acos(std::min(T(1),std::max(T(-1),T(0.5)*tmp)));

Example

rot = np.array([[
        [-0.994789, 0.0789573, -0.0644982],
        [0.0790982, 0.198569, -0.97689],
        [-0.0643253, -0.976901, -0.20378]
]])

with tf.Session() as sess:
    r = tf.Variable(rot, dtype=tf.float32)
    sess.run(tf.global_variables_initializer())
    aa = sops.rotation_matrix_to_angle_axis(r)
    loss = tf.reduce_mean(sops.angle_axis_to_rotation_matrix(aa) - r)
    var_grad = tf.gradients(loss, [r])

    print(aa.eval())
    # [[  197.05762  3106.6025  -2531.7622 ]] norm: 4012.4346

    print(var_grad[0].eval())
    # [[[ inf  inf  inf]
    #  [-inf  inf -inf]
    #  [-inf  inf  inf]]]

From @pengwangucla moved from lmb-freiburg/demon#7

HI Benjamin,

Bug in your depth2normal code ? I recently use your depth2normal code and find the tensorflow implementation.

    const T cx = 1/(intrinsics[4*z+2]*x_size);
    const T cy = 1/(intrinsics[4*z+3]*y_size);

you have cx, cy as inverse of the input. I think it might be a bug since when we compute cx, cy, we do not need to inverse it.

I am using following specifications:

Ubuntu 18.04LTS
GPU = GeForce MX130
CUDA = 10.0.130 (Thats what nvcc -V says)
Driver = NVIDIA-418.67 (CUDA-10.1)(That's what nvidia-smi says)
But I installed only CUDA 10.0
Python = 3.6.7
But GPU is being used by other codes/programs without any problem.

For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From ../python/depthmotionnet/helpers.py:140: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.conv2d instead.
WARNING:tensorflow:From /home/zorawar/.local/share/virtualenvs/demon_venv/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From ../python/depthmotionnet/blocks_original.py:73: conv2d_transpose (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.conv2d_transpose instead.
WARNING:tensorflow:From /home/zorawar/.local/share/virtualenvs/demon_venv/lib/python3.6/site-packages/tensorflow/contrib/layers/python/layers/layers.py:1624: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From ../python/depthmotionnet/blocks_original.py:395: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dense instead.
WARNING:tensorflow:From ../python/depthmotionnet/blocks_original.py:165: calling norm (from tensorflow.python.ops.linalg_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From /home/zorawar/.local/share/virtualenvs/demon_venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
2019-05-30 14:08:18.209430: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
terminate called after throwing an instance of 'std::runtime_error'
  what():  /home/zorawar/Desktop/demon/lmbspecialops/src/warp2d_cuda.cu:218: cuda error: no kernel image is available for execution on the device

Aborted (core dumped)

Hello I'm trying to install/use the library and I'm having some issues.

My configuration is:

Ubuntu 16.04 LTS
tensorflow 1.2.1
cmake 3.5.1
python 3.5.2
cuda 8.0.61

I've cloned your project into ~/src/lmbspecialops and when I try to import the lib I get the error included bellow.
Any help will be appreciated, I'm afraid I've used all my arsenal.
Thanks in advance.

bou@bou-yoga:~/src$ export PYTHONPATH="lmbspecialops/python"
bou@bou-yoga:~/src$ export | grep PYTHON*
declare -x PYTHONPATH="lmbspecialops/python"
bou@bou-yoga:~/src$ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import lmbspecialops
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bou/src/lmbspecialops/python/lmbspecialops.py", line 28, in <module>
    lmbspecialopslib = tf.load_op_library(_lib_path)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/load_library.py", line 64, in load_op_library
    None, None, error_msg, error_code)
tensorflow.python.framework.errors_impl.NotFoundError: /home/bou/src/lmbspecialops/build/lib/lmbspecialops.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv

HI,

Could you tell me why I have this error with the new installation? Thanks!

[ 18%] No install step for 'lz4'
[ 20%] Completed 'lz4'
[ 20%] Built target lz4
CMakeFiles/Makefile2:107: recipe for target 'lib/CMakeFiles/lmbspecialops.dir/rule' failed
make[1]: *** [lib/CMakeFiles/lmbspecialops.dir/rule] Error 2
Makefile:129: recipe for target 'lmbspecialops' failed
make: *** [lmbspecialops] Error 2
Traceback (most recent call last):
File "setup.py", line 18, in
zip_safe=False,
File "/usr/local/lib/python3.7/dist-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 67, in run
self.do_egg_install()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 109, in do_egg_install
self.run_command('bdist_egg')
File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/bdist_egg.py", line 164, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/bdist_egg.py", line 150, in call_command
self.run_command(cmdname)
File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/usr/lib/python3.7/distutils/command/install_lib.py", line 109, in build
self.run_command('build_ext')
File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/usr/local/lib/python3.7/dist-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/usr/local/lib/python3.7/dist-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/usr/local/lib/python3.7/dist-packages/cmake_setuptools/cmake/init.py", line 59, in build_extension
env=env)
File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-j', 'lmbspecialops']' returned non-zero exit status 2.

Is this related?

-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found PythonInterp: /usr/local/bin/python (found version "2.7.17")
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named tensorflow
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named tensorflow
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named tensorflow

--
-- TENSORFLOW_FRAMEWORK_LIB-NOTFOUND
-- TensorFlow compile definitions:
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda (found version "11.1")

In tensorflow r1.14, libtensorflow_framework.so was renamed to libtensorflow_framework.so.1. This breaks CMake, which does not find the library anymore (line in sourcecode)

find_library( TENSORFLOW_FRAMEWORK_LIB tensorflow_framework PATHS "${TENSORFLOW_LIB_DIR}" NO_DEFAULT_PATH )

I fixed it in my branch by using tf.sysconfig.get_link_flags. It is only available in tensorflow >= r1.05, therefore the minium version is bumped to r1.05. If that's ok, I can create a PR.

diff --git a/cmake/tensorflow_build_flags.cmake b/cmake/tensorflow_build_flags.cmake
index b98feea..1c8991b 100644
--- a/cmake/tensorflow_build_flags.cmake
+++ b/cmake/tensorflow_build_flags.cmake
@@ -6,15 +6,12 @@ find_package( PythonInterp REQUIRED )
 
 execute_process( COMMAND ${PYTHON_EXECUTABLE} "-c" "from __future__ import print_function; import tensorflow as tf; print(tf.sysconfig.get_include(), end='')"
         OUTPUT_VARIABLE TENSORFLOW_INCLUDE_DIR )
-execute_process( COMMAND ${PYTHON_EXECUTABLE} "-c" "from __future__ import print_function; import tensorflow as tf; print(tf.sysconfig.get_lib(), end='')"
-        OUTPUT_VARIABLE TENSORFLOW_LIB_DIR )
 execute_process( COMMAND ${PYTHON_EXECUTABLE} "-c" "from __future__ import print_function; import tensorflow as tf; \
         print(';'.join(flag[2:] for flag in tf.sysconfig.get_compile_flags() if flag.startswith('-D')), end='')"
         OUTPUT_VARIABLE TENSORFLOW_COMPILE_DEFINITIONS )
-
-find_library( TENSORFLOW_FRAMEWORK_LIB tensorflow_framework PATHS "${TENSORFLOW_LIB_DIR}" NO_DEFAULT_PATH )
+execute_process(COMMAND ${PYTHON_EXECUTABLE} "-c" "import tensorflow as tf; print(' '.join(tf.sysconfig.get_link_flags()))"
+        OUTPUT_VARIABLE TENSORFLOW_LINK_FLAGS OUTPUT_STRIP_TRAILING_WHITESPACE)
 
 message( STATUS "${TENSORFLOW_INCLUDE_DIR}" )
-message( STATUS "${TENSORFLOW_LIB_DIR}" )
-message( STATUS "${TENSORFLOW_FRAMEWORK_LIB}" )
 message( STATUS "TensorFlow compile definitions: ${TENSORFLOW_COMPILE_DEFINITIONS}" )
+message( STATUS "TensorFlow link flags: ${TENSORFLOW_LINK_FLAGS}" )
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 8463d92..1228ad1 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -86,6 +86,6 @@ target_link_libraries( lmbspecialops
         ${lz4_STATIC_LIB}
         ${webp_LIBRARY}
 )
-if( TENSORFLOW_FRAMEWORK_LIB )
-    target_link_libraries( lmbspecialops ${TENSORFLOW_FRAMEWORK_LIB} )
+if( TENSORFLOW_LINK_FLAGS )
+    target_link_libraries( lmbspecialops ${TENSORFLOW_LINK_FLAGS} )
 endif()

The LeakyRelu custom op defined in src/leakyrelu.cc appears to be shadowed by TensorFlow's tf.nn.leaky_relu when loading the built lmbspecialops library with tf.load_op_library() from TensorFlow 1.13.1.
More specifically, leaky_relu does not get listed in the Python module returned by tf.load_op_library(), causing leaky_relu = lmbspecialopslib.leaky_relu to subsequently fail in lmbspecialops.py.

Renaming LeakyReluOp and LeakyReluGradOp to some other name fixes this issue for me as a workaround. Replacing the custom op with TensorFlow's implementation might be a better solution, but I'm not sure if the implementations are similar enough.

I am using TensorFlow 1.13.1 from inoryy/tensorflow-optimized-wheels with Python 3.7 and CUDA 10.0. I did not have this issue earlier when using TF 1.12. Even more so, tf.nn.leaky_relu was added way back in TF 1.4, which lmbspecialops should explicitly support.

Hello,

In flowtodepth.cc, it requires the normalized focal length and pos. of the principal point. I am working with a dataset where I'm unsure whether the given intrinsic values are normalized or not...

How are you normalizing these inputs?

Best,

Sam Pepose

In depthtoflow_cuda.cu I see the following pattern has been used:

//device.memcpyDeviceToHost(in_cpu, in_gpu, sizeof(T)*w_size*step); // Is this async?
cudaMemcpyAsync(in_cpu, in_gpu, sizeof(T)*w_size*step, cudaMemcpyDeviceToHost, device.stream() );
cudaStreamSynchronize(device.stream());
...
depthtoflow_gpu(device.stream(),

As I understand CUDA, the cudaStreamSynchronize shouldn't be necessary as the Memory Copy and the Kernel Copy should be enqueued on the same stream.

However, I've had to employ the same patten in my code, otherwise I occasionally run into subtle errors that are seemingly caused by data not being properly copied to the device.

Could someone provide me with the reason why the above is necessary?

Thanks a lot for the contribution!
Since the correlation layer proposed by FlowNet is also very widely used, is it possible to also integrate the correlation layer in this library?

error building

depthtoflow_cuda.cu(282): error: identifier "angleaxis_to_rotmatrix_gpu" is undefined
detected during:
instantiation of "void DepthToFlowOp_GPU::Compute(tensorflow::OpKernelContext *) [with T=double]"

I'm try to build on windows 10, 64bit with CUDA 9.0 and tensoflow 1.12.0 and I have follow error:
D:\Program Files\Python35\lib\site-packages\tensorflow\include\third_party/eigen3/unsupported/Eigen/CXX11/Tensor(1): fatal error C1083: 'unsupported/Eigen/CXX11/Tensor': No such file or directory
CMake Error at lmbspecialops_generated_depthtoflow_cuda.cu.obj.Debug.cmake:219

What should be the issue?