zhouyanzhao / orn Goto Github PK

View Code? Open in Web Editor NEW

223.0 28.0 51.0 32.29 MB

Oriented Response Networks, in CVPR 2017

Home Page: http://yzhou.work/ORN

License: BSD 3-Clause "New" or "Revised" License

Jupyter Notebook 100.00%

torch pytorch caffe cvpr rotation-invariant-features

orn's People

Contributors

Stargazers

Watchers

orn's Issues

Question about ARF

Hi
I am a little bit confused with ARF cuda kernel.

ORN/src/orn/lib/active_rotating_filters.cu

Lines 19 to 33 in d6b38aa

 ) { 

 // index 

 const int n = blockIdx.x * blockDim.x + threadIdx.x; 

 if (n < count) { 

 const uint16_t l = n % nEntry; 

 const uint16_t j = (n / nEntry) % nInputPlane; 

 const uint16_t i = n / nEntry / nInputPlane; 

 const scalar_t val = weight_flatten[n]; 

 for (uint16_t k = 0; k < nRotation; k++) { 

 const uint16_t index = (uint16_t)indices_flatten[l][k] - 1; 

 output[i][k][j][index] = val; 

 } 

 } 

 }

Let's say, assume thread 0 and thead 1 has:

i₀ == i₁
j₀ == j₁
k₀ == k₁
index₀ == index₁

So output[i][k][j][index]. I think that is not expected. Different threads write data to the same memory address leading to unpredictable results.

Did I misunderstand the implementation?

PS: Is yzhou.work still available?

how to make the feature map visualization？

Sincerely Zhou, I appreciate your great work on oriented feature. Now I have a question that how you make the feature map visualization. so would you like to give me some advice?

Hello,
I was using the oralign1d in the pythorch implementation like this:
x, main_dir = oralign1d(x, self.nOrientation, True)
main_dir is a vector of 80 size of values in [0, 7], can you tell what do this values and their positions mean? and how can I use this information to get the dominate orientation in the image?

ORN in detection framework

First, thank you for this great contribution.

In your CVPR 2017 poster at Application 5 section there is a comparison of Faster R-CNN detection (bounding box only) with an ORN detection (bounding box + orientation).
Could your elaborate on how you used ORN in a detection framework (maybe you have a github repo) and what dataset was used.

How can I run the pytorch branch on win10?

Does anyone change the pytorch branch to a win-version?

about Norientation channel and Nrotated versions

Dear author, I have read your paper recently, but I do not quite understand some details in it.It is ,why Norientation channels when had eight rotational versions.Looking forward to your answer.Thank you.

ORN for pytorch1.0

Hello, is there any plan to update the pytorch branch to pytorch1.0?

default train_mnist.sh can't get the desired result.

note [Rot_ORN_8_Align] | epoch 197 | elapse 17.76 | remain  0: 0:53 | testLoss 0.046485 | testAcc 98.7000 | trainLoss 0.039256 | trainAcc 98.8201
note [Rot_ORN_8_Align] | epoch 198 | elapse 17.78 | remain  0: 0:35 | testLoss 0.056883 | testAcc 98.5600 | trainLoss 0.038547 | trainAcc 98.8722
note [Rot_ORN_8_Align] | epoch 199 | elapse 17.81 | remain  0: 0:17 | testLoss 0.068868 | testAcc 98.3600 | trainLoss 0.038445 | trainAcc 98.8421
note [Rot_ORN_8_Align] | epoch 200 | elapse 17.82 | remain  0: 0: 0 | testLoss 0.046446 | testAcc 98.6600 | trainLoss 0.041818 | trainAcc 98.7800
| Thu Jul 20 15:49:03 2017 | best model: /data/Repo/ORN.torch/demo/logs/Rot_ORN_8_Align/192-99.04-20170720154638.t7
{{MSG:| Thu Jul 20 15:49:03 2017 | [Rot_ORN_8_Align] all done. bestValidationAcc: 99.04, finalTestAcc: 99.02}}

Not as good as the paper presented.

Trouble installing pytorch-v2 branch

I'm having some trouble installing the pytorch-v2 branch.

I've created a new conda environment and installed pytorch 1.12.0 with the command from here:
conda install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 cudatoolkit=11.6 -c pytorch -c conda-forge

Then I cloned this repository with
git clone --single-branch --branch pytorch-v2 https://github.com/ZhouYanzhao/ORN

Then I follow the instructions with the following and get:
cd ORN
pip install .

Processing /home/inno/devel/object-detection/ORN
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: orn
  Building wheel for orn (setup.py) ... done
  Created wheel for orn: filename=orn-1.0.0-py3-none-any.whl size=10661 sha256=b160fda674b97330dc00d5fd1d429513cb3b52f5b4697b63f8e6935329ab284a
  Stored in directory: /tmp/pip-ephem-wheel-cache-v4f25yce/wheels/1a/89/cd/c894f8cb9255d7ae6ca139b0e35bc59952eaa14c4bbb0b7195
Successfully built orn
Installing collected packages: orn
  Attempting uninstall: orn
    Found existing installation: orn 1.0.0
    Uninstalling orn-1.0.0:
      Successfully uninstalled orn-1.0.0
Successfully installed orn-1.0.0

Then when executing a python script I do import orn and get:

Using ~/.cache/torch_extensions/py37_cu116 as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file ~/.cache/torch_extensions/py37_cu116/active_rotating_filters/build.ninja...
Building extension module active_rotating_filters...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] /usr/bin/nvcc  -DTORCH_EXTENSION_NAME=active_rotating_filters -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1013\" -isystem /opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/include -isystem /opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/include/TH -isystem /opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/include/THC -isystem /opt/anaconda3/envs/CHPDet/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -std=c++14 -c /opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/orn/lib/active_rotating_filters.cu -o active_rotating_filters.cuda.o
FAILED: active_rotating_filters.cuda.o
/usr/bin/nvcc  -DTORCH_EXTENSION_NAME=active_rotating_filters -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1013\" -isystem /opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/include -isystem /opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/include/TH -isystem /opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/include/THC -isystem /opt/anaconda3/envs/CHPDet/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -std=c++14 -c /opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/orn/lib/active_rotating_filters.cu -o active_rotating_filters.cuda.o
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1814, in _run_ninja_build
    env=env)
  File "/opt/anaconda3/envs/CHPDet/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/orn/__init__.py", line 1, in <module>
    from orn.oriented_response_convolution import ORConv2d
  File "/opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/orn/oriented_response_convolution.py", line 8, in <module>
    from orn.active_rotating_filters import mapping_rotate
  File "/opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/orn/active_rotating_filters.py", line 10, in <module>
    ]], verbose=True)
  File "/opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1214, in load
    keep_intermediates=keep_intermediates)
  File "/opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1435, in _jit_compile
    is_standalone=is_standalone)
  File "/opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1540, in _write_ninja_file_and_build_library
    error_prefix=f"Error building extension '{name}'")
  File "/opt/anaconda3/envs/CHPDet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'active_rotating_filters'

Any idea on how to fix this?

Function in Pytorch and caffe support

This wok is very nice. And thank you very much for release the codes.
My issue is that I can not find orpooling in pytorch version and oralign in caffe version.
Will you plan to release the corresponding codes in future? @ZhouYanzhao

Little bug in main.py

Hi, ZhouYanzhao

Thanks for your sharing code!

line 163 should change to :
if test_acc > best_test_acc[0]: => if test_acc.float() > best_test_acc[0]:

Is there any plan to release tensorflow impl. of ORN?

First, thank you for your great research and contribution.

I really need tensorflow implementation of this repo.
Is there any plan to release tensorflow impl.?

If so, I will just wait your release. If not, I have the plan to implement it in long-term on my hands.

Error in liborn for pytorch branch

At first I ran

python3 setup.py install

which created build, dist and orn.egg-info folders.
Inside the build folder there was the orn folder containing _ext which has liborn. But init.py file inside the liborn folder gives reference error on the line :

from ._liborn import lib as _lib, ffi as _ffi

Upgrade the pytorch branch to 1.0, but fails while grad checking

Thanks for sharing!

Recently, I upgrade the pytorch branch to be compatible with pytorch 1.0. When I conduct the grad check for activerotaingfilter, I failed. But the rotationinvariantencoding success. I have checked my codes many times, but still don't solve it. Do you plan to upgrade this branch to python1.0?

FYI: Also need openblas dependency

ATT.

when i input bash instal.sh, it shows error : orn/src/liborn_kernel.cu.o: 没有那个文件或目录.

Hello,ZhouYanzhao.
when i input bash instal.sh, it shows error : orn/src/liborn_kernel.cu.o: 没有那个文件或目录.
Can you help me? Thank you!

The instructions:

bash instal.sh
...
...
...
build/temp.linux-x86_64-3.5/home/yangyu/ORN/ORN.pytorch/install/orn/src/liborn_cuda.o orn/src/liborn_kernel.cu.o -o build/lib.linux-x86_64-3.5/orn/_ext/liborn/_liborn.abi3.so
x86_64-linux-gnu-gcc: error: orn/src/liborn_kernel.cu.o: 没有那个文件或目录

Will you release the caffe or pytorch code?

Thanks!

the meaning of nOrientation and nRotation

according to my understanding, nOrientation means the dim of each point(unit) in the feature map or kernel, and nRotation means the copy-rotated number of the kernel. Points in the feature map or kernel, are not always a scalar but a vector, or a n-dim point as your paper say.

So,
ORConv2d(1,10,arf_config=(1,8), kernel_size=3) means input is with 1 channel, where points are scalars, and conv_kernel is with 1 in_ch, 10 out_ch, 8 rotated copy, where points are scalars, too.
ORConv2d(10,20,arf_config=8, kernel_size=3) means input is with 10 channel, where points are 8-dim vector, and conv_kernel is with 10 in_ch, 20 out_ch, 8 rotated copy, where points are 8-dim vector, too.

As a word, nOrientation is the dim of points in input, while nRotation is the dim of points in output.

Is it right?

PyTorch Using filters with kernel_size=1 causes exception.

Exception: 'int' object does not support indexing,
for line kernel = kernel_indices[kW][angle][j]

Wrapping the kernel_indices for size 1 as a list fixes the issue. New kernel_indices:

    kernel_indices = {
        1: {
            0: [1],
            45: [1],
            90: [1],
            135: [1],
            180: [1],
            225: [1],
            270: [1],
            315: [1]
        },
        3: {
            0: (1,2,3,4,5,6,7,8,9),
            45: (2,3,6,1,5,9,4,7,8),
            90: (3,6,9,2,5,8,1,4,7),
            135: (6,9,8,3,5,7,2,1,4),
            180: (9,8,7,6,5,4,3,2,1),
            225: (8,7,4,9,5,1,6,3,2),
            270: (7,4,1,8,5,2,9,6,3),
            315: (4,1,2,7,5,3,8,9,6)
        }
    }

Question about using or-resnet for original --> rot experiments

I use the network defined in the demo.py, it is ok to reproduce the similar results for original --> rot experiments on mnist. But when I changed to or-resnet, it got very low results, the accuracy is no more than 50%(even lower than the simple OR-CNN defined in demo.py ). I do not know what's wrong with it. The following is my code.
https://paste.ubuntu.com/p/NssKJbxDqZ/
What's more, when I do original --> rot experiments on cifar-10. I use the network defined in the demo.py. And got results of 35.26 and 44.97 for without/with orn respectively.
Look forward to your reply.

	) {
	// index
	const int n = blockIdx.x * blockDim.x + threadIdx.x;
	if (n < count) {
	const uint16_t l = n % nEntry;
	const uint16_t j = (n / nEntry) % nInputPlane;
	const uint16_t i = n / nEntry / nInputPlane;
	const scalar_t val = weight_flatten[n];
	for (uint16_t k = 0; k < nRotation; k++) {
	const uint16_t index = (uint16_t)indices_flatten[l][k] - 1;
	output[i][k][j][index] = val;
	}
	}
	}

zhouyanzhao / orn Goto Github PK

orn's People

Contributors

Stargazers

Watchers

Forkers

orn's Issues

Recommend Projects

Recommend Topics

Recommend Org