Giter Club home page Giter Club logo

hawp's People

Contributors

alwc avatar awarebayes avatar cherubicxn avatar ckanesan avatar encounter1997 avatar pinto0309 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hawp's Issues

Questions about the paper and code version

Thanks for your work! That‘s excellent! I am very interested in your research. When I learned about you, I found that there are several versions of the paper and code. I want to ask about the relationship between them.
Q1:Is《Holistically-Attracted Wireframe Parsing: From Supervised to Self-Supervised Learning》 an improvement of 《Holistically-Attracted Wireframe Parsing》?
Q2:If I want to run the code, which one is better to learn? Is it the latest version? But I haven't seen the relevant running tutorials in the new version. Do I still need to refer to the previous code links?

Some questions about the pytorch version

Results from pytorch1.9 have more disorderly lines compared with ones from pytorch1.3.1.
Are there any bugs for it?
Lines from pytorch1.9:
image

Lines from pytorch1.3.1:
image

My HAWP model was trained in pytorch1.3.1

Training with own data

Hello and thank you for your excellent work. Can you please tell me how I can train my own unlabeled data by pre-training the model and how should I organize the config file.

test可视化

你好,请问预测结果可视化的代码可以公布一下吗

There are some errors about Training of HAWPv3

I read the HAWPv3.train.md and tried to follow the command to have a self-training.
However, there are some errors in step 1.

for example:
for single-image mode, in --datacfg, the 'exports' was mistakenly written as 'export'.
for batch mode, the default config file "YAML_TAMPLATE" in homoadp-bm.py is missing.

Also, the dataset instruction is also missing. I use the wireframe/train in wireframe dataset as training data. But still get some error during construct_dataset.

Sincerely hope that you can check the SSL-related code and update the HAWPv3.train.md documentation. Thank you.

AttributeError: module 'torch.hub' has no attribute 'get_dir'

Hello~

Thanks for the nice work.

When I use the 'hawp/src/predict.py' code to predict new images, I met the following error. I checked the attributes in torch.hub, seeing that 'get_dir' does not exist and there are some other similar attributes e.g. 'set_dir()', 'hub_dir'. I have tried other attributes but still failed in loading the pretrained model.

File "/home/yyz/hawp/parsing/detector.py", line 430, in get_hawp_model
hubdir = torch.hub.get_dir()
AttributeError: module 'torch.hub' has no attribute 'get_dir'

Could you please give me some suggestions in dealing with the problem?

Question about the pretrained model

Hello,
Sorry to bother you via this platform. When I was running the quickstart with the pretrained model, there exists some errors related to _C.so files, which is displayed as follow:

albert@albert-Precision-5820-Tower:~/hawp$ python scripts/predict.py --config-file config-files/hawp.yaml --img figures/example.png Traceback (most recent call last): File "scripts/predict.py", line 6, in <module> from parsing.detector import WireframeDetector File "/home/albert/hawp/scripts/parsing/detector.py", line 4, in <module> from parsing.encoder.hafm import HAFMencoder File "/home/albert/hawp/scripts/parsing/encoder/__init__.py", line 1, in <module> from .hafm import HAFMencoder File "/home/albert/hawp/scripts/parsing/encoder/hafm.py", line 5, in <module> from parsing import _C ImportError: /home/albert/hawp/scripts/parsing/_C.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _Z13lsencode_cudaRKN2at6TensorEiiiii

Could you please indicate how to solve this? Thanks a lot!

Weipeng

how to training custom dataset?

when I use wireframe dataset, and the result is successfully, but when use custom datase, the result is very bad.
could you provide some detail information or some comment for train custom dataset?

thanks

Publish code training for Hawpv3

Hi,
I 've a question, have you got a date when the HAWPv3 training code will be publish ? Otherwide, how can I train Hawp on my own dataset ? I've read the topic about "training own dataset" but I didn't interstand it

Thank you !

torch版本问题

您好,我发现torch版本高于1.2时会报下列错误
undefined symbol: _ZN6caffe26detail37_typeMetaDataInstance_preallocated_32E

请问一下您的torch版本是多少

Can't reproduce the sAP10 result

Hi @cherubicXN ,

After training the model with your given code and data, I tested with

$ CUDA_VISIBLE_DEVICES=0, python scripts/test.py --config-file config-files/hawp.yaml                                                                            
2020-05-26 11:02:32,633 hawp INFO: Namespace(config_file='config-files/hawp.yaml', display=False, opts=[])
2020-05-26 11:02:32,633 hawp INFO: Loaded configuration file config-files/hawp.yaml
2020-05-26 11:02:34,451 hawp.testing INFO: Loading checkpoint from outputs/hawp/model_00030.pth
2020-05-26 11:02:34,630 hawp.testing INFO: Testing on wireframe_test dataset
100%|█████████| 462/462 [00:14<00:00, 32.98it/s]
2020-05-26 11:02:48,641 hawp.testing INFO: Writing the results of the wireframe_test dataset into outputs/hawp/wireframe_test.json

and evaluated with

$ python scripts/eval_sap.py --path outputs/hawp/wireframe_test.json --threshold 10
sAP10.0 = 58.4

Note that I'm getting sAP10.0 = 58.4, which is much lower than the stated result in the paper (i.e. 66.5). If I ran the code above using your provided pre-trained model. I could get sAP10.0 = 66.5.

FYI, I'm using PyTorch 1.4.0 with Python 3.6, trained on one 2080Ti GPU.

Here are the settings I used:

2020-05-25 14:36:35,208 hawp INFO: Namespace(clean=False, config_file='config-files/hawp.yaml', opts=[], seed=2)
2020-05-25 14:36:35,208 hawp INFO: Loaded configuration file config-files/hawp.yaml
2020-05-25 14:36:35,208 hawp INFO:
SOLVER:
  IMS_PER_BATCH: 6
  BASE_LR: 0.0004
  MAX_EPOCH: 30
  STEPS: (25,)
  WEIGHT_DECAY: 0.0001
  CHECKPOINT_PERIOD: 1
  OPTIMIZER: "ADAM"
  AMSGRAD: True

DATALOADER:
  NUM_WORKERS: 8
DATASETS:
  IMAGE:
    PIXEL_MEAN: [109.730, 103.832, 98.681]
    PIXEL_STD: [22.275, 22.124, 23.229]
    TO_255: True

  TEST: ("wireframe_test","york_test")

MODEL:
  NAME: "Hourglass"
  HEAD_SIZE: [[3], [1], [1], [2], [2]] #Order: ang, dis, dis_residual, jloc, joff
  OUT_FEATURE_CHANNELS: 256
  HGNETS:
    DEPTH: 4
    NUM_STACKS: 2
    NUM_BLOCKS: 1
    INPLANES: 64
    NUM_FEATS: 128

  PARSING_HEAD:
    USE_RESIDUAL: True
    MAX_DISTANCE: 5.0
    N_DYN_JUNC:   300
    N_DYN_POSL:   300
    N_DYN_NEGL:   0
    N_DYN_OTHR:   0
    N_DYN_OTHR2:  300
    N_PTS0: 32
    N_PTS1: 8
    DIM_LOI: 128
    DIM_FC: 1024
    N_OUT_JUNC: 250
    N_OUT_LINE: 2500

  LOSS_WEIGHTS:
    loss_md: 1.0 # angle regression
    loss_dis: 1.0 # dis   regression
    loss_res: 1.0      # residual regression
    loss_joff: 0.25    # joff  regression
    loss_jloc: 8.0     # jloc  classification
    loss_pos: 1.0      # pos   classification
    loss_neg: 1.0      # neg   classification

OUTPUT_DIR: "outputs/hawp"

2020-05-25 14:36:35,208 hawp INFO: Running with config:
DATALOADER:
  NUM_WORKERS: 8
DATASETS:
  DISTANCE_TH: 0.02
  IMAGE:
    HEIGHT: 512
    PIXEL_MEAN: [109.73, 103.832, 98.681]
    PIXEL_STD: [22.275, 22.124, 23.229]
    TO_255: True
    WIDTH: 512
  NUM_STATIC_NEGATIVE_LINES: 40
  NUM_STATIC_POSITIVE_LINES: 300
  TARGET:
    HEIGHT: 128
    WIDTH: 128
  TEST: ('wireframe_test', 'york_test')
  TRAIN: ('wireframe_train',)
  VAL: ('wireframe_test',)
ENCODER:
  ANG_TH: 0.1
  DIS_TH: 5
  NUM_STATIC_NEG_LINES: 40
  NUM_STATIC_POS_LINES: 300
MODEL:
  DEVICE: cuda
  HEAD_SIZE: [[3], [1], [1], [2], [2]]
  HGNETS:
    DEPTH: 4
    INPLANES: 64
    NUM_BLOCKS: 1
    NUM_FEATS: 128
    NUM_STACKS: 2
  LOSS_WEIGHTS:
    loss_dis: 1.0
    loss_jloc: 8.0
    loss_joff: 0.25
    loss_md: 1.0
    loss_neg: 1.0
    loss_pos: 1.0
    loss_res: 1.0
  NAME: Hourglass
  OUT_FEATURE_CHANNELS: 256
  PARSING_HEAD:
    DIM_FC: 1024
    DIM_LOI: 128
    MATCHING_STRATEGY: junction
    MAX_DISTANCE: 5.0
    N_DYN_JUNC: 300
    N_DYN_NEGL: 0
    N_DYN_OTHR: 0
    N_DYN_OTHR2: 300
    N_DYN_POSL: 300
    N_OUT_JUNC: 250
    N_OUT_LINE: 2500
    N_PTS0: 32
    N_PTS1: 8
    N_STC_NEGL: 40
    N_STC_POSL: 300
    USE_RESIDUAL: True
  SCALE: 1.0
  WEIGHTS:
OUTPUT_DIR: outputs/hawp
SOLVER:
  AMSGRAD: True
  BACKBONE_LR_FACTOR: 1.0
  BASE_LR: 0.0004
  BIAS_LR_FACTOR: 1
  CHECKPOINT_PERIOD: 1
  GAMMA: 0.1
  IMS_PER_BATCH: 6
  MAX_EPOCH: 30
  MOMENTUM: 0.9
  OPTIMIZER: ADAM
  STEPS: (25,)
  WEIGHT_DECAY: 0.0001
  WEIGHT_DECAY_BIAS: 0

Log from the last epoch:

2020-05-26 02:57:25,525 hawp.trainer INFO: eta: 0:00:00 epoch: 30 iter: 3333 data: 0.0080 (0.0091) loss: 0.8587 (0.8574) loss_dis: 0.1496 (0.1461) loss_jloc: 0.0318 (0.0327) loss_joff: 0.3663 (0.3664) loss_md: 0.1448 (0.1417) loss_neg: 0.0771 (0.0809) loss_pos: 0.0451 (0.0456) loss_res: 0.0915 (0.0904) time: 0.4403 (0.4446) lr: 0.000040 max mem: 6232

Model evaluation missing files

Hi, I was trying to compute F-score of your algorithm, but apparentely there are essential files missing.
In example_evaluation.py you import "from evaluation.prmeter import PrecisionRecallMeter", and in prmeter you import:

  • from .EdgeEval import correspond
  • from .RasterizeLine import drawfn
    whitch are missing. Am I doing something wrong?

index1 = lcnn_line_map[idx, 0] IndexError: index 0 is out of bounds for axis 1 with size 0

Hi,
I 've create my own dataset in wireframe format but made a preprocessed with wireframe.py in L-CNN github. There is no error during the preprocessed but when I tried to train with Hawpv3, I had this issue :

Traceback (most recent call last):
File "/opt/conda/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/conda/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/project_ghent/HAWP/hawp/ssl/train.py", line 233, in
main()
File "/project_ghent/HAWP/hawp/ssl/train.py", line 145, in main
train(model, train_loader, optimizer, scheduler, loss_reducer, arguments, output_dir)
File "/project_ghent/HAWP/hawp/ssl/train.py", line 170, in train
for it, data in enumerate(train_loader):
File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/opt/conda/lib/python3.9/site-packages/torch/_utils.py", line 425, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/project_ghent/HAWP/hawp/ssl/datasets/wireframe_dataset.py", line 1341, in getitem
data = self.train_preprocessing(data)
File "/project_ghent/HAWP/hawp/ssl/datasets/wireframe_dataset.py", line 871, in train_preprocessing
line_map_neg = self.convert_line_map(line_neg, num_junctions)
File "/project_ghent/HAWP/hawp/ssl/datasets/wireframe_dataset.py", line 667, in convert_line_map
index1 = lcnn_line_map[idx, 0]
IndexError: index 0 is out of bounds for axis 1 with size 0

I saw there was empty Lneg (cause only one line in a picture for the train maybe) I don't know if it came from this variable. What do you suggest ?

Thank in advance !

Want to add docker build and run example for your prediction

Hello, sorry to bother you via issue creation, but I failed to find another way for communication.
I have created a basic example of using your prediction from a Docker container.
It allows to run it every time in clear environment and easily run on any host just after docker and Nvidia drivers installation.
Could you please allow me to push a new branch and create a pull request with this changes.
I think it could be helpful for others who wants to see how you perfect research works.

Thanks,
Felix

Line segment and endpoint binding problem

There are several questions when reading the excellent paper:
Is the line segment and point binding step input into the network? Is it also feasible to do post-processing directly without inputting the network?
I would appreciate your reply

Merge hawp head with swin-l in semantic segmentation

Hi !

I don't know if is-it possible but Can I merge your head of your model with the swin-L model ?
Maybe this will improve the model and it will detect polygons and lines with differents category on picture
I'm waiting for your opinion

Thanks !

Need to uncomment a line in setup.py for pip install

Hi,

Thank you again for the great work and for the latest updates of the HAWP models. This is a great work and is very useful for the community!

I just have a minor question regarding the setup.py file. The line

hawp/setup.py

Line 19 in f18f10e

# packages=find_packages(exclude=("configs", "tests",)),
is currently commented out and prevents the package from being installed with pip. Running pip install -e . results in the following error for me:

error: Multiple top-level packages discovered in a flat-layout: ['hawp', 'docker', 'configs', 'weights', 'evaluation'].
      
To avoid accidental inclusion of unwanted files or directories,
setuptools will not proceed with this build.

If you are trying to create a single distribution with multiple packages
on purpose, you should not rely on automatic discovery.
Instead, consider the following options:

1. set up custom discovery (`find` directive with `include` or `exclude`)
2. use a `src-layout`
3. explicitly set `py_modules` or `packages` with a list of names

To find more information, look for "package discovery" on setuptools docs.
[end of output]
  
 note: This error originates from a subprocess, and is likely not a problem with pip.

Uncommenting this line fixes the issue and allows the installation with pip. What is the reasoning behind commenting this line out? Would it be possible to uncomment it in the public repo? I can of course fork it and do the change myself, but some people might also benefit from it.

libtorch 导出后在c++上加载失败

libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
File "code/torch/parsing/detector.py", line 231, in forward
133 = torch.mul(torch.select(_132, 1, 0), sx)
_134 = torch.slice(lines_final, 0, 0, 9223372036854775807, 1)
135 = torch.copy(torch.select(_134, 1, 0), torch.view(_133, [762]), False)
~~~~~~~~~~ <--- HERE
_136 = torch.slice(lines_final, 0, 0, 9223372036854775807, 1)
137 = torch.mul(torch.select(_136, 1, 1), sy)

Traceback of TorchScript, original code (most recent call last):
/share/group_guoxiaoyan/group_ar/caidongyang/plane/export_hawp/parsing/detector.py(245): forward_test
/share/group_guoxiaoyan/group_ar/caidongyang/plane/export_hawp/parsing/detector.py(134): forward
/opt/conda/envs/pytorch1.7/lib/python3.6/site-packages/torch/nn/modules/module.py(709): _slow_forward
/opt/conda/envs/pytorch1.7/lib/python3.6/site-packages/torch/nn/modules/module.py(725): _call_impl
/opt/conda/envs/pytorch1.7/lib/python3.6/site-packages/torch/jit/_trace.py(940): trace_module
/opt/conda/envs/pytorch1.7/lib/python3.6/site-packages/torch/jit/_trace.py(742): trace
scripts/predict_t.py(159): test
scripts/predict_t.py(338):
RuntimeError: shape '[762]' is invalid for input of size 4715

请问有计划建立一个导出工具吗

关于数据标注的问题。

你好,请教你一下,我有一个项目,需要对仓库里托盘上包装箱进行计数,需要分割出包装箱之间的缝隙,我想训练自己采集的数据集,请问需要严格按照你提供的train.json来进行标注嘛,或者说有没有什么快速的方法进行标注。

python setup.py

Hello, sorry to bother you via issue creation.
I would like to ask you about compiling cuda files, python setup.py build_ext --inplace.
First I compile in cuda11.0 and it fails. Then compile in cuda10.0 to get the _C.***.so file. Then move it to the parsing directory, run the train.py file and still report an error.
Traceback (most recent call last):
File "train.py", line 8, in
from parsing.detector import WireframeDetector
File "/home/sun/line/hawp-master/parsing/detector.py", line 4, in
from parsing.encoder.hafm import HAFMencoder
File "/home/sun/line/hawp-master/parsing/encoder/init.py", line 1, in
from .hafm import HAFMencoder
File "/home/sun/line/hawp-master/parsing/encoder/hafm.py", line 4, in
from parsing import _C
ImportError: cannot import name '_C' from 'parsing' (/home/sun/line/hawp-master/parsing/init.py)
Thanks,
Hao Sun

exprot to onnx error

RuntimeError: output 1 (6.94717
[ CPUDoubleType{} ]) of traced region did not have observable data dependence with trace inputs; this probably indicates your program cannot be understood by the tracer.

CPU only?

I would like to test your pretrained model on some custom images. I tried to follow the instructions in your inference branch. However, my graphics card is too old (although CUDA Compute Compatiblity 3.0 should be supported until CUDA Toolkit 10.2, I couldn't get it to work).

I tried to run the command:

python -m hawp.predict --disable-cuda -o out/ example.png

but get the following error:

AssertionError: Torch not compiled with CUDA enabled

Although, the readme is stating the --disable-cuda flag, it seems to be nowhere considered in predict.py or any other files.

Is there any way to test your code and pretrained model without CUDA?

License

Hi, what is the license for this code? MIT License would be great. Thanks!

the new pth

Great work.
Is hawpv3-ce3ae2cb.pth not available right now?
Thank you

error: could not create 'parsing/_C.cpython-36m-x86_64-linux-gnu.so': No such file or directory

Hi,
Thanks a lot for your contribution.
Unfortunately, I receive the following error message:
error: could not create 'parsing/_C.cpython-36m-x86_64-linux-gnu.so': No such file or directory

Dockerfile:

FROM nvcr.io/nvidia/pytorch:19.09-py3

# add current folder where Dockerfile and requirements are located to workspace
WORKDIR /workspace
ADD . /workspace

ENV DEBIAN_FRONTEND="noninteractive" TZ="Europe/London"
RUN apt-get update && apt-get install -qy libsm6 libxext6 libxrender-dev python3-tk apt-utils openmpi-bin liblzma-dev libfreetype6 libpng16-16

ENV CUDA_HOME="/usr/local/cuda"

# update pip
RUN pip install --upgrade pip

# Install Python dependencies
RUN git clone https://github.com/cherubicXN/hawp.git
RUN pip install -r /workspace/hawp/requirement.txt
RUN conda develop .
RUN python /workspace/hawp/setup.py build_ext --inplace
RUN chmod 777 . -R

# Clean up APT when done.
RUN apt-get clean

### docker build -t hawp:v1.0 .
###nd-run -it --rm --name test hawp:v1.0

Details:

 ---> Running in d33aa28ee7a1
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
running build_ext
building 'parsing._C' extension
creating build
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/workspace
creating build/temp.linux-x86_64-3.6/workspace/hawp
creating build/temp.linux-x86_64-3.6/workspace/hawp/parsing
creating build/temp.linux-x86_64-3.6/workspace/hawp/parsing/csrc
creating build/temp.linux-x86_64-3.6/workspace/hawp/parsing/csrc/cuda
gcc -pthread -B /opt/conda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/workspace/hawp/parsing/csrc -I/opt/conda/lib/python3.6/site-packages/torch/include -I/opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.6/site-packages/torch/include/TH -I/opt/conda/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/include/python3.6m -c /workspace/hawp/parsing/csrc/vision.cpp -o build/temp.linux-x86_64-3.6/workspace/hawp/parsing/csrc/vision.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/workspace/hawp/parsing/csrc -I/opt/conda/lib/python3.6/site-packages/torch/include -I/opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.6/site-packages/torch/include/TH -I/opt/conda/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/include/python3.6m -c /workspace/hawp/parsing/csrc/cuda/linesegment.cu -o build/temp.linux-x86_64-3.6/workspace/hawp/parsing/csrc/cuda/linesegment.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++11
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/parsing
g++ -pthread -shared -B /opt/conda/compiler_compat -L/opt/conda/lib -Wl,-rpath=/opt/conda/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/workspace/hawp/parsing/csrc/vision.o build/temp.linux-x86_64-3.6/workspace/hawp/parsing/csrc/cuda/linesegment.o -L/usr/local/cuda/lib64 -lcudart -o build/lib.linux-x86_64-3.6/parsing/_C.cpython-36m-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-3.6/parsing/_C.cpython-36m-x86_64-linux-gnu.so -> parsing
error: could not create 'parsing/_C.cpython-36m-x86_64-linux-gnu.so': No such file or directory
The command '/bin/sh -c python /workspace/hawp/setup.py build_ext --inplace #--user' returned a non-zero code: 1

I sincerely hope that you can help me out.

Predicting the model with rectangular input

Hi @cherubicXN

I'm trying to train the wireframe dataset with rectangular input shape, i.e. in hawp.yaml, I set

DATASETS:
  IMAGE:
    HEIGHT: 576
    WIDTH: 448
  TARGET:
    HEIGHT: 144
    WIDTH: 112

but it doesn't seem to be trainable at all. The highest sAP10.0 I'm getting is around ~7. Do you know what shall I be aware of if I want to train my model with rectangular input? Thanks!

Cannot use the inference brach in my own code

I am using the HAWP inference brach. I can use python -m hawp.predict test.jpg --show to get the results from terminal. However I want to combine hawp in my own code to combine with other algorithm, I tried:

from hawp import show
from hawp import predicting

wireframe_parser = predicting.WireframeParser(visualize_image=True)
wireframe_painter = show.painters.WireframePainter()
predict, _, meta = wireframe_parser.images("./test.jpg")
print(predict)

and I got the error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "G:\Anaconda\envs\pytorch_gpu\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "G:\Anaconda\envs\pytorch_gpu\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "G:\Anaconda\envs\pytorch_gpu\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "G:\Anaconda\envs\pytorch_gpu\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "G:\Anaconda\envs\pytorch_gpu\lib\runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "G:\Anaconda\envs\pytorch_gpu\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "G:\Anaconda\envs\pytorch_gpu\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "E:\LBW\line_3d_construction\ALGO\hawp_video_test.py", line 14, in <module>
    predict, _, meta = wireframe_parser.images("./test.jpg")
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\hawp\predicting.py", line 63, in images
    yield from self.dataset(data, **kwargs)
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\hawp\predicting.py", line 40, in dataset
    yield from self.dataloader(dataloader)
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\hawp\predicting.py", line 43, in dataloader
    for batch_i, item in enumerate(dataloader):
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\torch\utils\data\dataloader.py", line 359, in __iter__
    return self._get_iterator()
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\torch\utils\data\dataloader.py", line 918, in __init__
    w.start()
  File "G:\Anaconda\envs\pytorch_gpu\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "G:\Anaconda\envs\pytorch_gpu\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "G:\Anaconda\envs\pytorch_gpu\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "G:\Anaconda\envs\pytorch_gpu\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "G:\Anaconda\envs\pytorch_gpu\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "G:\Anaconda\envs\pytorch_gpu\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
Traceback (most recent call last):
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\torch\utils\data\dataloader.py", line 990, in _try_get_data
    data = self._data_queue.get(timeout=timeout)
  File "G:\Anaconda\envs\pytorch_gpu\lib\queue.py", line 178, in get
    raise Empty
_queue.Empty

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "E:/LBW/line_3d_construction/ALGO/hawp_video_test.py", line 14, in <module>
    predict, _, meta = wireframe_parser.images("./test.jpg")
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\hawp\predicting.py", line 63, in images
    yield from self.dataset(data, **kwargs)
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\hawp\predicting.py", line 40, in dataset
    yield from self.dataloader(dataloader)
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\hawp\predicting.py", line 43, in dataloader
    for batch_i, item in enumerate(dataloader):
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\torch\utils\data\dataloader.py", line 521, in __next__
    data = self._next_data()
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\torch\utils\data\dataloader.py", line 1186, in _next_data
    idx, data = self._get_data()
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\torch\utils\data\dataloader.py", line 1142, in _get_data
    success, data = self._try_get_data()
  File "G:\Anaconda\envs\pytorch_gpu\lib\site-packages\torch\utils\data\dataloader.py", line 1003, in _try_get_data
    raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e
RuntimeError: DataLoader worker (pid(s) 26680) exited unexpectedly

Is there any way to use the inference brach in my own code rather than commond from the terminal?

Thanks!

error!!!!

when i run the train.py.
the error is
from parsing import _C
hawp/parsing/_C.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _Z13lsencode_cudaRKN2at6TensorEiiiii

Training code

Hi Nan Xue,

Will you be posting the training code this week? Last week from your README it says, "I will finish it in this week". If possible, I would like to reproduce the code sometime this week so that I can play around the model. Much appreciated!

Failed to run the program

Hi, Nan Xue,

Thanks for your excellent work.

I really like your work. After following the instructions to prepare the env and running the script, I got the issue below

(hawp) root@ubuntu1:~/home/deng/projects/wire_frame/hawp# CUDA_VISIBLE_DEVICES=1 python scripts/train.py --config-file config-files/hawp.yaml Traceback (most recent call last): File "scripts/train.py", line 8, in <module> from parsing.detector import WireframeDetector File "/root/home/deng/projects/wire_frame/hawp/parsing/detector.py", line 4, in <module> from parsing.encoder.hafm import HAFMencoder File "/root/home/deng/projects/wire_frame/hawp/parsing/encoder/__init__.py", line 1, in <module> from .hafm import HAFMencoder File "/root/home/deng/projects/wire_frame/hawp/parsing/encoder/hafm.py", line 2, in <module> from parsing import _C ImportError: /root/home/deng/projects/wire_frame/hawp/parsing/_C.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _Z13lsencode_cudaRKN2at6TensorEiiiii

My system is
(hawp) root@ubuntu1:~/home/deng/projects/wire_frame/hawp# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.4 LTS Release: 18.04 Codename: bionic

My cuda driver and toolkit version is
NVIDIA-SMI 440.64.00 Driver Version: 440.64.00 CUDA Version: 10.2

Do you have any clue that can help me out? Really appreciate your assistance.

reproduce mAPJ

Hi @cherubicXN ,

Nice work!
Currently I am not able to obtain the mAPJ values in the papers using the pretrained model from you.
The sAP values are the same as in the table, but the mAPJ is much lower (24.6 vs 31.6 on York). Since the mAPJ script is not included in repo, I used the one from LCNN with minor modification (the junctions are in different formats). I was wondering if you can kindly provide the script for mAPJ evaluation. Still have no idea what is wrong and would like to figue it out. Thanks in advance!

BTW, congrats on your promotion, Dr.Xue.!

Segmentation fault (core dumped)

Hi, thank you for your work. When I train with your code, I met 'Segmentation fault (core dumped)' error, can you help check the code, thank you!

关于预训练模型

您好!请问可以放出预训练模型吗,我想试着做一些测试,谢谢!

Training with dataset wireframe

Hi !
I worked with LETR with wireframe dataset with json like "filename": 'Image1',lines : [[0,1],[1,1]] etc...
How to train your hawp code with this dataset ? I didn't find tutorial for training

Thanks !

what is wireframe-10iters.yaml like, how can I set my dataset?

In [docs/HAWPv3.train.md], there is code:

python -m hawp.ssl.homoadp-bm --metarch HAWP-heatmap \
    --datacfg hawp/ssl/config/exports/wireframe-10iters.yaml \
    --workdir exp-ssl/hawpv3-round0 \
    --epoch 10 \
    --modelcfg exp-ssl/hawpv3-round0/model.yaml \
    --min-score 0.75   --batch-size=16

but I can't find hawp/ssl/config/exports/wireframe-10iters.yaml, what is this file?

Idea: How to make inference on large image

Hello,
Thank you for your code and your work !
I have a suggestion about about your code.
I work with large image like 10000x5000. Do you think we can create a method for inference like SAHI on coco dataset ?

Thank you !

Negative samples from the static line sampler

Hi @cherubicXN

I'm trying to pre-process the dataset from scratch using the train.json file from https://github.com/cherubicXN/afm_cvpr2019 and some modified code from https://github.com/zhou13/lcnn

I can reproduce edges_positive and juctions correctly, but I can't seem to exactly match the edges_negative numbers found in your provided json file.

Here is my code modified from L-CNN's repo:

afm_train_json = data_dir / 'train.json'

with afm_train_json.open('r') as jf:
    dataset = json.load(jf)

data = dataset[1]
image = cv2.imread(str(afm_im_dir / data["filename"]))
lines = np.array(data["lines"], dtype=np.float64).reshape(-1, 2, 2)

def to_int(x):
    return tuple(map(int, x))

im_scale = (image.shape[1], image.shape[0])
lmap = np.zeros(im_scale, dtype=np.float32)

lines[:, :, 0] = np.clip(lines[:, :, 0], 0, im_scale[0] - 1e-4)
lines[:, :, 1] = np.clip(lines[:, :, 1], 0, im_scale[1] - 1e-4)

junc = []
jids = {}

def jid(jun):
    jun = tuple(jun[:2])
    if jun in jids:
        return jids[jun]
    jids[jun] = len(junc)
    junc.append(np.array(jun))
    return len(junc) - 1

lnid = []
lpos = []
for v0, v1 in lines:
    lnid.append((jid(v0), jid(v1)))
    lpos.append([junc[jid(v0)], junc[jid(v1)]])
    rr, cc, value = skimage.draw.line_aa(*to_int(v0), *to_int(v1))
    lmap[rr, cc] = np.maximum(lmap[rr, cc], value)

# Read LCNN, Section "3.5. Line Sampling Module"
lneg = []
divisor = 2
llmap = zoom(lmap, [1 / divisor, 1 / divisor])
lineset = set([frozenset(l) for l in lnid])
for i0, i1 in combinations(range(len(junc)), 2):
    if frozenset([i0, i1]) not in lineset:
        v0, v1 = junc[i0], junc[i1]
        vint0, _vint1 = to_int(v0[:2] / divisor), to_int(v1[:2] / divisor)
        vint0 = (
	    np.clip(_vint0[0], 0, llmap.shape[0] - 1),
	    np.clip(_vint0[1], 0, llmap.shape[1] - 1),
	)
	vint1 = (
	    np.clip(_vint1[0], 0, llmap.shape[0] - 1),
	    np.clip(_vint1[1], 0, llmap.shape[1] - 1),
	)        
        rr, cc, value = skimage.draw.line_aa(*vint0, *vint1)
        lneg.append([v0, v1, i0, i1, np.average(np.minimum(value, llmap[rr, cc]))])
        
assert len(lneg) != 0
lneg.sort(key=lambda l: -l[-1]) 
        
Lneg = np.array([l[2:4] for l in lneg][:4000], dtype=np.int)

train with my own datasets, the results is unstable

Hi,
Thanks for your sharing.
I train with my own datasets. The results are as below:
Two consecutive images are extracted from a video stream. the detected line is unstable, What reason is this?
Thanks a lot.

微信截图_20230217153439
微信截图_20230217153519

line 97 in hawp/parsing/detector.py

line 97 in hawp/parsing/detector.py where ".reshape(128,-1,32)" should probably change to ".reshape(self.dim_loi, -1, self.n_pts0)" in case the dim_loi and n_pts0 are changed to different values.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.