Hi, great project!

I'm going through your paper and generating statistics for the SOBA dataset. However, I cannot figure out how you compute the proportion of image space occupied by shadow/object instances (right histogram, figure 3).

By proportion of image space on the "horizontal axis," do you mean something like: box_width / img_width?

Do you have the code so I can take a look?

I really appreciate any help you can provide. Thanks!

Hello,
There are some problem when I run demo.py.

File "/home/ftl/code/InstanceShadowDetection-master/detectron2/layers/deform_conv.py", line 10, in
from detectron2 import _C
ImportError: cannot import name '_C' from 'detectron2' (/home/ftl/code/InstanceShadowDetection-master/detectron2/init.py)

Could you give me some advice?

Regards.

`@RPN_HEAD_REGISTRY.register()
class LISARPNHead(StandardRPNHead):
def init(self, cfg, input_shape: List[ShapeSpec], shadow_object_part=False):
super().init(cfg,input_shape)

    if shadow_object_part:
        in_channels = [s.channels for s in input_shape]
        
        assert len(set(in_channels)) == 1, "Each level must have the same channel!"
        
        in_channels = in_channels[0]
        self.conv = nn.Conv2d(in_channels , in_channels, kernel_size=3, stride=1, padding=1)
        
        for l in [self.conv]:
            nn.init.normal_(l.weight, std=0.01)
            nn.init.constant_(l.bias, 0)

def forward(self, features):
    """
    Args:
        features (list[Tensor]): list of feature maps
    """
    
    pred_objectness_logits = []
    
    pred_anchor_deltas = []

    for i,x in enumerate(features):

        t = F.relu(self.conv(x))
        
        pred_objectness_logits.append(self.objectness_logits(t))
        pred_anchor_deltas.append(self.anchor_deltas(t))
    
    return pred_objectness_logits, pred_anchor_deltas`

I think you duplicate the standard one. Did you check what you implement? The code is terrible. You delete some arguments. Probably, your network is not working. It probably remembers samples. How these papers are accepted to be published :(

What the light and relation numbers mean in SOBA.Such as light [375.5,527.0,375.5,527.0] and relation[288.0,623.5]

It's not clear within the code how to compute/visualize the light direction. How do you do that?

From the output I can see pred_boxes and pred_light. The units of pred_light are not clear - I can see from the paper/code you use the atan2 function but I am not sure how to use those units. Can you elaborate, please?

Hi, I have successfully run your demo and it's very interesting. I want to know how to render a shadow with the estimated light direction and a virtual object. Because we don't know the position of the camera, it may be difficult to conduct 3D rendering.

在LISA_rcnn.py 脚本第94行，生成pred_associations时， self._forward_association_box的参数是proposals， 为什么不是association_proposals

Hello, @stevewongv
Correct me, if I'm wrong. We have only 2 categories, that model could recognize: object and object`s shadow. Model doesn't recognize object by specific categories like persons, cars, dogs, etc. And in case we don't have any shadow on image, but object (person, for example) present, we don't get any predictions at all.

I use your docker image by nvidia-docker and follow your README to run demo.py
but i got "cannot import name '_get_trace_graph' error" in running
"python demo.py --input ./demo/web-shadow0573.jpg --output ./ --config ./config/LISA_101_FPN_3x_demo.yaml"

How can i solve this?

Is there a version available which works with an updated version of pytorch or with a current version of detectron2? Any help is appreciated!

Currently I get the following error while installing the package:

In file included from /home/appuser/ISD/detectron2/layers/csrc/vision.cpp:2:0:
/home/appuser/ISD/detectron2/layers/csrc/ROIAlign/ROIAlign.h: In function 'at::Tensor ROIAlign_forward(const at::Tensor&, const at::Tensor&, float, int, int, int, bool)':
/home/appuser/ISD/detectron2/layers/csrc/ROIAlign/ROIAlign.h:60:18: warning: 'at::DeprecatedTypeProperties& at::Tensor::type() const' is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
if (input.type().is_cuda()) {
^

Hi, I have successfully run the demo.py in the xshell terminal of a remote server. However, when I use pycharm to debug, there is a error occurs:

CUDA error: no kernel image is available for execution on the device (ROIAlign_forward_cuda at /detectron2/detectron2/layers/csrc/ROIAlign/ROIAlign_cuda.cu:359)

I have no idea about that. Could you plz give me some suggestion? I didn't use Docker to set the environment. My pytorch version is py3.6_cuda10.0.130_cudnn7.6.3_0, my cuda has 3 versions of 10.0, 10.1 and 11.1 and I set the version 10.0 in the .bashrc and pycharm configrations. My gpu is Nvidia 2080ti * 4.

你好，我在远程服务器的xshell可以正常运行demo文件。但是当我用pycharm连接服务器远程调试的时候出现了如下报错，

CUDA error: no kernel image is available for execution on the device (ROIAlign_forward_cuda at /detectron2/detectron2/layers/csrc/ROIAlign/ROIAlign_cuda.cu:359)

我查阅了相关资料没有找到出现问题的原因。我没用docker， 直接手动安装了环境。我的pytorch版本是py3.6_cuda10.0.130_cudnn7.6.3_0，我的cuda安装了三个版本，我修改了bashrc里面的环境变量和pycharm运行时的环境变量，但是仍然跑不通。我的显卡是4块2080Ti。

Hi,

I encountered error when running setup.py in InstanceShadowDetection/. The error message I saw was:
error: command 'gcc' failed with exit status 1
In traceback I saw
/InstanceShadowDetection/detectron2/layers/csrc/deformable/deform_conv.h:339:5: error: ‘AT_CHECK’ was not declared in this scope AT_CHECK(input.type().is_cuda(), "input tensor is not on GPU!");

and a note:
/home/arky/files/background_switch/InstanceShadowDetection/detectron2/layers/csrc/deformable/deform_conv.h:339:5: note: suggested alternative: ‘DCHECK’ AT_CHECK(input.type().is_cuda(), "input tensor is not on GPU!"); ^~~~~~~~ DCHECK
It seems that it is from the detectron package but the detectron2 package has updated them to TORCH_CHECK. Not sure what the problem was.

Thank you for sharing this work!

How to integrate into android

Hi,

I am working under Windows Subsytem for Linux (Ubuntu 18.04. LTS). As required, I have installed LISA and pysobatools, and downloaded the model from Google Drive.

I am trying to run demo.py on the image web-shadow0573.jpg but I get the following ImportError:

Traceback (most recent call last):
  File "demo.py", line 16, in <module>
    from predictor import VisualizationDemo
  File "/home/guillaume/InstanceShadowDetection/projects/LISA/predictor.py", line 14, in <module>
    from LISA import add_lisa_config
  File "/home/guillaume/InstanceShadowDetection/projects/LISA/LISA/__init__.py", line 3, in <module>
    from .LISA_meta_arch import LISARCNN
  File "/home/guillaume/InstanceShadowDetection/projects/LISA/LISA/LISA_meta_arch.py", line 10, in <module>
    from detectron2.modeling.postprocessing import detector_postprocess, matchor, combine_association
ImportError: cannot import name 'matchor' from 'detectron2.modeling.postprocessing' (/home/guillaume/miniconda3/envs/detectron2/lib/python3.7/site-packages/detectron2/modeling/postprocessing.py)

I checked the LISA_meta_arch.py and indeed matchor and combine_association are missing.

Any idea how to solve this issue? Thanks!

Guillaume

Hey,
is it possible to run a pre-trained model like they did it in detectron2? link to how they did it:
https://colab.research.google.com/drive/16jcaJoc6bCFAQ96jDe2HwtXj7BMD_-m5#scrollTo=dq9GY37ml1kr
and it worked perfectly fine for me when running just on Windows (no need for cuda, linux or WSL)

I don't need to train the data, just want to run the model on my pictures.
Hope you can help me

pysobatools/_mask.c:10773:5: error: lvalue required as increment operand
This issue occurred during the build process using Python 3.10. I have resolved it in the latest version, but this may result in errors for previous versions of Python. If you encounter this problem, please revert back to an earlier version: https://github.com/stevewongv/InstanceShadowDetection/blob/dacaaea89d9892923d9b44629a9e0bb8ee03528b/PythonAPI/pysobatools/_mask.c

I've successfully pytorch by running pip install torch . Then, I printed successfully the version but the problem No module named torch.library still occured. Can you help please?

When I run demo.py after following instruction, the following error happened.
I guess PythonAPI/pysobatools/cocoeval.py should be renamed to PythonAPI/pysobatools/sobaeval.py ?

$ python demo.py --input ./demo/web-shadow0573.jpg --output ./ --config ./config/LISA_101_FPN_3x_demo.yaml
Traceback (most recent call last):
  File "demo.py", line 16, in <module>
    from predictor import VisualizationDemo
  File "/home3/inoue/experiment/relighting/InstanceShadowDetection/projects/LISA/predictor.py", line 10, in <module>
    from detectron2.engine.defaults import DefaultPredictor
  File "/home3/inoue/experiment/relighting/InstanceShadowDetection/.env/lib/python3.7/site-packages/detectron2-0.1-py3.7-linux-x86_64.egg/detectron2/engine/__init__.py", line 12, in <module>
  File "/home3/inoue/experiment/relighting/InstanceShadowDetection/.env/lib/python3.7/site-packages/detectron2-0.1-py3.7-linux-x86_64.egg/detectron2/engine/defaults.py", line 34, in <module>
    from detectron2.modeling import build_model
  File "/home3/inoue/experiment/relighting/InstanceShadowDetection/.env/lib/python3.7/site-packages/detectron2-0.1-py3.7-linux-x86_64.egg/detectron2/modeling/__init__.py", line 17, in <module>
  File "/home3/inoue/experiment/relighting/InstanceShadowDetection/.env/lib/python3.7/site-packages/detectron2-0.1-py3.7-linux-x86_64.egg/detectron2/modeling/meta_arch/__init__.py", line 6, in <module>
  File "/home3/inoue/experiment/relighting/InstanceShadowDetection/.env/lib/python3.7/site-packages/detectron2-0.1-py3.7-linux-x86_64.egg/detectron2/modeling/meta_arch/panoptic_fpn.py", line 10, in <module>
  File "/home3/inoue/experiment/relighting/InstanceShadowDetection/.env/lib/python3.7/site-packages/detectron2-0.1-py3.7-linux-x86_64.egg/detectron2/modeling/postprocessing.py", line 8, in <module>
ModuleNotFoundError: No module named 'pysobatools.sobaeval'

hi Steve! thank you for this repo as I'm really interested in it. I just wonder whether there is any workaround for pysobatools to be installing on Windows?

hi @stevewongv, this is a very meaningful job. I have a question to ask. Is 'pred_light' a quaternion? Is it in the order of 'wxyz'? What kind of coordinate system is the result in?