pytorch=1.7.1 cuda=11.2 GeForce RTX 3080
when i run ~/WIDERFACE_train/predict.py, the error occurs:

File "predict.py", line 22, in
results = config_dict['model'].predict_for_single_image(image, aug_pipeline=simple_widerface_val_pipeline, classification_threshold=0.5, nms_threshold=0.3)
File "/root/zkx/LFD-A-Light-and-Fast-Detector-master/lfd/model/lfd.py", line 634, in predict_for_single_image
nms_bboxes, nms_labels = multiclass_nms(
File "/root/zkx/LFD-A-Light-and-Fast-Detector-master/lfd/model/utils/nms.py", line 214, in multiclass_nms
dets, keep = batched_nms(bboxes, scores, labels, nms_cfg)
File "/root/zkx/LFD-A-Light-and-Fast-Detector-master/lfd/model/utils/nms.py", line 153, in batched_nms
nms_bboxes, kept_indexes = nms_op(torch.cat([bboxes_for_nms, scores[:, None]], -1), **nms_cfg_)
File "/root/zkx/LFD-A-Light-and-Fast-Detector-master/lfd/model/utils/nms.py", line 53, in nms
inds = nms_ext.nms(dets_th, iou_thr)
RuntimeError: CUDA error: no kernel image is available for execution on the device

Could u help? Thank you very much!

Hello, when I use the model for prediction, I find that this warning always appears, but it does not affect the final result. May I ask how should I deal with this warning, or do I not need to deal with it：
[WARNING: ResNet pretrained weights load] unexpected keys:
_neck.neck0.0.weight _neck.neck0.1.weight _neck.neck0.1.bias _neck.neck0.1.running_mean _neck.neck0.1.running_var _neck.neck0.1.num_batches_tracked _neck.neck1.0.weight _neck.neck1.1.weight _neck.neck1.1.bias _neck.neck1.1.running_mean _neck.neck1.1.running_var _neck.neck1.1.num_batches_tracked _neck.neck2.0.weight _neck.neck2.1.weight _neck.neck2.1.bias _neck.neck2.1.running_mean _neck.neck2.1.running_var _neck.neck2.1.num_batches_tracked _neck.neck3.0.weight _neck.neck3.1.weight _neck.neck3.1.bias _neck.neck3.1.running_mean _neck.neck3.1.running_var _neck.neck3.1.num_batches_tracked _neck.neck4.0.weight _neck.neck4.1.weight _neck.neck4.1.bias _neck.neck4.1.running_mean _neck.neck4.1.running_var _neck.neck4.1.num_batches_tracked _head._scales.0._scale _head._scales.1._scale _head._scales.2._scale _head._scales.3._scale _head._scales.4._scale _head.head0_classification_path.0.weight _head.head0_classification_path.0.bias _head.head0_regression_path.0.weight _head.head0_regression_path.0.bias _head.head0_merge_path.0.weight _head.head0_merge_path.1.weight _head.head0_merge_path.1.bias _head.head0_merge_path.3.weight _head.head0_merge_path.4.weight _head.head0_merge_path.4.bias _head.head1_classification_path.0.weight _head.head1_classification_path.0.bias _head.head1_regression_path.0.weight _head.head1_regression_path.0.bias _head.head1_merge_path.0.weight _head.head1_merge_path.1.weight _head.head1_merge_path.1.bias _head.head1_merge_path.3.weight _head.head1_merge_path.4.weight _head.head1_merge_path.4.bias _head.head2_classification_path.0.weight _head.head2_classification_path.0.bias _head.head2_regression_path.0.weight _head.head2_regression_path.0.bias _head.head2_merge_path.0.weight _head.head2_merge_path.1.weight _head.head2_merge_path.1.bias _head.head2_merge_path.3.weight _head.head2_merge_path.4.weight _head.head2_merge_path.4.bias _head.head3_classification_path.0.weight _head.head3_classification_path.0.bias _head.head3_regression_path.0.weight _head.head3_regression_path.0.bias _head.head3_merge_path.0.weight _head.head3_merge_path.1.weight _head.head3_merge_path.1.bias _head.head3_merge_path.3.weight _head.head3_merge_path.4.weight _head.head3_merge_path.4.bias _head.head4_classification_path.0.weight _head.head4_classification_path.0.bias _head.head4_regression_path.0.weight _head.head4_regression_path.0.bias _head.head4_merge_path.0.weight _head.head4_merge_path.1.weight _head.head4_merge_path.1.bias _head.head4_merge_path.3.weight _head.head4_merge_path.4.weight _head.head4_merge_path.4.bias

I have tried to get the result with WIDERFACE evaluation and get the result for LFD_L — Large

Below is the score
==================== Results ====================
Easy Val AP: 0.5960946420764421
Medium Val AP: 0.6995630658106545
Hard Val AP: 0.7579042623709009

These values are quite different from the results given on the repo.

Am I missing something?

The coco bbox format is: [x, y, w, h], then this line should be wrong.

Do LFD supports variable input resolution training or do we need keep all the training images in same size?

xxx

Most detection algorithms don't require negative samples because everything outside bounding box regions is considered "negative". However, looking at some of the example datasets for LFD, generating negative samples explicitly seems to be part of the training data generation process. Can you provide some guidance on the proper way to put together a dataset for training the detector on a set of objects?

Hi, I found that If I set crop_size to 400, then I can not detect large objects. even I large crop_size to 600 later, still can not detect large objects.

Do u know why?

如题

Thanks for sharing great code.
I have a question.

What is the difference between LFFD-Backbone and LFD-Backbone?

You mentioned that LFFD v2 greate improvement on Hard Set. And i want know that:

• What is different between V2 and V1(tips, trick about training process, backbone, data,..) to get this quality ?
  Thank you so much ! Nice research

Hi, thank you very much for your brillant work.
In WIDERFACE_train/evaluation.py ,classification_threshold was set to 0.01, if I do evaluation with this value, I can get the extact mAP 0.866 | 0.877 | 0.839 as you mentioned in README, but it seems there are to many FP results. You may check the following image for reference.

In WIDERFACE_train/predict.py, classification_threshold was set to 0.5, then it can get right result, but when I use this value to do evaluation with LS model, it can only get mAP 0.812/0.794/0.637.
So I think maybe mAP results in LFD for WIDERFACE are reached with an improper parameter. I would be appreciate if you can save image results in evaluation.py with classification_threshold=0.01 and check those results.
Looking forward to your reply.

/lfd/lfd/data_pipeline/dataset/utils/turbojpeg.py", line 455, in
target_libturbojpeg_path = max(libturtojpeg_path_list)
ValueError: max() arg is an empty sequence

I can see in the setup.py file that this is a pure cuda build release but it would be nice to have a cpu version of the code as well.

Thanks.

如何训练自己的模型？

Win10-x64, powershell

I'm trying to use WIDERFACE predict.py, but i have the following error:

ImportError: cannot import name 'nms_ext' from 'lfd.model.utils.libs'

I think this is because of libjpeg-turbo.

Can anyone help me on how to build and compile libjpeg-turbo on windows?

你好，我在构建TensorRT时，运行了以下命令
python predict_tensorrt.py
遇到了以下的报错
......
[TensorRT] VERBOSE: Layer: 501 copy Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: 539 copy Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: 577 copy Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: 615 copy Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: 653 copy Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: 520 copy Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: 558 copy Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: 596 copy Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: 634 copy Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: 672 copy Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Total Host Persistent Memory: 153184
[TensorRT] VERBOSE: Total Device Persistent Memory: 12235264
[TensorRT] VERBOSE: Total Weight Memory: 0
[TensorRT] VERBOSE: Builder timing cache: created 94 entries, 231 hit(s)
[TensorRT] VERBOSE: Engine generation completed in 5.63893 seconds.
[TensorRT] VERBOSE: Calculating Maxima
[TensorRT] INFO: Starting Calibration.
[TensorRT] INFO: Post Processing Calibration data in 9.89e-07 seconds.
Engine build unsuccessfully!
Traceback (most recent call last):
File "predict_tensorrt.py", line 110, in
int8_calibrator=int8_calibrator)
AssertionError

是否能提供接口调用的API文档

只看到有TensorRT deployment 部署，请问onnx导出要运行那个脚本?

what's the difference between lfd-resnet and resnet?

hello, thank you for your work!
Could you tell me your system? When I install pycuda on Centos, there are lots of error about installing, like this:
致命错误：boost/numeric/conversion/detail/preprocessed/numeric_cast_traits_common.hpp：没有那个文件或目录
when I search on the internet, it seems that there is no pycuda for the Centos.
I am confused, so can you tell me you operating system or add it in the wiki?
Thank you very much!

Assertion failed: !isDynamic(tensor_ptr->getDimensions()) && "InstanceNormalization does not support dynamic inputs!".

do u have a solution ?

Hi,

How to select a suitable min_size_threshold for other datasets? Do you have some advice?

Can it add mAP evaluation every several epochs for comparsion ?

Any model design doc or inference result images?

How the model predicts in the video？

To use the quality focal loss did you passed to it the quality score like [0.1 0.6 0.2 0.4] or you passed the onehot from the target like [0 1 0 0] ?

DataParallel的GPU利用率比较低

Hello,

I'm using a small modification of predict_tensorrt.py from the WIDERFACE example (on my own data), which works fine on my x64 linux PC. However, I'm having an issue on jetson xavier. After the first execute_async call, I'm getting the below error on subsequent calls:

python predict_tensorrt.py
Engine info:
	max batch size:  1
	device memory_size:  2417152
before memcpy_htod_async
after memcpy_htod_async
before memcpy_dtod_async
after memcpy_dtod_async
after stream.synchronize()
before memcpy_htod_async
after memcpy_htod_async
[TensorRT] ERROR: 1: [reformat.cu::NCHWToNCQHW4::826] Error Code 1: Cuda Runtime (invalid resource handle)
before memcpy_dtod_async
after memcpy_dtod_async
after stream.synchronize()
before memcpy_htod_async
after memcpy_htod_async
[TensorRT] ERROR: 1: [reformat.cu::NCHWToNCQHW4::826] Error Code 1: Cuda Runtime (invalid resource handle)
before memcpy_dtod_async
after memcpy_dtod_async
after stream.synchronize()

Code from lfd/model/lfd.py

        print("before memcpy_htod_async")
        [cuda.memcpy_htod_async(inp.device, inp.host, stream) for inp in input_buffers]
        print("after memcpy_htod_async")
        tensorrt_engine_context.execute_async(batch_size=1, bindings=bindings, stream_handle=stream.handle)
        print("before memcpy_dtod_async")
        [cuda.memcpy_dtoh_async(out.host, out.device, stream) for out in output_buffers]
        print("after memcpy_dtod_async")
        stream.synchronize()
        print("after stream.synchronize()")

Any insight helpful.

Thanks,
Marc

Win10-x64, Pycharm
Errors in line 455 of turbojpeg.py:

lib_path = os.path.join(os.path.dirname(__file__), 'libs')
libturtojpeg_path_list = [os.path.join(lib_path, file_name) for file_name in os.listdir(lib_path) if file_name.lower().startswith('libturbojpeg')]

# multiple paths may be got, choose the one with the longest chars
target_libturbojpeg_path = max(libturtojpeg_path_list)
turbojpeg = TurboJPEG(lib_path=target_libturbojpeg_path)

error message:
_

target_libturbojpeg_path = max(libturtojpeg_path_list)
ValueError: max() arg is an empty sequence

_
It seems like the lack of some files?

/lfd/data_pipeline/sampler/region_sampler.py", line 300, in crop_from_image
    image[max(0, crop_y):min(im_h, crop_h + crop_y), max(0, crop_x):min(im_w, crop_w + crop_x)]
ValueError: could not broadcast input array from shape (0,400,3) into shape (191,400,3)

Default behavior seems using (crop_size, crop_size) as target size, if set crop_size to 512, it will got this error fianlly (here I set to 400).

Does this a bug?

How's this part contribute to the whole model performance? If we dont generate neg images, how many points will drop in terms of precision and recall?

I didn't understand from what you extract the quality scores

您好，我的显卡是3090，用的是CUDA11.0，在执行 python3 setup.py build_ext 的时候遇到问题：

/home/dddzz/worksapce/Codes/FaceDetection2/LFD-A-Light-and-Fast-Detector-master/venv/lib/python3.8/site-packages/torch/cuda/init.py:104: UserWarning:

GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.

The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70 sm_75.

If you want to use the GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))

:/usr/local/cuda-11.0/bin/nvcc -DWITH_CUDA -I/home/dddzz/worksapce/Codes/FaceDetection2/LFD-A-Light-and-Fast-Detector-master/venv/lib/python3.8/site-packages/torch/include -I/home/dddzz/worksapce/Codes/FaceDetection2/LFD-A-Light-and-Fast-Detector-master/venv/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/dddzz/worksapce/Codes/FaceDetection2/LFD-A-Light-and-Fast-Detector-master/venv/lib/python3.8/site-packages/torch/include/TH -I/home/dddzz/worksapce/Codes/FaceDetection2/LFD-A-Light-and-Fast-Detector-master/venv/lib/python3.8/site-packages/torch/include/THC -I:/usr/local/cuda-11.0/include -I/home/dddzz/worksapce/Codes/FaceDetection2/LFD-A-Light-and-Fast-Detector-master/venv/include -I/usr/include/python3.8 -c lfd/model/utils/build/nms/src/cuda/nms_kernel.cu -o build/temp.linux-x86_64-3.8/lfd/model/utils/build/nms/src/cuda/nms_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=nms_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=sm_86 -std=c++14

unable to execute ':/usr/local/cuda-11.0/bin/nvcc': No such file or directory

error: command ':/usr/local/cuda-11.0/bin/nvcc' failed with exit status 1

请问应该从文件中哪些地方进行修改，来匹配我的硬件设备。非常感谢！

thx this awesome work! I just want to deploy this awesome work in practice, but i find that trt mode consume more GPU and MEM space.The GPU refered above is Tesla T4, i'd appreciate if the prompt repley.

你好，我直接运行了 “WIDERFACE_LFD_XS.py“ 这个脚本，大约一天半的时间完成了1000个epoch的训练。
接下来运行 evaluation.py 脚本获取测试结果。

在使用 Retinaface.pytorch 中提供的 widerface_evaluate 文件夹下面的脚本进行测试，结果仅为：
Easy Val AP: 0.5808434018537905
Medium Val AP: 0.6870650160504126
Hard Val AP: 0.7353618031863371

这个测试结果与您公布的差距比较大。官方的 matlab 脚本我运行以后发现只绘制了3张图，图中也没有 LFD对应的曲线及数值上的指标。

请问我该怎么做，希望得到您的帮助，非常感谢！

WIDERFACE_train

lib_path: F:\Desktop\LFD-A-Light-and-Fast-Detector-master\lfd\data_pipeline\dataset\utils\libs
libturtojpeg_path_list: []
Traceback (most recent call last):
File "F:/Desktop/LFD-A-Light-and-Fast-Detector-master/WIDERFACE_train/predict.py", line 4, in
from lfd.data_pipeline.augmentation import *
File "F:\Desktop\LFD-A-Light-and-Fast-Detector-master\lfd\data_pipeline_init_.py", line 4, in
from .data_loader import *
File "F:\Desktop\LFD-A-Light-and-Fast-Detector-master\lfd\data_pipeline\data_loader_init_.py", line 3, in
from .data_loader import DataLoader
File "F:\Desktop\LFD-A-Light-and-Fast-Detector-master\lfd\data_pipeline\data_loader\data_loader.py", line 8, in
from ..dataset import turbojpeg, reserved_keys
File "F:\Desktop\LFD-A-Light-and-Fast-Detector-master\lfd\data_pipeline\dataset_init_.py", line 7, in
from .utils import *
File "F:\Desktop\LFD-A-Light-and-Fast-Detector-master\lfd\data_pipeline\dataset\utils_init_.py", line 3, in
from .turbojpeg import turbojpeg
File "F:\Desktop\LFD-A-Light-and-Fast-Detector-master\lfd\data_pipeline\dataset\utils\turbojpeg.py", line 457, in
target_libturbojpeg_path = max(libturtojpeg_path_list)
ValueError: max() arg is an empty sequence

Process finished with exit code 1

All the predictions run fine and when the predict.py terminates it generates a 'segmentation fault' error message.

Is the program not supposed to call some proper termination expression to get pytorch terminates properly and in order, e.g. freeing up memory?
All seems to work though, and memory is ultimately released I can see that, but it seems to be done in a brutal way by the OS.

如题

When I showed the warning as shown in the title during training and prediction, I found that my dist.is_initialized() is False. How should I solve it?

Can you put out a number among the best to know where you stand?

Hi @YonghaoHe , thanks for your work!
I'm trying to train charachter detection model using LFD, I'm using traffic sign detector as an example. Which parameters and config do you suggest change to get better result?

hi

thanks for your great project, really cool.

I just got it to work and I was thinking it could deserve a slightly more flexible inference code, e.g. adding command shell parameters rather than modifying the .py file directly to change the model used, or adding batch image file processing or webcam support, tweaking threshold/NMS values, etc.

I ll add some of those in the next few days but I am not a good programmer so just fyi. Happy to share stuff though if that is of any help.

I am on Jetson Xavier NX btw, I ll share some performance stats as well for reference once I got a chance to measure those.

Some notes on the installation for Jetson users:
Note1: I successfully installed and ran the code with pycuda 2019.1.2
Note2: watch out the installation of albumentations => 0.4.6 with pip3 install albumentations, this package somehow automatically forced an update of the opencv module and the numpy module to recent versions which were incompatible with Jetpack. So I had to manually uninstall the installed python-opencv and numpy packages and re-install the correct ones! Not the end of the world but definitely would be a killer for someone not used to those type of hurdles.

As the title said, thanks !

我现在需要将onnx模型转成rknn模型，因此我需要获得预训练模型的归一化参数，这个可以提供吗？

I have made the following changes in the file lfd/deployment/tensorrt/build_engine.py

#assert int8_calibrator is not None, 'calibrator is not provided!'

if precision_mode == 'int8':
    config.int8_calibrator = INT8Calibrator(data='',cache_file='int8_calibrator_cache')#int8_calibrator

I don't understand the data parameter. Can you please help with passing images in this data attribute?

在推理阶段, 图片预处理Normalize占用了大量CPU资源, tensorrt跑推理速度只有使用timing_inference_latency预计的0.5倍.
在网络内有BN的情况下, 可以省略预处理的Normalize. 即在训练时和推理时同时去掉Normalize. 肉眼看性能明显提升, 准召差别不大

As is known to all, it is better using CUDA for the training procedure. However, it is not practical for inference since many edge devices have not a GPU/CUDA hardway, such as my Surface.
In your README file, it is your wish that making LFD popular as YOLO series. So, I suggest that you release a cpu-based (even arm-based) inference version for edge devices, just like the Libfacedetection.

I tried train a single class dataset of hand:
here is the result:

It looks like a lot of false positives.....