zhiyuanyou / safecount Goto Github PK

When I try to execute the train_val.py , I got this error.

Traceback (most recent call last):
File "tools/train_val.py", line 12, in
from datasets.data_builder import build_dataloader
ModuleNotFoundError: No module named 'datasets'

https://arxiv.org/pdf/2211.08217v1.pdf

@zhiyuanyou could you read through this once. author has explained the logic .
for attention feature they are using I think Transformer encoder layer . Later part not clear to me.
So wanted to understand if there is big or minor difference between your paper and this new one

@zhiyuanyou is it possible to convert this into batch operation currently its a single image based both in training and inference.
1)can this converted to batch based operation.
2) Would increasing the batch size improve the gradients ?

3. I do not have DDP , i.e one gpu only
   can i comment this out in training

if distributed:
        
        dist.all_reduce(train_loss)
        dist.all_reduce(iter)

4. Now I am using backend as resnet50 and doing retraining using its pretrained wts but I get too less training loss ,is it expected

Train | Epoch : 1 / 200 | Iter: 1 / 3659 | lr: 2e-05 | Data: 0.38, Time: 12.12 | Loss: 0.0008869824814610183
 Train | GT: 200.0, Pred: 299.8 | Best Val MAE: inf, Best Val RMSE: inf
 Train | Epoch : 1 / 200 | Iter: 801 / 3659 | lr: 2e-05 | Data: 0.00, Time: 0.18 | Loss: 0.2721385396662299

5. Did you train for 200 epochs ?

Hi, may I know for the FSC147 to CARPK experiment, after I pretrain the model, must I finetune it or can I go straight to eval? If I finetune it, won't the unseen class be exposed to the model? Also, is there support images used during eval or finetuning?

Could you please tell me where are these module in your code?

When I run the file train_val.py, but I meet a error below .How I resolve it?
`Traceback (most recent call last):
File "E:\python code\SAFECount-main\tools\train_val.py", line 325, in
main()
File "E:\python code\SAFECount-main\tools\train_val.py", line 43, in main
with open(args.config) as f:
FileNotFoundError: [Errno 2] No such file or directory: './config.yaml'

Process finished with exit code 1`

Hi, thanks for your good work, I am trying to reproduce your method. When I train by running (./train_torch.sh #Num_GPUS), if I use one GPU (i.e., ./train_torch.sh 1), the network can be trained. However, if I use more gpus (e.g., ./train_torch.sh 2), it will shows that "RuntimeError: Address already in use".
Moreover, I have tried to change the master_port, but it is still wrong.

Hi,

Thank you for sharing wonderful work!

Could you elaborate the performance differences between safecount.py (when initializing fsc146 pre-trained model) and safecount_crossdataset.py ?

I can't create the dataset the FSC147 dataset directory. Because there aren't file 'images_384_VarV2' and 'gt_density_map_adaptive_384_VarV2' in the FSC147 dataset file which I downloade in the offical github website

@zhiyuanyou
Can the model count effectively if there are more than one class objects in image considerably varrying in their shape & color ?

When I run the order that "python train_val.py -t" in the terminal , a error was found .
` File "D:\E-data\python-code\Counting\SAFECount-main\tools\train_val.py", line 302, in eval
visualizer.vis_batch(outputs)
File "D:\E-data\python-code\Counting\SAFECount-main\tools..\utils\vis_helper.py", line 85, in vis_batch
self.vis_result(filename, resname, height, width, output)
File "D:\E-data\python-code\Counting\SAFECount-main\tools..\utils\vis_helper.py", line 58, in vis_result
output = cv2.resize(output, (width, height))
cv2.error: OpenCV(4.7.0) 👎 error: (-5:Bad argument) in function 'resize'

Overload resolution failed:

• Can't parse 'dsize'. Sequence item with index 0 has a wrong type
• Can't parse 'dsize'. Sequence item with index 0 has a wrong type
  `
  Can you tell me how to address it please.

Hello and thanks for the nice work,
I am having trouble reproducing the results of the pretrain phase for cross-dataset generalization on CARPK (MAE: 17.78, RMSE: 20.95). Could you specify which car samples of FSC147 you removed from the training set ? Also could you give more information on how to run the evaluation of the pretrain phase model on CARPK (i.e do you use the config file in the finetune folder) ? Thank you!

Thank you for your work. I would like to understand how to use a pre-trained model to test on other datasets. The pre-trained model can produce normal results on FSC147, but when inference on other datasets that use 'exemplar.json', such as CARPK, the count results for each image are zero. Could you please advise me on how I should use the pre-trained model to test other datasets

Hi @zhiyuanyou Thanks for publishing such an amazing tool. I'm wondering, if it's possible to use pre-trained models (without fine-tuning) for object counting on custom dataset. May I ask you to give some hints for creating a simple demo script for such use cases?

Hello , when I run the order that : 'sh test_torch.sh 1 1', a error was found:
ModuleNotFoundError: No module named 'datasets'
Could you tell me how to address the error , please!

Thanks for your amazing work ! I try to train this code im my own dataset , I generate train.json and test.json. and hvae checked bboxes and point is true , my orginal image size is (1920 ,1080） ， when i train this code, this error info

i debug this code, find error happened in this function rop_roi_feat in models.utils
def crop_roi_feat(feat, boxes, out_stride): """ feat: 1 x c x h x w boxes: m x 4, 4: [y_tl, x_tl, y_br, x_br] """ _, _, h, w = feat.shape boxes_scaled = boxes / out_stride boxes_scaled[:, :2] = torch.floor(boxes_scaled[:, :2]) # y_tl, x_tl: floor boxes_scaled[:, 2:] = torch.ceil(boxes_scaled[:, 2:]) # y_br, x_br: ceil boxes_scaled[:, :2] = torch.clamp_min(boxes_scaled[:, :2], 0) boxes_scaled[:, 2] = torch.clamp_max(boxes_scaled[:, 2], h) boxes_scaled[:, 3] = torch.clamp_max(boxes_scaled[:, 3], w) feat_boxes = [] for idx_box in range(0, boxes.shape[0]): y_tl, x_tl, y_br, x_br = boxes_scaled[idx_box] y_tl, x_tl, y_br, x_br = int(y_tl), int(x_tl), int(y_br), int(x_br) feat_box = feat[:, :, y_tl : (y_br + 1), x_tl : (x_br + 1)] if feat_box.shape[2] == 0: continue feat_boxes.append(feat_box) return feat_boxes
in this function , (y_br + 1) > x_tl is the reason of feat_box.shape[2]=0 , how to solve this , please help me!

Hi @zhiyuanyou, I am following #9 for testing if the SAFECount model can accurately count the number of objects in my own small dataset of ~10 images. While following the previously existing thread, I came into some problems/questions:

Remove the codes about "density" in ./datasets/custom_dataset.py.

1. I am simply removing everything that contains "density", "MAE", and "RMSE", is this the intended behavior?

Remove the codes about "density", "MAE", and "RMSE" in eval() function of ./tools/train_val.py.

2. If so, the eval function in train_val.py returns only val_mae, val_rmse. Do I leave it so that there is no return value?

Create your own dataset about inference, i.e., create a json file like example (but with no need of "density").

3. Do the numbers in the brackets represent the bounding boxes?

Revise config.yaml to use your own dataset, and run python ./tools/train_val.py -e.

4. There are a lot of config.yaml files in the repository. Which one do you want us to revise?

Thanks in advance!

@zhiyuanyou
I am viewing the below density dotted image. How can we interpret which heatmapped dot are likely to be counted.
Currently I do simply output.sum() ,is there any way i can do sum based on some threshold value which indicates high likelyhood of an object count ?

I was able to run your code properly on the FSC dataset and then I made my own small sample dataset to test on using the FSC147 trained weights. I mimicked your FSC147 test.json to generate the json file for my own small sample dataset, and I tried a number of ways to generate it, but it still reported frequent errors related to the json file reading. Since I can run through the FSC147 dataset, I think it should be my json file format problem, can you please provide the code to convert to train.json,val.json,test.json using COCO of FSC147 :). Below is the error report:

Traceback (most recent call last):
File "/home/chuanzhi/zyt/SAFECount-main/experiments/douzhu/../../tools/train_val.py", line 328, in
main()
File "/home/chuanzhi/zyt/SAFECount-main/experiments/douzhu/../../tools/train_val.py", line 147, in main
train_loader, val_loader, test_loader = build_dataloader(
File "/home/chuanzhi/zyt/SAFECount-main/datasets/data_builder.py", line 44, in build_dataloader
test_loader = build(cfg_dataset, dataset_type="test", distributed=distributed)
File "/home/chuanzhi/zyt/SAFECount-main/datasets/data_builder.py", line 22, in build
data_loader = build_custom_dataloader(cfg, training, distributed)
File "/home/chuanzhi/zyt/SAFECount-main/datasets/custom_dataset.py", line 46, in build_custom_dataloader
dataset = CustomDataset(
File "/home/chuanzhi/zyt/SAFECount-main/datasets/custom_dataset.py", line 100, in init
meta = json.loads(line)
File "/home/chuanzhi/anaconda3/lib/python3.9/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/home/chuanzhi/anaconda3/lib/python3.9/json/decoder.py", line 340, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 40 (char 39)
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 267494) of binary: /home/chuanzhi/anaconda3/bin/python
Traceback (most recent call last):
File "/home/chuanzhi/anaconda3/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/chuanzhi/anaconda3/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/chuanzhi/anaconda3/lib/python3.9/site-packages/torch/distributed/launch.py", line 193, in
main()
File "/home/chuanzhi/anaconda3/lib/python3.9/site-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/home/chuanzhi/anaconda3/lib/python3.9/site-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/home/chuanzhi/anaconda3/lib/python3.9/site-packages/torch/distributed/run.py", line 710, in run
elastic_launch(
File "/home/chuanzhi/anaconda3/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/chuanzhi/anaconda3/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 259, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

../../tools/train_val.py FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2023-08-26_18:16:18
host : chuanzhi-MS-7D18
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 267494)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

Hi, I'm trying to run test.sh under /experiments/FSC147, but it shows that
s_p_parse_file: unable to status file /etc/slurm-llnl/slurm.conf

Is it possible to execute on a single PC without using distributed system?
How can I modified to run on my PC?

I appreciate your excellent work.

What confuses me is where the support images are read and fed to network and extract feature?

Thank you for the response.

Hi, in cross-dataset (CARPK) experiment, you pre-train the model on FSC-147, can you provide a pre-train script? Moreover, can you also provide a fine-tune script for easy use? Thanks.

thanks for you greatful work! When i sh train.sh, there are "train.sh: 1: srun: not found
". How can i solve it?

hello, I read your paper, in the paper the support feature(FS) is obtained by applying ROI pooling on the feature extracted from its parent before cropping,but in the code is exemplar.json,which contain coordinate(is ROI?), how to genenrate exemplar.json and what's means. looking forward to your reply,thanks!

We would like to train our own dataset after reproducing your paper. I wonder if you can provide us with the train.json of how to generate our own dataset, and if you can tell us how to get the mask.png file in the mall dataset?

Hi, Nice to see your open source code! but I have a doubt, how should I write a config.yaml file? Can you provide a template of config.yaml?

您好，我的训练数据只有rgb图像以及对应每个目标的一个[x,y]的中心点作为标注，请问exemplar.json这个文件要怎么生成呢？以及这个文件的意义是什么呢？我只想获得每张图片的人群密度

Hello, could you tell me how the density map is generated

Hi, thanks for sharing the source code for your paper.

I am struggling to understand the reason to make the scaled query image size dividable by 16.

Suppose the examplar scale is 0.8 and the query image size is 520 * 520. Its scaled query image size becomes 409.6 * 409.6. Applying int(409.6) can lead to 409 * 409. Your implementation above will lead to 400 * 400. Can you kindly share the reason behind it?

Hi @zhiyuanyou,Can you share the files train.json, val.json, test.json, fold0.json, fold1.json, fold2.json, fold3.json in the FSC-147 dataset to generate code

Hi,

I am trying to understand the implementation of your equation 5, 6 and 7 in your paper, and would be thankful if you can help.

According to your paper, the support feature f_S is flipped before being used to convolve the R. However, there is a 1x1 convolution (line 247 below) applied on the support feature f_S before implementing the equation 5. Are you using the projected support feature f_S for the equation 5?

Additionally, there is another 1x1 convolution (line 261) after implementing the equation 6. Are you using the projected f_R to implement your equation 7?