ubicomplab / rppg-toolbox Goto Github PK

rPPG-Toolbox: Deep Remote PPG Toolbox (NeurIPS 2023)

Home Page: https://arxiv.org/abs/2210.00716

License: Other

Python 78.33% Shell 0.19% Jupyter Notebook 21.48%

camera health mobile-health physiological-computing ppg remote-ppg rppg health-sensing mobilehealth physiologicalsensing

rppg-toolbox's People

Contributors

Stargazers

Watchers

Forkers

xiaoyu-sz zyzzzz-123 nancylyj zhangys2019 yiranzhang69 mauricerohr jackzhousz mcjacktang crocecanale yahskapar ycyoon ouyuxuanbridge bnucsy comicxmz001 vulus98 neilm21 ghulam-nabii kokilayadav chg0901 gaoxu007 harshithshankar13 trieuhaivo aaaaaabaaba gongbang02 prateekiitr laurasetnof ravcare pedrombmachado peterzs b-acharya kentee xu-xh cjr4702 gogoho88 jvd9kh96 lhw0772 linhuaiyi sunlifev felipekohut laadjal07 pwtll orishub chainshoetall saiakhil19932 peizhengfu yz2545 dsamuelhodge greatyang wjulyw ankitaggarwal1 mmm84766 abdelrhmanelmahdy trishnasaikia abrahammmmmmmmmm adamh18 alolohk physiologicailab jvp-labs lboxiang eluctra pi7orubkeua yuer629 dalaoplan sugaoji andychak hopern dglzj harry2345 harring 07sarthak jbarber07 vineetp6 petterlopes changzhiy ipsych zackary-lykos arritmic moon-ligh schengal4 health-universe ialzyoud domizianoscarcelli linnkyawswaraung jintengl marshb tjl081 becuriouss siriusyh

rppg-toolbox's Issues

Issue about setting up

Hi, I'd like to ask if there is a way to set up with pip install?

Hi,for the Physnet model, the input of the network mentioned by the author in the original text is the original video frame, but the frame difference normalization method is used in the yaml file of config. I don't know whether this will affect the test results

Questions about the training results

Dear authors,

Thank you for this toolbox which really helps a lot.

I followed the instructions to train the TSCAN on PURE dataset and tested on the UBFC dataset. The whole propress went well without error. However, I feel confused about the final testing results. I got
FFT MAE (FFT Label):2.7413504464285716
FFT RMSE (FFT Label):10.761795973358529
FFT MAPE (FFT Label):2.5531960308525843
FFT Pearson (FFT Label):0.8154277192381875
However, in the Deep Physiological Sensing Toolbox paper, the expected results are MAE 0.99 RMSE 2.41 MAPE 1.17 Pearson 0.99. I have no idea what causes such as difference. I only modified the data path, cache path, and do procress operation in the PURE_PURE_UBFC_TSCAN_BASIC.yaml file. All the parameters remain unchanged.
The only difference I can find is the platform configuration. I ran the codes on our GPU server which has the following configuration. However, since the codes can run succefully, the different configuration should not matter. Would it be possible to provide any ideas about the accuracy gap? Thank you.

Best,
Yue

Collecting environment information...
PyTorch version: 1.7.0
Is debug build: True
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: CentOS Linux release 8.2.2004 (Core) (x86_64)
GCC version: (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5)
Clang version: 9.0.1 (Red Hat 9.0.1-2.module_el8.2.0+309+0c7b6b03)
CMake version: version 3.11.4

Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: Tesla V100-SXM2-32GB
Nvidia driver version: 470.57.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] advertorch==0.2.3
[pip3] numpy==1.19.5
[pip3] torch==1.7.0
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.8.0
[conda] advertorch 0.2.3 pypi_0 pypi
[conda] blas 1.0 mkl
[conda] cudatoolkit 10.1.243 h6bb024c_0
[conda] mkl 2021.3.0 h06a4308_520
[conda] mkl-service 2.4.0 py37h7f8727e_0
[conda] mkl_fft 1.3.0 py37h42c9631_2
[conda] mkl_random 1.2.2 py37h51133e4_0
[conda] numpy 1.18.5 pypi_0 pypi
[conda] torch 1.7.0 pypi_0 pypi
[conda] torchsummary 1.5.1 pypi_0 pypi
[conda] torchvision 0.8.0 pypi_0 pypi

Can we infer the pre-trained model on macbook? (only cpu available)

First, thank you for provide such useful toolbox for rPPG field.

As title, I am currently experiment with TSCAN model by using UBFC dataset on my macbook. In order to run model on cpu:

I set the DEVICE from 'cuda: 0' to 'cpu' in PURE_UBFC-rPPG_TSCAN_BASIC.yaml
I try to load the model with the following change in TscanTrainer.py:

In the test function, I have modified the load model method from: self.model.load_state_dict(torch.load(self.config.INFERENCE.MODEL_PATH)) to self.model.load_state_dict(torch.load(self.config.INFERENCE.MODEL_PATH, map_location=torch.device(self.device)))

My questions are:

Except the above modification, is there anything left I need to modify to run model on cpu?
When run model on cpu, will the infer result be influenced?

Native Windows UBFC Testing gives an Error due to path splitting in DataLoader

Good afternoon,

Thank you for this toolbox first off, it was a struggle finding a deep learning rPPG repo with established dataloaders for the main public datasets.

I was running a train, test cycle on DeepPhys using the SCAMPS_SCAMPS_UBFC_DEEPPHYS_BASIC yaml file and I noticed that an error was popping up when I got to the inference phase, both in train and test and test_only mode. You may find the stack trace below.

I've run the pipeline through Pycharm's debugger and I believe it's down to the getitem dunder function in the BaseLoader class. It looks like on Windows, the "\" characters implicitly inserted when the program creates directories in the preprocessing phase are causing the issue, since the split() function only considers the "/" character (assuming this is based on a Linux distro or similar). Can you confirm whether I've tackled it correctly? I'm quite new to PyTorch and the repo in general and want to make sure I'm working correctly. I did manage to get the MAE etc. by replacing with the single slash.

If the approach is correct holistically, it's only a matter of inserting an extra line for the replace function for Windows users.

Trace of original getitem function on Native Windows in Pycharm

`Traceback (most recent call last):
File "C:\NIVS Project\GitHub Repos\rPPG-Toolbox-main\main.py", line 243, in
test(config, data_loader_dict)
File "C:\NIVS Project\GitHub Repos\rPPG-Toolbox-main\main.py", line 85, in test
model_trainer.test(data_loader_dict)
File "C:\NIVS Project\GitHub Repos\rPPG-Toolbox-main\neural_methods\trainer\DeepPhysTrainer.py", line 136, in test
sort_index = int(test_batch[3][idx])
ValueError: invalid literal for int() with base 10: '72_SizeH72_ClipLength180_DataTypeNormalized_Standardized_LabelTypeNormalized_Large_boxTrue_Large_size1'

Process finished with exit code 1
`

How to use HSV instead of RGB in TS-CAN algorithm

Hi, Can someone please help me this. I wanted to use HSV in TS-CAN algorithm with MMPD datasets, Instead of using RGB. I am using PURE_MMPD_TSCAN_BASIC.yaml

For the dataset part

Hi, I do think you guys support 4 types of datasets. Please revise it in the readme.

RuntimeError: Trying to resize storage that is not resizable

Error While running the below command

Command executed in terminal

python main.py --config_file ./configs/infer_configs/PURE_UBFC-rPPG_EFFICIENTPHYS.yaml

Terminal output

=> Merging a config file from ./configs/infer_configs/PURE_UBFC-rPPG_EFFICIENTPHYS.yaml
Configuration:
=> Merging a config file from ./configs/infer_configs/PURE_UBFC-rPPG_EFFICIENTPHYS.yaml
Configuration:

BASE: ['']
DEVICE: cpu
INFERENCE:
  BATCH_SIZE: 32
  EVALUATION_METHOD: FFT
  EVALUATION_WINDOW:
    USE_SMALLER_WINDOW: False
    WINDOW_SIZE: 10
  MODEL_PATH: ./final_model_release/PURE_EfficientPhys.pth
LOG:
  PATH: runs/exp/
MODEL:
  BIGSMALL:
    FRAME_DEPTH: 3
  DROP_RATE: 0.2
  EFFICIENTPHYS:
    FRAME_DEPTH: 10
  MODEL_DIR: PreTrainedModels/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
  NAME: EfficientPhys
  PHYSNET:
    FRAME_NUM: 64
  RESUME: 
  TSCAN:
    FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxTrue
    DATASET: UBFC-rPPG
    DATA_FORMAT: NDCHW
    DATA_PATH: RawData
    DO_PREPROCESS: True
    END: 1.0
    EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxTrue
    FILE_LIST_PATH: PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxTrue_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: True
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['Standardized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False
  METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR']
  USE_LAST_EPOCH: True
TOOLBOX_MODE: only_test
TRAIN:
  BATCH_SIZE: 4
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    DATASET: 
    DATA_FORMAT: NDCHW
    DATA_PATH: 
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 0
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['']
      DO_CHUNK: True
      LABEL_TYPE: 
      RESIZE:
        H: 128
        W: 128
      USE_PSUEDO_PPG_LABEL: False
  EPOCHS: 50
  LR: 0.0001
  MODEL_FILE_NAME: 
  OPTIMIZER:
    BETAS: (0.9, 0.999)
    EPS: 0.0001
    MOMENTUM: 0.9
UNSUPERVISED:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    DATASET: 
    DATA_FORMAT: NDCHW
    DATA_PATH: 
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 0
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['']
      DO_CHUNK: True
      LABEL_TYPE: 
      RESIZE:
        H: 128
        W: 128
      USE_PSUEDO_PPG_LABEL: False
  METHOD: []
  METRICS: []
VALID:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData
    DATASET: 
    DATA_FORMAT: NDCHW
    DATA_PATH: 
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: 
    FILE_LIST_PATH: PreprocessedData/DataFileLists
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 0
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['']
      DO_CHUNK: True
      LABEL_TYPE: 
      RESIZE:
        H: 128
        W: 128
      USE_PSUEDO_PPG_LABEL: False

Initializing UBFC-rPPG dataset...
data_path: RawData
Preprocessing dataset...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [01:07<00:00, 33.54s/it]
Total Number of raw files preprocessed: 2

Cached Data Path PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxTrue

File List Path PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxTrue_0.0_1.0.csv
test Preprocessed Dataset Length: 18

===Testing===
Testing uses pretrained model!
Traceback (most recent call last):
File "main.py", line 288, in
test(config, data_loader_dict)
File "main.py", line 95, in test
model_trainer.test(data_loader_dict)
File "/Users/vikashpr/Dev/Python/rPPG-Toolbox/neural_methods/trainer/EfficientPhysTrainer.py", line 171, in test
for _, test_batch in enumerate(data_loader['test']):
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 633, in next
data = self._next_data()
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1345, in _next_data
return self._process_data(data)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
data.reraise()
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/_utils.py", line 644, in reraise
raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
data = fetcher.fetch(index)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch
return self.collate_fn(data)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
return collate(batch, collate_fn_map=default_collate_fn_map)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 142, in collate
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 142, in
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 119, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 171, in collate_numpy_array_fn
return collate([torch.as_tensor(b) for b in batch], collate_fn_map=collate_fn_map)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 119, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/utils/collate.py", line 161, in collate_tensor_fn
out = elem.new(storage).resize(len(batch), *list(elem.size()))
RuntimeError: Trying to resize storage that is not resizable

I am using MacBook Air 2017 i5 5th generation. My system don't have CUDA. Please help me resolve this error.

Error occuring when running pretrained model trained on PURE and tested on UBFC with PhysNet

Preprocessing dataset...
  0%|                                                                                                                                                                                       | 0/2 [00:00<?, ?it/s]Warning: More than one faces are detected(Only cropping the biggest one.)
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:35<00:00, 17.57s/it]
Total Number of raw files preprocessed: 2

Cached Data Path C:\Users\bsing.BHUPI\Desktop\pyt\rPPG-Toolbox-main\cached\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse

File List Path C:\Users\bsing.BHUPI\Desktop\pyt\rPPG-Toolbox-main\cached\DataFileLists\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
 test Preprocessed Dataset Length: 27

C:\Users\bsing.BHUPI\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\utils\data\dataloader.py:560: UserWarning: This DataLoader will create 16 worker processes in total. Our suggested max number of worker in current system is 12 (`cpuset` is not taken into account), which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  warnings.warn(_create_warning_msg(

===Testing===
Testing uses pretrained model!
./final_model_release/PURE_PhysNet_DiffNormalized.pth

100%|█████████████████████████████████████████████| 2/2 [00:01<00:00,  1.10it/s]
FFT MAE (FFT Label): 0.0 +/- 0.0
FFT RMSE (FFT Label): 0.0 +/- 0.0
FFT MAPE (FFT Label): 0.0 +/- 0.0
C:\Users\bsing.BHUPI\anaconda3\envs\rppg-toolbox\lib\site-packages\numpy\lib\function_base.py:2829: RuntimeWarning: invalid value encountered in true_divide
  c /= stddev[:, None]
FFT Pearson (FFT Label): nan +/- nan
FFT SNR (FFT Label): -1.496827352337295 +/- 1.9491494704719077 (dB)

About gt_hr when using FFT method

hi~

I've observed that the gt_hr values obtained via the following two methods exhibit inconsistencies(on UBFC dataset).

default method: read_wave()--->calculate_metric_per_video--->gt_hr_fft_all
by reading the FFT column in the wip/label/comparison_UBFC.csv file.

However, for certain subjects, the results derived from these two methods are inconsistent.

	default	comparison_UBFC.csv
subject 8	112.5	101.07
subject 10	119.53125	108.984375
subject 14	77.34375	70.3125
subject 16	94.48242188	90.96679688
subject 24	95.80078125	87.01171875

What factors contribute to this difference?
How are the manually corrected values in comparison_UBFC.csv derived?
When executing the program, which reference value should be selected?

thanks!

Has anyone successfully obtained reasonable results from supervised neural models on the UBFC-rPPG dataset or own recorded video?

Hi,

I am currently experimenting with supervised neural models on my Mac (CPU only). I have tried using the TSCAN and DeepPhys models to test the UBFC-rPPG dataset. However, I am facing the following problems:

The inference time of the model is very slow, taking approximately 30 seconds for each test video to output predicted results (excluding preprocessing time). Could you please advise me on how to speed up the inference speed of the model? Does the length of the video affect the inference time? For example, if I input a 1-minute video and a 10-minute video, will the model's inference time be different?
The output ground truth and predicted results are always constant values. Regardless of the models I try, the subjects I use, or changing the evaluation window size, I always obtain constant HR (Heart Rate) values. I have no idea where to trace the issues. Could someone provide me with some guidance?
If someone has not encountered the above problems, could you please share how you have achieved that? I would greatly appreciate it!

Thanks.

Real time rPPG extraction through camera feed.

is there any real-time deep learning repo that measures heart rate pulse?

Problems encountered while preprocessing the model on Windows

Dear Team,
I am trying to use the toolbox, but I keep getting the error below. I am training and testing on the UBFC-rPPG dataset.

Warning: More than one faces are detected(Only cropping the biggest one.)
Process Process-2:
Traceback (most recent call last):
  File "D:\Anaconda\envs\rppg-toolbox\lib\multiprocessing\process.py", line 313, in _bootstrap
    self.run()
  File "D:\Anaconda\envs\rppg-toolbox\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "G:\Project\Fatigue\program\rPPG-Toolbox-main\dataset\data_loader\UBFCrPPGLoader.py", line 92, in preprocess_dataset_subprocess
    input_name_list, label_name_list = self.save_multi_process(frames_clips, bvps_clips, saved_filename)
  File "G:\Project\Fatigue\program\rPPG-Toolbox-main\dataset\data_loader\BaseLoader.py", line 404, in save_multi_process
    os.makedirs(self.cached_path, exist_ok=True)
  File "D:\Anaconda\envs\rppg-toolbox\lib\os.py", line 221, in makedirs
    mkdir(name, mode)
FileNotFoundError: [WinError 3] 系统找不到指定的路径。: 'G:/Project/Fatigue/program/rPPG-Toolbox-main/PreprocessedData\\UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse'
  8%|▊         | 1/13 [00:10<02:08, 10.69s/it]Process Process-8:
Traceback (most recent call last):
  File "D:\Anaconda\envs\rppg-toolbox\lib\multiprocessing\process.py", line 313, in _bootstrap
    self.run()
  File "D:\Anaconda\envs\rppg-toolbox\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "G:\Project\Fatigue\program\rPPG-Toolbox-main\dataset\data_loader\UBFCrPPGLoader.py", line 92, in preprocess_dataset_subprocess
    input_name_list, label_name_list = self.save_multi_process(frames_clips, bvps_clips, saved_filename)
  File "G:\Project\Fatigue\program\rPPG-Toolbox-main\dataset\data_loader\BaseLoader.py", line 404, in save_multi_process
    os.makedirs(self.cached_path, exist_ok=True)
  File "D:\Anaconda\envs\rppg-toolbox\lib\os.py", line 221, in makedirs
    mkdir(name, mode)
FileNotFoundError: [WinError 3] 系统找不到指定的路径。: 'G:/Project/Fatigue/program/rPPG-Toolbox-main/PreprocessedData\\UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse'
Process Process-4:

YAML Configuration file

BASE: ['']
TOOLBOX_MODE: "only_test"      # "train_and_test"  or "only_test"
TEST:
  METRICS: ['MAE','RMSE','MAPE','Pearson', 'SNR']
  USE_LAST_EPOCH: True
  DATA:
    FS: 30
    DATASET: UBFC-rPPG
    DO_PREPROCESS: True                    # if first time, should be true
    DATA_FORMAT: NDCHW
    DATA_PATH: "G:/Project/Fatigue/dataset/UBFC-rPPG/RawData"                     # Raw dataset path, need to be updated
    CACHED_PATH: "G:/Project/Fatigue/program/rPPG-Toolbox-main/PreprocessedData"    # Processed dataset save path, need to be updated
    EXP_DATA_NAME: ""
    BEGIN: 0.0
    END: 1.0
    PREPROCESS:
      DATA_TYPE: [ 'DiffNormalized','Standardized' ]
      LABEL_TYPE: DiffNormalized
      DO_CHUNK: True
      CHUNK_LENGTH: 180
      CROP_FACE:
        DO_CROP_FACE: True
        USE_LARGE_FACE_BOX: True
        LARGE_BOX_COEF: 1.5
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY : 30
          USE_MEDIAN_FACE_BOX: False    # This should be used ONLY if dynamic detection is used
      RESIZE:
        H: 72
        W: 72
DEVICE: cuda:0
NUM_OF_GPU_TRAIN: 1
LOG:
  PATH: runs/exp
MODEL:
  DROP_RATE: 0.2
  NAME: Tscan
  TSCAN:
    FRAME_DEPTH: 10
INFERENCE:
  BATCH_SIZE: 4
  EVALUATION_METHOD: FFT        # "FFT" or "peak detection"
  EVALUATION_WINDOW:
    USE_SMALLER_WINDOW: False        # Change this if you'd like an evaluation window smaller than the test video length
    WINDOW_SIZE: 10        # In seconds
  MODEL_PATH: "./final_model_release/PURE_TSCAN.pth"

RuntimeError: Caught RuntimeError in DataLoader worker process 0 and Trying to resize storage that is not resizable

Executed command

 python main.py --config_file ./configs/infer_configs/PURE_UBFC-rPPG_EFFICIENTPHYS.yaml

Output

=> Merging a config file from ./configs/infer_configs/PURE_UBFC-rPPG_EFFICIENTPHYS.yaml
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
  BATCH_SIZE: 4
  EVALUATION_METHOD: FFT
  EVALUATION_WINDOW:
    USE_SMALLER_WINDOW: False
    WINDOW_SIZE: 10
  MODEL_PATH: ./final_model_release/PURE_EfficientPhys.pth
LOG:
  PATH: runs/exp/
MODEL:
  BIGSMALL:
    FRAME_DEPTH: 3
  DROP_RATE: 0.2
  EFFICIENTPHYS:
    FRAME_DEPTH: 10
  MODEL_DIR: PreTrainedModels/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
  NAME: EfficientPhys
  PHYSNET:
    FRAME_NUM: 64
  RESUME:
  TSCAN:
    FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    DATASET: UBFC-rPPG
    DATA_FORMAT: NDCHW
    DATA_PATH: data/RawData/UBFC
    DO_PREPROCESS: True
    END: 1.0
    EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    FILE_LIST_PATH: PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['Standardized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False
  METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR']
  USE_LAST_EPOCH: True
TOOLBOX_MODE: only_test
TRAIN:
  BATCH_SIZE: 4
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    DATASET:
    DATA_FORMAT: NDCHW
    DATA_PATH:
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 0
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['']
      DO_CHUNK: True
      LABEL_TYPE:
      RESIZE:
        H: 128
        W: 128
      USE_PSUEDO_PPG_LABEL: False
  EPOCHS: 50
  LR: 0.0001
  MODEL_FILE_NAME:
  OPTIMIZER:
    BETAS: (0.9, 0.999)
    EPS: 0.0001
    MOMENTUM: 0.9
UNSUPERVISED:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    DATASET:
    DATA_FORMAT: NDCHW
    DATA_PATH:
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 0
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['']
      DO_CHUNK: True
      LABEL_TYPE:
      RESIZE:
        H: 128
        W: 128
      USE_PSUEDO_PPG_LABEL: False
  METHOD: []
  METRICS: []
VALID:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData
    DATASET:
    DATA_FORMAT: NDCHW
    DATA_PATH:
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME:
    FILE_LIST_PATH: PreprocessedData/DataFileLists
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 0
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['']
      DO_CHUNK: True
      LABEL_TYPE:
      RESIZE:
        H: 128
        W: 128
      USE_PSUEDO_PPG_LABEL: False

Preprocessing dataset...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:28<00:00,  3.58s/it]
Total Number of raw files preprocessed: 8

Cached Data Path PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse

File List Path PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_1.0.csv
 test Preprocessed Dataset Length: 94


===Testing===
Testing uses pretrained model!
Traceback (most recent call last):
  File "main.py", line 289, in <module>
    test(config, data_loader_dict)
  File "main.py", line 96, in test
    model_trainer.test(data_loader_dict)
  File "/home/bioinformaticslab/Desktop/Python/rPPG-Toolbox-main/neural_methods/trainer/EfficientPhysTrainer.py", line 171, in test
    for _, test_batch in enumerate(data_loader['test']):
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 633, in __next__
    data = self._next_data()
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1345, in _next_data
    return self._process_data(data)
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
    data.reraise()
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/_utils.py", line 644, in reraise
    raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch
    return self.collate_fn(data)
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
    return collate(batch, collate_fn_map=default_collate_fn_map)
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 142, in collate
    return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed]  # Backwards compatibility.
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 142, in <listcomp>
    return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed]  # Backwards compatibility.
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 119, in collate
    return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 171, in collate_numpy_array_fn
    return collate([torch.as_tensor(b) for b in batch], collate_fn_map=collate_fn_map)
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 119, in collate
    return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
  File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 161, in collate_tensor_fn
    out = elem.new(storage).resize_(len(batch), *list(elem.size()))
RuntimeError: Trying to resize storage that is not resizable

nvidia-smi

Wed Jul  5 12:55:53 2023      
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.54.03              Driver Version: 535.54.03    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA RTX A5000               Off | 00000000:02:00.0  On |                  Off |
| 30%   37C    P8              24W / 230W |    392MiB / 24564MiB |      3%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      1761      G   /usr/lib/xorg/Xorg                          169MiB |
|    0   N/A  N/A      1962      G   /usr/bin/gnome-shell                         43MiB |
|    0   N/A  N/A     90367      G   ...sion,SpareRendererForSitePerProcess       79MiB |
|    0   N/A  N/A     90720      G   ...0210443,15007061678920115541,262144       62MiB |
+---------------------------------------------------------------------------------------+

conda list
# packages in environment at /home/bioinformaticslab/anaconda3/envs/rppg-toolbox:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
blas                      1.0                         mkl  
brotlipy                  0.7.0           py38h27cfd23_1003  
bzip2                     1.0.8                h7b6447c_0  
ca-certificates           2023.05.30           h06a4308_0  
certifi                   2023.5.7         py38h06a4308_0  
cffi                      1.15.1           py38h5eee18b_3  
charset-normalizer        2.0.4              pyhd3eb1b0_0  
cmake                     3.26.4                   pypi_0    pypi
cryptography              39.0.1           py38h9ce1e76_2  
cudatoolkit               10.2.89              h8f6ccaa_8    nvidia
cycler                    0.11.0                   pypi_0    pypi
ffmpeg                    4.3                  hf484d3e_0    pytorch
filelock                  3.12.2                   pypi_0    pypi
freetype                  2.12.1               h4a9f257_0  
giflib                    5.2.1                h5eee18b_3  
gmp                       6.2.1                h295c915_3  
gnutls                    3.6.15               he1e5248_0  
h5py                      2.10.0                   pypi_0    pypi
idna                      3.4              py38h06a4308_0  
imageio                   2.31.1                   pypi_0    pypi
intel-openmp              2023.1.0         hdb19cb5_46305  
jinja2                    3.1.2                    pypi_0    pypi
joblib                    1.3.1                    pypi_0    pypi
jpeg                      9e                   h5eee18b_1  
kiwisolver                1.4.4                    pypi_0    pypi
lame                      3.100                h7b6447c_0  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.38                 h1181459_1  
lerc                      3.0                  h295c915_0  
libdeflate                1.17                 h5eee18b_0  
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libiconv                  1.16                 h7f8727e_2  
libidn2                   2.3.4                h5eee18b_0  
libpng                    1.6.39               h5eee18b_0  
libstdcxx-ng              11.2.0               h1234567_1  
libtasn1                  4.19.0               h5eee18b_0  
libtiff                   4.5.0                h6a678d5_2  
libunistring              0.9.10               h27cfd23_0  
libwebp                   1.2.4                h11a3e52_1  
libwebp-base              1.2.4                h5eee18b_1  
lit                       16.0.6                   pypi_0    pypi
lz4-c                     1.9.4                h6a678d5_0  
markupsafe                2.1.3                    pypi_0    pypi
mat73                     0.59                     pypi_0    pypi
matplotlib                3.1.2                    pypi_0    pypi
mkl                       2023.1.0         h6d00ec8_46342  
mkl-service               2.4.0            py38h5eee18b_1  
mkl_fft                   1.3.6            py38h417a72b_1  
mkl_random                1.2.2            py38h417a72b_1  
mpmath                    1.3.0                    pypi_0    pypi
ncurses                   6.4                  h6a678d5_0  
nettle                    3.7.3                hbbd107a_1  
networkx                  3.1                      pypi_0    pypi
numpy                     1.22.0                   pypi_0    pypi
nvidia-cublas-cu11        11.10.3.66               pypi_0    pypi
nvidia-cuda-cupti-cu11    11.7.101                 pypi_0    pypi
nvidia-cuda-nvrtc-cu11    11.7.99                  pypi_0    pypi
nvidia-cuda-runtime-cu11  11.7.99                  pypi_0    pypi
nvidia-cudnn-cu11         8.5.0.96                 pypi_0    pypi
nvidia-cufft-cu11         10.9.0.58                pypi_0    pypi
nvidia-curand-cu11        10.2.10.91               pypi_0    pypi
nvidia-cusolver-cu11      11.4.0.1                 pypi_0    pypi
nvidia-cusparse-cu11      11.7.4.91                pypi_0    pypi
nvidia-nccl-cu11          2.14.3                   pypi_0    pypi
nvidia-nvtx-cu11          11.7.91                  pypi_0    pypi
opencv-python             4.5.2.54                 pypi_0    pypi
openh264                  2.1.1                h4ff587b_0  
openssl                   3.0.9                h7f8727e_0  
pandas                    1.1.5                    pypi_0    pypi
pillow                    9.4.0            py38h6a678d5_0  
pip                       23.1.2           py38h06a4308_0  
protobuf                  4.23.3                   pypi_0    pypi
pycparser                 2.21               pyhd3eb1b0_0  
pyopenssl                 23.0.0           py38h06a4308_0  
pyparsing                 3.1.0                    pypi_0    pypi
pysocks                   1.7.1            py38h06a4308_0  
python                    3.8.16               h955ad1f_4  
python-dateutil           2.8.2                    pypi_0    pypi
pytorch-mutex             1.0                        cuda    pytorch
pytz                      2023.3                   pypi_0    pypi
pywavelets                1.4.1                    pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
readline                  8.2                  h5eee18b_0  
requests                  2.29.0           py38h06a4308_0  
scikit-image              0.17.2                   pypi_0    pypi
scikit-learn              1.0.2                    pypi_0    pypi
scipy                     1.5.2                    pypi_0    pypi
setuptools                67.8.0           py38h06a4308_0  
six                       1.16.0                   pypi_0    pypi
sqlite                    3.41.2               h5eee18b_0  
sympy                     1.12                     pypi_0    pypi
tbb                       2021.8.0             hdb19cb5_0  
tensorboardx              2.4.1                    pypi_0    pypi
threadpoolctl             3.1.0                    pypi_0    pypi
tifffile                  2023.4.12                pypi_0    pypi
tk                        8.6.12               h1ccaba5_0  
torch                     2.0.1                    pypi_0    pypi
torchaudio                0.12.1               py38_cu102    pytorch
torchvision               0.15.2                   pypi_0    pypi
tqdm                      4.64.0                   pypi_0    pypi
triton                    2.0.0                    pypi_0    pypi
typing_extensions         4.6.3            py38h06a4308_0  
urllib3                   1.26.16          py38h06a4308_0  
wheel                     0.38.4           py38h06a4308_0  
xz                        5.4.2                h5eee18b_0  
yacs                      0.1.8                    pypi_0    pypi
zlib                      1.2.13               h5eee18b_0  
zstd                      1.5.5                hc292b87_0

Github repo: Link

Please ignore all the previous commits made in the repo, we have completely reverted our old commits where we tried to run the algo using CPU #176, we are now trying to run rPPG-Toolbox using the above configurations and facing issues, can someone please help us resolve this error

How to download the UBFC-rPPG Dataset?

May I know how to download the UBFC-rPPG dataset from this link?
I have try to contact with the authors for requesting the dataset, but I didn't get the reply.

The color channel order of images saved in synthetic dataset is 'BGR', which is inconsistent with other dataset.

The images in other dataset (UBFC and PURE) is read with the 'RGB'.

ValueError: invalid literal for int() with base 10: '72_SizeH72_ClipLength180_DataTypeNormalized_Standardized_LabelTypeNormalized_Large_boxTrue_Large_size1'

RuntimeError in TS_CAN.py

Hi,

I want to use the pretrained TSCAN model to get the rppg data from my own videos, and the shape of my input to the model is [300, 3, 128, 128]. But, I am getting a runtime error given below:

RuntimeError: Given groups=1, weight of size [32, 3, 3, 3], expected input[300, 0, 128, 128] to have 3 channels, but got 0 channels instead

The error is occurring at /neural_methods/model/TS_CAN.py", line 127, in forward
r1 = torch.tanh(self.apperance_conv1(raw_input))

Because in line 120 in TS_CAN.py, where we get the raw_data, i.e., raw_input = inputs[:, 3:, :, :], it is getting raw_input which is of shape [300, 0, 128, 128]. Could you tell me what is the expected shape of raw_input here?

My code is given below:

import torch
from neural_methods.model.TS_CAN import TSCAN
from config import get_config
import cv2
import numpy as np
from math import ceil

def read_video(video_file):
"""Reads a video file, returns frames(T, H, W, 3) """
VidObj = cv2.VideoCapture(video_file)
VidObj.set(cv2.CAP_PROP_POS_MSEC, 0)
success, frame = VidObj.read()
frames = list()
while success:
frame = cv2.cvtColor(np.array(frame), cv2.COLOR_BGR2RGB)
frame = np.asarray(frame)
frames.append(frame)
success, frame = VidObj.read()
return np.asarray(frames)

def preprocess(frames, config_preprocess):

frames = crop_face_resize(frames,
    True,
    True,
    1.5,
    False,
    30,
    False,
    128,
    128)

# Check data transformation type
data = list()  # Video data
f_c = frames.copy()
data.append(f_c)
data = np.concatenate(data, axis=-1)  # concatenate all channels
frames_clips = np.array([data])

return frames_clips

def crop_face_resize(frames, use_face_detection, use_larger_box, larger_box_coef, use_dynamic_detection,
detection_freq, use_median_box, width, height):
# Face Cropping
if use_dynamic_detection:
num_dynamic_det = ceil(frames.shape[0] / detection_freq)
else:
num_dynamic_det = 1
face_region_all = []
# Perform face detection by num_dynamic_det" times.
for idx in range(num_dynamic_det):
if use_face_detection:
face_region_all.append(face_detection(frames[detection_freq * idx], use_larger_box, larger_box_coef))
else:
face_region_all.append([0, 0, frames.shape[1], frames.shape[2]])
face_region_all = np.asarray(face_region_all, dtype='int')
if use_median_box:
# Generate a median bounding box based on all detected face regions
face_region_median = np.median(face_region_all, axis=0).astype('int')

# Frame Resizing
resized_frames = np.zeros((frames.shape[0], height, width, 3))
for i in range(0, frames.shape[0]):
    frame = frames[i]
    if use_dynamic_detection:  # use the (i // detection_freq)-th facial region.
        reference_index = i // detection_freq
    else:  # use the first region obtrained from the first frame.
        reference_index = 0
    if use_face_detection:
        if use_median_box:
            face_region = face_region_median
        else:
            face_region = face_region_all[reference_index]
        frame = frame[max(face_region[1], 0):min(face_region[1] + face_region[3], frame.shape[0]),
                max(face_region[0], 0):min(face_region[0] + face_region[2], frame.shape[1])]
    resized_frames[i] = cv2.resize(frame, (width, height), interpolation=cv2.INTER_AREA)
return resized_frames

def face_detection(frame, use_larger_box=False, larger_box_coef=1.0):

detector = cv2.CascadeClassifier('./dataset/haarcascade_frontalface_default.xml')
face_zone = detector.detectMultiScale(frame)
if len(face_zone) < 1:
    print("ERROR: No Face Detected")
    face_box_coor = [0, 0, frame.shape[0], frame.shape[1]]
elif len(face_zone) >= 2:
    face_box_coor = np.argmax(face_zone, axis=0)
    face_box_coor = face_zone[face_box_coor[2]]
    print("Warning: More than one faces are detected(Only cropping the biggest one.)")
else:
    face_box_coor = face_zone[0]
if use_larger_box:
    face_box_coor[0] = max(0, face_box_coor[0] - (larger_box_coef - 1.0) / 2 * face_box_coor[2])
    face_box_coor[1] = max(0, face_box_coor[1] - (larger_box_coef - 1.0) / 2 * face_box_coor[3])
    face_box_coor[2] = larger_box_coef * face_box_coor[2]
    face_box_coor[3] = larger_box_coef * face_box_coor[3]
return face_box_coor

def predict_vitals():

# configurations.
config = get_config()
print('Configuration:')
print(config, end='\n\n')
config = config
frame_depth = 10

kam_device = torch.device("cuda")  # Use CUDA device
TSCAN_model = TSCAN(frame_depth=frame_depth, img_size=72).to(kam_device)

# Load the saved state_dict
state_dict = torch.load('final_model_release/UBFC_TSCAN.pth')
# Create a new state_dict with modified keys
new_state_dict = {}
for key, value in state_dict.items():
    if key.startswith('module.'):
        new_key = key[len('module.'):]
    else:
        new_key = key
    new_state_dict[new_key] = value

# Use the new_state_dict in your model
TSCAN_model.load_state_dict(new_state_dict)
TSCAN_model.eval()
vid_path = 'my_vid.mp4'
frames = read_video(vid_path)
frames_clips = preprocess(frames, config.TEST.DATA)
frames_clips = np.squeeze(frames_clips, axis=0)
frames_clips_transposed = np.transpose(frames_clips, (0, 3, 1, 2))
data = np.float32(frames_clips_transposed)
data_tnsr = torch.from_numpy(data)
data_tnsr = data_tnsr.to(kam_device)
num_of_gpu = 1
frame_depth = 10
base_len = num_of_gpu * frame_depth
ND, C, H, W = data_tnsr.shape
data_tnsr = data_tnsr.view(ND, C, H, W)
data_test = data_tnsr[:(ND) // base_len * base_len]
#print('data_test shape:', data_test.shape)

with torch.no_grad():
    pred_ppg_test = TSCAN_model(data_test)
    print('pred_ppg_test:', pred_ppg_test)

if name == "main":
predict_vitals()

Why does the physnet model need DiffNormalized?

Yu Z, Li X, Zhao G. Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks[M]. arXiv, 2019.

I can't find anything about DiffNormalized in this article.

Jan - TODOS

TODOs in Jan / Feb

Deep learning models implementations - Jan - Xin
Traditional signal processing - - Jan - Xiaoyu
Training DL - Jan - Xin
Datasets Support - Jan - Xiaoyu
Inference DL - Jan - Xin
Providing ground truth labels for UBFC, MMSE, PURE - Jan - Xiaoran
Benchmarking - Feb (pre-trained models (PURE + UBFC, Test on MMSE)) - Xin
Visualization - Jan - Xiaoyu / Xiaoran
Synchronization - Feb - Xiaoran
Highlight what conditions the toolbox will work well. E.g., stationary, studio light, heavily compressed etc. - Jan - Xin

@Xiaoyu-SZ @Jiang-Xiaoran

KeyError: 'Non-existent config key: DATA'

Hi, I got some troubles here. I try to train the TSCAN on PURE dataset, then I follow the steps. However, the ERROR mentioned:

_Traceback (most recent call last):
File "main_neural_method.py", line 107, in
config = get_config(args)
File "E:\code\rPPG-Toolbox\config.py", line 266, in get_config
update_config(config, args)
File "E:\code\rPPG-Toolbox\config.py", line 195, in update_config
_update_config_from_file(config, args.config_file)
File "E:\code\rPPG-Toolbox\config.py", line 190, in _update_config_from_file
config.merge_from_file(cfg_file)
File "D:\Anaconda3\envs\python38\lib\site-packages\yacs\config.py", line 213, in merge_from_file
self.merge_from_other_cfg(cfg)
File "D:\Anaconda3\envs\python38\lib\site-packages\yacs\config.py", line 217, in merge_from_other_cfg
_merge_a_into_b(cfg_other, self, self, [])
File "D:\Anaconda3\envs\python38\lib\site-packages\yacs\config.py", line 491, in merge_a_into_b
raise KeyError("Non-existent config key: {}".format(full_key))
KeyError: 'Non-existent config key: DATA'

Could you tell me how to solve the problems? thanks very much.

Is it possible that the predict heartrate is exactly same as groundtruth heartrate?

I am currently experiment with UBFC dataset (subject10 ~ subject14) with PURE_UBFC-rPPG_TSCAN_BASIC.yaml. I have slightly modified the yaml file as follow:

Set METRICS to ['MAE','RMSE']
Set the DO_PREPROCESS to True.
Set my own DATA_PATH and CACHED_PATH.
Set the DEVICE to cpu.

When call the calculate_metrics() function, I try to inspect the variables of gt_hr_fft_all and predict_hr_fft_all, and found the result as follow:

gt_hr: [119.53125 128.3203125 69.43359375 109.86328125 77.34375 ],
pred_hr: [119.53125 128.3203125 67.67578125 109.86328125 70.3125 ]
FFT MAE (FFT Label): 1.7578125 +/- 1.2178482240718669
FFT RMSE (FFT Label): 3.241246098267031 +/- 8.72207385201761

The output gt/pred hr of subject10, subject11 and subject13 are exactly the same, is it possible?

Problems about VIPL data set...

Dear author,

Hi! Thanks for your contribution, and I have made good results on PURE, UBFC, and CONHFACE because of this toolbox. Recently I tried to train on the VIPL data set. Here is what I did:

VIPL data set contains two files: video.avi and wave.csv. So I changed def read_wave(bvp_file) function to read CSV file.
The video format in CONHFACE is also avi, so I didn't change the function of the read video:
The length of the wave doesn't match the length of the video in VIPL, so following your code, I did the sample too:

Apart from these, I did not make any other modifications. I used TSCAN to train, but I couldn't get good results. Here are the predicted PPG signal and ground-truth signal:

In the predicted PPG signal, it can be seen that the model predicts the periodicity, but the frequency is wrong.

The MAE is over 30, sorry I didn't have a screenshot of the last results.

Do you have any clues about this problem?

A question about COHFACEloader

Hello, I see that there is COHFACEloader in the dataloader folder, but there is no description of COHFACE dataset in the readme. Is there a problem with the COHFACEloader file, can I use it to load the COHFACE dataset?

The details of 'Preprocessed ' part

Hi! I am reading the Preprocessed part of the code, but I'm confused that there is no detail about it in the article "Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement"...
Could you please tell me where I can read the details of the Preprocessed part?

About Postprocess

Will this toolbox provide the estimation of HRV, RF and other physiological signals from rPPG? I can only find the HR estimation in this project currently.

problems about preprocessing and test on UBFC dataset

Dear author, thanks for your dedication, I believe such a valuable tool will promote the development of the rPPG field and allow researchers to more effectively build upon existing work.

I have encountered a few issues while using the tool, and I was hoping to seek your guidance in resolving them.

When I follow the steps in readme.md to preprocess the UBFC dataset, I encountered some warnings/errors, but the program can still continue to run. What caused this? Do these warnings have an impact on the experimental results?
Apart from DO_PREPROCESS & DATA_PATH & CACHED_PATH, I followed your default parameters in UBFC_UNSUPERVISED.yaml without making any changes. But the results were not as good as yours (see the figure below).
For example, the MAE and RMSE of POS and CHROM in my result:
POS: 4.00 7.58
CHROM: 3.98 8.72

Am I ignoring some details? Or your parameters are different from mine?
When using pre-trained models PURE_UBFC_TSCAN_BASIC.yaml, the inference/predict process is very slow, lasting up to 2 hours. Is this normal? I ran the program on the server.
I noticed that the framerates of different videos in the UBFC dataset are inconsistent. For example, subject12’s framerate is 29 and subject23’s framerate is 28.9, but in .yaml, the value of the FS parameter is fixed to 30. Does this affect the results?
Each video will calculate an MAE value, and the MAE in the final result is the average of these video MAEs. Am I correct?This may be a naive question...
Is the Frame Resizing operation (cv2.resize) necessary? What is its role? Can I use a raw frame? How are W=72 and H=72 determined?

I appreciate any guidance you can provide. thanks.

File not found error in running model trained on PURE and tested on UBFC with Physnet

Preprocessing dataset...
  0%|                                                                                                                                                                                       | 0/1 [00:00<?, ?it/s]Warning: More than one faces are detected(Only cropping the biggest one.)
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:11<00:00, 11.06s/it]
Traceback (most recent call last):
  File "main.py", line 237, in <module>
    test_data = test_loader(
  File "C:\Users\bsing\Documents\rPPG-Toolbox-main\dataset\data_loader\UBFCrPPGLoader.py", line 42, in __init__
    super().__init__(name, data_path, config_data)
  File "C:\Users\bsing\Documents\rPPG-Toolbox-main\dataset\data_loader\BaseLoader.py", line 67, in __init__
    self.preprocess_dataset(self.raw_data_dirs, config_data.PREPROCESS, config_data.BEGIN, config_data.END)
  File "C:\Users\bsing\Documents\rPPG-Toolbox-main\dataset\data_loader\BaseLoader.py", line 208, in preprocess_dataset
    self.build_file_list(file_list_dict)  # build file list
  File "C:\Users\bsing\Documents\rPPG-Toolbox-main\dataset\data_loader\BaseLoader.py", line 485, in build_file_list
    file_list_df.to_csv(self.file_list_path)  # save file list to .csv
  File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\pandas\core\generic.py", line 3170, in to_csv
    formatter.save()
  File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\pandas\io\formats\csvs.py", line 185, in save
    f, handles = get_handle(
  File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\pandas\io\common.py", line 493, in get_handle
    f = open(path_or_buf, mode, encoding=encoding, errors=errors, newline="")
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\bsing\\Documents\\rPPG-Toolbox-main\\cache\\DataFileLists\\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv'

about bandpass filter in post_process.py

hi, @yahskapar @girishvn
In the function _calculate_fft_hr(), since there is an ideal bandpass filter, do you still need to use a Butterworth bandpass filter in the calculate_metric_per_video() function? All their passband frequencies are the same, isn't this redundant?

Could you please explain it? Thank you.

About the generalization ability of which train on MMPD (not mini-MMPD) and test on UBFC

Config file : MMPD_MMPD_UBFC_TSCAN_BASIC.yaml

TRAIN:
BATCH_SIZE: 4
EPOCHS: 30
LR: 9e-3
MODEL_FILE_NAME: MMPD_MMPD_UBFC_tscan
DATA:
INFO:
LIGHT: [1, 2, 3] # 1 - LED-Low, 2 - LED-high, 3 - Incandescent, 4 - Nature
MOTION: [1] # 1 - Stationary, 2 - Rotation, 3 - Talking, 4 - Walking
EXERCISE: [2] # 1 - True, 2 - False
SKIN_COLOR: [3] # Fitzpatrick Scale Skin Types - 3, 4, 5, 6
GENDER: [1, 2] # 1 - Male, 2 - Female
GLASSER: [1, 2] # 1 - True, 2 - False
HAIR_COVER: [1, 2] # 1 - True, 2 - False
MAKEUP: [1, 2] # 1 - True, 2 - False`

Finally I got the Test result as follow:

FFT MAE (FFT Label):18.10546875
FFT RMSE (FFT Label):28.37179621284168
FFT MAPE (FFT Label):16.860544848875065
FFT Pearson (FFT Label):0.33862065963207777

I have checked the data, label, filename, but all of these seem right. Can you help me find what's going on here?

Error while installing requirements

I am encountering the following error while running the pip install -r requirements.txt inside my rppg-toolbox environment.

Failed to build numpy
      ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
      [end of output]

I have followed all previous steps properly.

TSCAN: size mismatch between model parameter and the pretrained model parameter

Hi,

First of all thank you for this amazing work and sharing it.

I want to use the pretrained TSCAN model to get the rppg data from my own videos, but when I load the pretrained weights from PURE_TSCAN.pth or UBFC_TSCAN.pth, I get the following error:

size mismatch for final_dense_1.weight: copying a param with shape torch.Size([128, 16384]) from checkpoint, the shape in current model is torch.Size([128, 57600]).

My code and the output of the code is given below. Could you please tell me why there exists a mismatch between the parameters and how to solve this issue?

import torch
from neural_methods.model.TS_CAN import TSCAN
from config import get_config
def predict_vitals():

# configurations.
config = get_config()
print('Configuration:')
print(config, end='\n\n')

frame_depth = config.MODEL.TSCAN.FRAME_DEPTH
config = config

#if torch.cuda.is_available():
kam_device = torch.device("cuda")  # Use CUDA device
TSCAN_model = TSCAN(frame_depth=frame_depth, img_size=config.TEST.DATA.PREPROCESS.RESIZE.H).to(kam_device)

# Get the state dictionary of the model
TSCAN_model_state_dict = TSCAN_model.state_dict()

print('************ keys and size of kam_model_state_dict ***************')
# Print keys and sizes of parameters
for key, value in TSCAN_model_state_dict.items():
    print(f'Key: {key}\tSize: {value.size()}')
print('***************************')

# Load the saved state_dict
state_dict = torch.load('final_model_release/UBFC_TSCAN.pth')

print('************ keys and size of old state dict ***************')
# Print keys and sizes of parameters
for key, value in state_dict.items():
    print(f'Key: {key}\tSize: {value.size()}')
print('***************************')

# Create a new state_dict with 'module.' removed in keys
new_state_dict = {}
for key, value in state_dict.items():
    if key.startswith('module.'):
        new_key = key[len('module.'):]
    else:
        new_key = key
    new_state_dict[new_key] = value

# Print keys and sizes of parameters
print('************ keys and size of new state dict ***************')
for key, value in new_state_dict.items():
    print(f'Key: {key}\tSize: {value.size()}')
print('***************************')

# Use the new_state_dict in the model
TSCAN_model.load_state_dict(new_state_dict)

if name == "main":
predict_vitals()

#************************** Outputs ******************************

Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
BATCH_SIZE: 4
EVALUATION_METHOD: FFT
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False
WINDOW_SIZE: 10
MODEL_PATH:
LOG:
PATH: runs/exp
MODEL:
DROP_RATE: 0.0
EFFICIENTPHYS:
FRAME_DEPTH: 10
MODEL_DIR: PreTrainedModels
NAME:
PHYSNET:
FRAME_NUM: 64
RESUME:
TSCAN:
FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME:
FILE_LIST_PATH: PreprocessedData/DataFileLists
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
METRICS: []
USE_LAST_EPOCH: True
TOOLBOX_MODE:
TRAIN:
BATCH_SIZE: 4
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME:
FILE_LIST_PATH: PreprocessedData/DataFileLists
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
EPOCHS: 50
LR: 0.0001
MODEL_FILE_NAME:
OPTIMIZER:
BETAS: (0.9, 0.999)
EPS: 0.0001
MOMENTUM: 0.9

/home/kamranali/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/cuda/init.py:146: UserWarning:
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 compute_37.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
************ keys and size of kam_model_state_dict ***************
Key: motion_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: motion_conv1.bias Size: torch.Size([32])
Key: motion_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: motion_conv2.bias Size: torch.Size([32])
Key: motion_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: motion_conv3.bias Size: torch.Size([64])
Key: motion_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: motion_conv4.bias Size: torch.Size([64])
Key: apperance_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: apperance_conv1.bias Size: torch.Size([32])
Key: apperance_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: apperance_conv2.bias Size: torch.Size([32])
Key: apperance_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: apperance_conv3.bias Size: torch.Size([64])
Key: apperance_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: apperance_conv4.bias Size: torch.Size([64])
Key: apperance_att_conv1.weight Size: torch.Size([1, 32, 1, 1])
Key: apperance_att_conv1.bias Size: torch.Size([1])
Key: apperance_att_conv2.weight Size: torch.Size([1, 64, 1, 1])
Key: apperance_att_conv2.bias Size: torch.Size([1])
Key: final_dense_1.weight Size: torch.Size([128, 57600])
Key: final_dense_1.bias Size: torch.Size([128])
Key: final_dense_2.weight Size: torch.Size([1, 128])
Key: final_dense_2.bias Size: torch.Size([1])

===Testing===
************ keys and size of old state dict ***************
Key: module.motion_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: module.motion_conv1.bias Size: torch.Size([32])
Key: module.motion_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: module.motion_conv2.bias Size: torch.Size([32])
Key: module.motion_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: module.motion_conv3.bias Size: torch.Size([64])
Key: module.motion_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: module.motion_conv4.bias Size: torch.Size([64])
Key: module.apperance_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: module.apperance_conv1.bias Size: torch.Size([32])
Key: module.apperance_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: module.apperance_conv2.bias Size: torch.Size([32])
Key: module.apperance_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: module.apperance_conv3.bias Size: torch.Size([64])
Key: module.apperance_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: module.apperance_conv4.bias Size: torch.Size([64])
Key: module.apperance_att_conv1.weight Size: torch.Size([1, 32, 1, 1])
Key: module.apperance_att_conv1.bias Size: torch.Size([1])
Key: module.apperance_att_conv2.weight Size: torch.Size([1, 64, 1, 1])
Key: module.apperance_att_conv2.bias Size: torch.Size([1])
Key: module.final_dense_1.weight Size: torch.Size([128, 16384])
Key: module.final_dense_1.bias Size: torch.Size([128])
Key: module.final_dense_2.weight Size: torch.Size([1, 128])
Key: module.final_dense_2.bias Size: torch.Size([1])

************ keys and size of new state dict ***************
Key: motion_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: motion_conv1.bias Size: torch.Size([32])
Key: motion_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: motion_conv2.bias Size: torch.Size([32])
Key: motion_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: motion_conv3.bias Size: torch.Size([64])
Key: motion_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: motion_conv4.bias Size: torch.Size([64])
Key: apperance_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: apperance_conv1.bias Size: torch.Size([32])
Key: apperance_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: apperance_conv2.bias Size: torch.Size([32])
Key: apperance_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: apperance_conv3.bias Size: torch.Size([64])
Key: apperance_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: apperance_conv4.bias Size: torch.Size([64])
Key: apperance_att_conv1.weight Size: torch.Size([1, 32, 1, 1])
Key: apperance_att_conv1.bias Size: torch.Size([1])
Key: apperance_att_conv2.weight Size: torch.Size([1, 64, 1, 1])
Key: apperance_att_conv2.bias Size: torch.Size([1])
Key: final_dense_1.weight Size: torch.Size([128, 16384])
Key: final_dense_1.bias Size: torch.Size([128])
Key: final_dense_2.weight Size: torch.Size([1, 128])
Key: final_dense_2.bias Size: torch.Size([1])

Traceback (most recent call last):
File "/home/kamranali/Kamran_data/Pycharm_projects/kamran_rPPG-Toolbox-main/my_test.py", line 271, in
predict_vitals()
File "/home/kamranali/Kamran_data/Pycharm_projects/kamran_rPPG-Toolbox-main/my_test.py", line 222, in predict_vitals
kam_model.load_state_dict(new_state_dict)
File "/home/kamranali/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for TSCAN:
size mismatch for final_dense_1.weight: copying a param with shape torch.Size([128, 16384]) from checkpoint, the shape in current model is torch.Size([128, 57600]).

Process finished with exit code 1

.pth file is not generating Training on SCAMPS and testing on UBFC With Physnet

(rppg-toolbox) PS C:\Users\bsing\rPPG-Toolbox-main> python main.py --config_file ./configs/train_configs/SCAMPS_SCAMPS_UBFC-rPPG_PHYSNET_BASIC.yaml
=> Merging a config file from ./configs/train_configs/SCAMPS_SCAMPS_UBFC-rPPG_PHYSNET_BASIC.yaml
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
  BATCH_SIZE: 1
  EVALUATION_METHOD: FFT
  EVALUATION_WINDOW:
    USE_SMALLER_WINDOW: False
    WINDOW_SIZE: 10
  MODEL_PATH:
LOG:
  PATH: runs/exp\
MODEL:
  BIGSMALL:
    FRAME_DEPTH: 3
  DROP_RATE: 0.2
  EFFICIENTPHYS:
    FRAME_DEPTH: 10
  MODEL_DIR: PreTrainedModels\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
  NAME: Physnet
  PHYSNET:
    FRAME_NUM: 128
  RESUME:
  TSCAN:
    FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
    DATASET: UBFC-rPPG
    DATA_FORMAT: NCDHW
    DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\UBFC
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
    FILE_LIST_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 128
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 32
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False
  METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR']
  USE_LAST_EPOCH: True
TOOLBOX_MODE: train_and_test
TRAIN:
  BATCH_SIZE: 1
  DATA:
    BEGIN: 0.0
    CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
    DATASET: SCAMPS
    DATA_FORMAT: NCDHW
    DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\SCAMPS\Train
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
    FILE_LIST_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 128
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 32
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False
  EPOCHS: 30
  LR: 0.009
  MODEL_FILE_NAME: SCAMPS_SCAMPS_UBFC_physnet
  OPTIMIZER:
    BETAS: (0.9, 0.999)
    EPS: 0.0001
    MOMENTUM: 0.9
UNSUPERVISED:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData\_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    DATASET:
    DATA_FORMAT: NDCHW
    DATA_PATH:
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    FILE_LIST_PATH: PreprocessedData\DataFileLists\_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 0
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['']
      DO_CHUNK: True
      LABEL_TYPE:
      RESIZE:
        H: 128
        W: 128
      USE_PSUEDO_PPG_LABEL: False
  METHOD: []
  METRICS: []
VALID:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache
    DATASET: SCAMPS
    DATA_FORMAT: NCDHW
    DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\SCAMPS\Val
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME:
    FILE_LIST_PATH: PreprocessedData\DataFileLists
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 128
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 32
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False

Cached Data Path C:\Users\bsing\rPPG-Toolbox-main\cache\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse

File List Path C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
 train Preprocessed Dataset Length: 32

Testing uses last epoch, validation dataset is not required.

Cached Data Path C:\Users\bsing\rPPG-Toolbox-main\cache\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse

File List Path C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
 test Preprocessed Dataset Length: 51


====Training Epoch: 0====
Train epoch 0: 100%|███████████████| 32/32 [00:44<00:00,  1.38s/it, loss=0.0573]
Traceback (most recent call last):
  File "main.py", line 287, in <module>
    train_and_test(config, data_loader_dict)
  File "main.py", line 78, in train_and_test
    model_trainer.train(data_loader_dict)
  File "C:\Users\bsing\rPPG-Toolbox-main\neural_methods\trainer\PhysnetTrainer.py", line 81, in train
    self.save_model(epoch)
  File "C:\Users\bsing\rPPG-Toolbox-main\neural_methods\trainer\PhysnetTrainer.py", line 180, in save_model
    torch.save(self.model.state_dict(), model_path)
  File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 376, in save
    with _open_file_like(f, 'wb') as opened_file:
  File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 230, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 211, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'PreTrainedModels\\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse\\SCAMPS_SCAMPS_UBFC_physnet_Epoch0.pth'
(rppg-toolbox) PS C:\Users\bsing\rPPG-Toolbox-main>

After setup and requirements installation torch.cuda.is_available() is False

After setup and requirements installation torch.cuda.is_available() is False
And can I run this rppg toolbox with just cpu. If yes than how

About CHROM method

Thanks very much about your job !

I have a question about this line:

rPPG-Toolbox/unsupervised_methods/methods/CHROME_DEHAAN.py

Line 36 in be4a0ac

RGBNorm[temp-WinS] = np.true_divide(RGB[temp], RGBBase)-1

About the nomalization method in CHROM, it only used $\frac{C_i}{\mu (C_i)}$ in my opinion, without using - 1.

- 1 was used in XoverY, not in XsminYs. Is there any problem in my point ?

Two functions not found in 'signal_predictor.py'

calculate_metric_peak_per_video in line 49
calculate_metric_per_video in line 54

ValueError: cannot reshape array of size 61936 into shape (180,72,72,6)

Dear rPPG-Toolbox team,
I am attempting to test on the UBFC-rPPG but I am getting the error ValueError: cannot reshape array of size 61936 into shape (180,72,72,6). I am inclined to think that it might be my dataset but nevertheless I would like your opinion.
The Yaml configuration file is as follows

BASE: ['']
TOOLBOX_MODE: "train_and_test"      # "train_and_test"  or "only_test"
TRAIN:
  BATCH_SIZE: 4
  EPOCHS: 30
  LR: 9e-3
  MODEL_FILE_NAME: UBFC_UBFC_deepphys
  DATA:
    FS: 30
    DATASET: UBFC-rPPG
    DO_PREPROCESS: False               # if first time, should be true
    DATA_FORMAT: NDCHW
    DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2"                     # Raw dataset path, need to be updated
    CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/PreprocessedData"    # Processed dataset save path, need to be updated
    EXP_DATA_NAME: ""
    BEGIN: 0.0
    END: 0.8
    PREPROCESS:
      DATA_TYPE: ['DiffNormalized','Standardized']
      DATA_AUG: ['None']    # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
      LABEL_TYPE: DiffNormalized
      DO_CHUNK: True
      CHUNK_LENGTH: 180
      CROP_FACE:
        DO_CROP_FACE: True
        USE_LARGE_FACE_BOX: True
        LARGE_BOX_COEF: 1.5
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY : 30
          USE_MEDIAN_FACE_BOX: False    # This should be used ONLY if dynamic detection is used
      RESIZE:
        H: 72
        W: 72
VALID:
  DATA:
    FS: 30
    DATASET: UBFC-rPPG
    DO_PREPROCESS: True                 # if first time, should be true
    DATA_FORMAT: NDCHW
    DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2"                     # Raw dataset path, need to be updated
    CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/PreprocessedData"    # Processed dataset save path, need to be updated
    EXP_DATA_NAME: ""
    BEGIN: 0.8
    END: 1.0
    PREPROCESS:
      DATA_TYPE: [ 'DiffNormalized','Standardized' ]
      DATA_AUG: ['None']    # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
      LABEL_TYPE: DiffNormalized
      DO_CHUNK: False
      CHUNK_LENGTH: 180
      CROP_FACE:
        DO_CROP_FACE: True
        USE_LARGE_FACE_BOX: True
        LARGE_BOX_COEF: 1.5
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY : 30
          USE_MEDIAN_FACE_BOX: False    # This should be used ONLY if dynamic detection is used
      RESIZE:
        H: 72
        W: 72
TEST:
  METRICS: ['MAE','RMSE','MAPE','Pearson', 'SNR']
  USE_LAST_EPOCH: True                      # to use provided validation dataset to find the best epoch, should be false
  DATA:
    FS: 30
    DATASET: UBFC-rPPG
    DO_PREPROCESS: True               # if first time, should be true
    DATA_FORMAT: NDCHW
    DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2"                     # Raw dataset path, need to be updated
    CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/PreprocessedData"    # Processed dataset save path, need to be updated
    EXP_DATA_NAME: ""
    BEGIN: 0.21
    END: 0.4
    PREPROCESS:
      DATA_TYPE: ['DiffNormalized','Standardized']
      LABEL_TYPE: DiffNormalized
      DO_CHUNK: True
      CHUNK_LENGTH: 180
      CROP_FACE:
        DO_CROP_FACE: True
        USE_LARGE_FACE_BOX: True
        LARGE_BOX_COEF: 1.5
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY : 30
          USE_MEDIAN_FACE_BOX: False    # This should be used ONLY if dynamic detection is used
      RESIZE:
        H: 72
        W: 72
DEVICE: cuda:0
NUM_OF_GPU_TRAIN: 1
LOG:
  PATH: runs/exp
MODEL:
  DROP_RATE: 0.2
  NAME: DeepPhys
INFERENCE:
  BATCH_SIZE: 4
  EVALUATION_METHOD: "FFT"        # "FFT" or "peak detection"
  EVALUATION_WINDOW:
    USE_SMALLER_WINDOW: False        # Change this if you'd like an evaluation window smaller than the test video length
    WINDOW_SIZE: 10        # In seconds
  MODEL_PATH: ""

The content of the dataset folder is

$ tree DATASET_2
DATASET_2
├── subject1
│   ├── ground_truth.txt
│   └── vid.avi
├── subject10
│   ├── ground_truth.txt
│   └── vid.avi
├── subject11
│   ├── ground_truth.txt
│   └── vid.avi
├── subject12
│   ├── ground_truth.txt
│   └── vid.avi
├── subject13
│   ├── ground_truth.txt
│   └── vid.avi
├── subject14
│   ├── ground_truth.txt
│   └── vid.avi
├── subject15
│   ├── ground_truth.txt
│   └── vid.avi
├── subject16
│   ├── ground_truth.txt
│   └── vid.avi
├── subject17
│   ├── ground_truth.txt
│   └── vid.avi
├── subject18
│   ├── ground_truth.txt
│   └── vid.avi
├── subject20
│   ├── ground_truth.txt
│   └── vid.avi
├── subject22
│   ├── ground_truth.txt
│   └── vid.avi
├── subject23
│   ├── ground_truth.txt
│   └── vid.avi
├── subject24
│   ├── ground_truth.txt
│   └── vid.avi
├── subject25
│   ├── ground_truth.txt
│   └── vid.avi
├── subject26
│   ├── ground_truth.txt
│   └── vid.avi
├── subject27
│   ├── ground_truth.txt
│   └── vid.avi
├── subject3
│   ├── ground_truth.txt
│   └── vid.avi
├── subject30
│   ├── ground_truth.txt
│   └── vid.avi
├── subject31
│   ├── ground_truth.txt
│   └── vid.avi
├── subject32
│   ├── ground_truth.txt
│   └── vid.avi
├── subject33
│   ├── ground_truth.txt
│   └── vid.avi
├── subject34
│   ├── ground_truth.txt
│   └── vid.avi
├── subject35
│   ├── ground_truth.txt
│   └── vid.avi
├── subject36
│   ├── ground_truth.txt
│   └── vid.avi
├── subject37
│   ├── ground_truth.txt
│   └── vid.avi
├── subject38
│   ├── ground_truth.txt
│   └── vid.avi
├── subject39
│   ├── ground_truth.txt
│   └── vid.avi
├── subject4
│   ├── ground_truth.txt
│   └── vid.avi
├── subject40
│   ├── ground_truth.txt
│   └── vid.avi
├── subject41
│   ├── ground_truth.txt
│   └── vid.avi
├── subject42
│   ├── ground_truth.txt
│   └── vid.avi
├── subject43
│   ├── ground_truth.txt
│   └── vid.avi
├── subject44
│   ├── ground_truth.txt
│   └── vid.avi
├── subject45
│   ├── ground_truth.txt
│   └── vid.avi
├── subject46
│   ├── ground_truth.txt
│   └── vid.avi
├── subject47
│   ├── ground_truth.txt
│   └── vid.avi
├── subject48
│   ├── ground_truth.txt
│   └── vid.avi
├── subject49
│   ├── ground_truth.txt
│   └── vid.avi
├── subject5
│   ├── ground_truth.txt
│   └── vid.avi
├── subject8
│   ├── ground_truth.txt
│   └── vid.avi
└── subject9
    ├── ground_truth.txt
    └── vid.avi

Error message

$ python main.py --config_file ./configs/train_configs/UBFC-rPPG_PHYSNET_laura.yaml
=> Merging a config file from ./configs/train_configs/UBFC-rPPG_PHYSNET_laura.yaml
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
  BATCH_SIZE: 4
  EVALUATION_METHOD: FFT
  EVALUATION_WINDOW:
    USE_SMALLER_WINDOW: False
    WINDOW_SIZE: 10
  MODEL_PATH: 
LOG:
  PATH: runs/exp/
MODEL:
  BIGSMALL:
    FRAME_DEPTH: 3
  DROP_RATE: 0.2
  EFFICIENTPHYS:
    FRAME_DEPTH: 10
  MODEL_DIR: PreTrainedModels/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
  NAME: DeepPhys
  PHYSNET:
    FRAME_NUM: 64
  RESUME: 
  TSCAN:
    FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
  DATA:
    BEGIN: 0.21
    CACHED_PATH: /media/laura/datasets/UBFC_rPPG/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    DATASET: UBFC-rPPG
    DATA_FORMAT: NDCHW
    DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
    DO_PREPROCESS: True
    END: 0.4
    EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    FILE_LIST_PATH: /media/laura/datasets/UBFC_rPPG/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.21_0.4.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized', 'Standardized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False
  METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR']
  USE_LAST_EPOCH: True
TOOLBOX_MODE: train_and_test
TRAIN:
  BATCH_SIZE: 4
  DATA:
    BEGIN: 0.0
    CACHED_PATH: /media/laura/datasets/UBFC_rPPG/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    DATASET: UBFC-rPPG
    DATA_FORMAT: NDCHW
    DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
    DO_PREPROCESS: False
    END: 0.8
    EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    FILE_LIST_PATH: /media/laura/datasets/UBFC_rPPG/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_0.8.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized', 'Standardized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False
  EPOCHS: 30
  LR: 0.009
  MODEL_FILE_NAME: UBFC_UBFC_deepphys
  OPTIMIZER:
    BETAS: (0.9, 0.999)
    EPS: 0.0001
    MOMENTUM: 0.9
UNSUPERVISED:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    DATASET: 
    DATA_FORMAT: NDCHW
    DATA_PATH: 
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 0
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['']
      DO_CHUNK: True
      LABEL_TYPE: 
      RESIZE:
        H: 128
        W: 128
      USE_PSUEDO_PPG_LABEL: False
  METHOD: []
  METRICS: []
VALID:
  DATA:
    BEGIN: 0.8
    CACHED_PATH: /media/laura/datasets/UBFC_rPPG/PreprocessedData
    DATASET: UBFC-rPPG
    DATA_FORMAT: NDCHW
    DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
    DO_PREPROCESS: True
    END: 1.0
    EXP_DATA_NAME: 
    FILE_LIST_PATH: PreprocessedData/DataFileLists
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized', 'Standardized']
      DO_CHUNK: False
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False

Cached Data Path /media/laura/datasets/UBFC_rPPG/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse

File List Path /media/laura/datasets/UBFC_rPPG/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_0.8.csv
 train Preprocessed Dataset Length: 47

Testing uses last epoch, validation dataset is not required.

Preprocessing dataset...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [07:37<00:00, 57.23s/it]
Total Number of raw files preprocessed: 8

Cached Data Path /media/laura/datasets/UBFC_rPPG/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse

File List Path /media/laura/datasets/UBFC_rPPG/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.21_0.4.csv
 test Preprocessed Dataset Length: 33


====Training Epoch: 0====
Train epoch 0:  33%|█▋   | 4/12 [01:00<02:00, 15.04s/it, loss=0.802, lr=0.00039]
Traceback (most recent call last):
  File "/home/laura/Development/rPPG-Toolbox/main.py", line 288, in <module>
    train_and_test(config, data_loader_dict)
  File "/home/laura/Development/rPPG-Toolbox/main.py", line 78, in train_and_test
    model_trainer.train(data_loader_dict)
  File "/home/laura/Development/rPPG-Toolbox/neural_methods/trainer/DeepPhysTrainer.py", line 62, in train
    for idx, batch in enumerate(tbar):
  File "/home/laura/anaconda3/envs/rppg-toolbox/lib/python3.10/site-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 633, in __next__
    data = self._next_data()
  File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1325, in _next_data
    return self._process_data(data)
  File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
    data.reraise()
  File "/home/laura/.local/lib/python3.10/site-packages/torch/_utils.py", line 644, in reraise
    raise exception
ValueError: Caught ValueError in DataLoader worker process 4.
Original Traceback (most recent call last):
  File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/laura/Development/rPPG-Toolbox/dataset/data_loader/BaseLoader.py", line 90, in __getitem__
    data = np.load(self.inputs[index])
  File "/home/laura/anaconda3/envs/rppg-toolbox/lib/python3.10/site-packages/numpy/lib/npyio.py", line 432, in load
    return format.read_array(fid, allow_pickle=allow_pickle,
  File "/home/laura/anaconda3/envs/rppg-toolbox/lib/python3.10/site-packages/numpy/lib/format.py", line 831, in read_array
    array.shape = shape
ValueError: cannot reshape array of size 61936 into shape (180,72,72,6)

test on ubfc-phys gives bad results.

I tested PURE, UBFC2, COFACE, and UBFC-Phys with mtts-can model.
Only a model trained with ubfc-phys gives bad results. (overfitted on the training dataset and fails on valid set)
I used s50~s56 as a valid set and trained with other videos.
Any idea? or does Anyone succeed in training with the UBFC-Phys dataset?

ValueError: ('test', 'File list empty. Check preprocessed data folder exists and is not empty.')

Dear Team,
I am trying to use the toolbox, but I keep getting the error below. I am training and testing on the UBFC-rPPG dataset

$ python main.py --config_file ./configs/train_configs/UBFC-rPPG_PHYSNET_laura.yaml 
=> Merging a config file from ./configs/train_configs/UBFC-rPPG_PHYSNET_laura.yaml
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
  BATCH_SIZE: 4
  EVALUATION_METHOD: FFT
  EVALUATION_WINDOW:
    USE_SMALLER_WINDOW: False
    WINDOW_SIZE: 10
  MODEL_PATH: 
LOG:
  PATH: runs/exp/
MODEL:
  BIGSMALL:
    FRAME_DEPTH: 3
  DROP_RATE: 0.2
  EFFICIENTPHYS:
    FRAME_DEPTH: 10
  MODEL_DIR: PreTrainedModels/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
  NAME: DeepPhys
  PHYSNET:
    FRAME_NUM: 64
  RESUME: 
  TSCAN:
    FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
  DATA:
    BEGIN: 0.8
    CACHED_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    DATASET: UBFC-rPPG
    DATA_FORMAT: NDCHW
    DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    FILE_LIST_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.8_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized', 'Standardized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False
  METRICS: []
  USE_LAST_EPOCH: True
TOOLBOX_MODE: train_and_test
TRAIN:
  BATCH_SIZE: 4
  DATA:
    BEGIN: 0.0
    CACHED_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    DATASET: UBFC-rPPG
    DATA_FORMAT: NDCHW
    DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
    DO_PREPROCESS: False
    END: 0.8
    EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
    FILE_LIST_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_0.8.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized', 'Standardized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False
  EPOCHS: 30
  LR: 0.009
  MODEL_FILE_NAME: UBFC_UBFC_deepphys
  OPTIMIZER:
    BETAS: (0.9, 0.999)
    EPS: 0.0001
    MOMENTUM: 0.9
UNSUPERVISED:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    DATASET: 
    DATA_FORMAT: NDCHW
    DATA_PATH: 
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 0
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['']
      DO_CHUNK: True
      LABEL_TYPE: 
      RESIZE:
        H: 128
        W: 128
      USE_PSUEDO_PPG_LABEL: False
  METHOD: []
  METRICS: []
VALID:
  DATA:
    BEGIN: 0.8
    CACHED_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData
    DATASET: UBFC-rPPG
    DATA_FORMAT: NDCHW
    DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: 
    FILE_LIST_PATH: PreprocessedData/DataFileLists
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME: 
      FOLD_PATH: 
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized', 'Standardized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False

Cached Data Path /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse

File List Path /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_0.8.csv
 train Preprocessed Dataset Length: 46

Testing uses last epoch, validation dataset is not required.

File list does not exist... generating now...
Traceback (most recent call last):
  File "main.py", line 238, in <module>
    test_data = test_loader(
  File "/home/laura/Development/rPPG-Toolbox/dataset/data_loader/UBFCrPPGLoader.py", line 42, in __init__
    super().__init__(name, data_path, config_data)
  File "/home/laura/Development/rPPG-Toolbox/dataset/data_loader/BaseLoader.py", line 76, in __init__
    self.build_file_list_retroactive(self.raw_data_dirs, config_data.BEGIN, config_data.END)
  File "/home/laura/Development/rPPG-Toolbox/dataset/data_loader/BaseLoader.py", line 516, in build_file_list_retroactive
    raise ValueError(self.dataset_name,
ValueError: ('test', 'File list empty. Check preprocessed data folder exists and is not empty.')

YAML Configuration file

BASE: ['']
TOOLBOX_MODE: "train_and_test"      # "train_and_test"  or "only_test"
TRAIN:
  BATCH_SIZE: 4
  EPOCHS: 30
  LR: 9e-3
  MODEL_FILE_NAME: UBFC_UBFC_deepphys
  DATA:
    FS: 30
    DATASET: UBFC-rPPG
    DO_PREPROCESS: False               # if first time, should be true
    DATA_FORMAT: NDCHW
    DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2"                     # Raw dataset path, need to be updated
    CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData"    # Processed dataset save path, need to be updated
    EXP_DATA_NAME: ""
    BEGIN: 0.0
    END: 0.8
    PREPROCESS:
      DATA_TYPE: ['DiffNormalized','Standardized']
      DATA_AUG: ['None']    # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
      LABEL_TYPE: DiffNormalized
      DO_CHUNK: True
      CHUNK_LENGTH: 180
      CROP_FACE:
        DO_CROP_FACE: True
        USE_LARGE_FACE_BOX: True
        LARGE_BOX_COEF: 1.5
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY : 30
          USE_MEDIAN_FACE_BOX: False    # This should be used ONLY if dynamic detection is used
      RESIZE:
        H: 72
        W: 72
VALID:
  DATA:
    FS: 30
    DATASET: UBFC-rPPG
    DO_PREPROCESS: False                 # if first time, should be true
    DATA_FORMAT: NDCHW
    DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2"                     # Raw dataset path, need to be updated
    CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData"    # Processed dataset save path, need to be updated
    EXP_DATA_NAME: ""
    BEGIN: 0.8
    END: 1.0
    PREPROCESS:
      DATA_TYPE: [ 'DiffNormalized','Standardized' ]
      DATA_AUG: ['None']    # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
      LABEL_TYPE: DiffNormalized
      DO_CHUNK: True
      CHUNK_LENGTH: 180
      CROP_FACE:
        DO_CROP_FACE: True
        USE_LARGE_FACE_BOX: True
        LARGE_BOX_COEF: 1.5
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY : 30
          USE_MEDIAN_FACE_BOX: False    # This should be used ONLY if dynamic detection is used
      RESIZE:
        H: 72
        W: 72
TEST:
  DATA:
    FS: 30
    DATASET: UBFC-rPPG
    DO_PREPROCESS: False                  # if first time, should be true
    DATA_FORMAT: NDCHW
    DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2"                     # Raw dataset path, need to be updated
    CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData"    # Processed dataset save path, need to be updated
    EXP_DATA_NAME: ""
    BEGIN: 0.8
    END: 1.0
    PREPROCESS:
      DATA_TYPE: [ 'DiffNormalized','Standardized' ]
      DATA_AUG: ['None']    # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
      LABEL_TYPE: DiffNormalized
      DO_CHUNK: True
      CHUNK_LENGTH: 180
      CROP_FACE:
        DO_CROP_FACE: True
        USE_LARGE_FACE_BOX: True
        LARGE_BOX_COEF: 1.5
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY : 30
          USE_MEDIAN_FACE_BOX: False    # This should be used ONLY if dynamic detection is used
      RESIZE:
        H: 72
        W: 72
DEVICE: cuda:0
NUM_OF_GPU_TRAIN: 1
LOG:
  PATH: runs/exp
MODEL:
  DROP_RATE: 0.2
  NAME: DeepPhys
INFERENCE:
  BATCH_SIZE: 4
  EVALUATION_METHOD: "FFT"        # "FFT" or "peak detection"
  EVALUATION_WINDOW:
    USE_SMALLER_WINDOW: False        # Change this if you'd like an evaluation window smaller than the test video length
    WINDOW_SIZE: 10        # In seconds
  MODEL_PATH:   ""

Are there any pre-trained models that can be used for inference?

I don't see any links to pre-trained models, are these available for any of the methods?

How to do Training on SCAMPS and testing on UBFC With PhysNet

File Not Found Error .pth file is not generating Training on SCAMPS and testing on UBFC With Physnet

(rppg-toolbox) PS C:\Users\bsing\rPPG-Toolbox-main> python main.py --config_file ./configs/train_configs/SCAMPS_SCAMPS_UBFC-rPPG_PHYSNET_BASIC.yaml
=> Merging a config file from ./configs/train_configs/SCAMPS_SCAMPS_UBFC-rPPG_PHYSNET_BASIC.yaml
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
  BATCH_SIZE: 1
  EVALUATION_METHOD: FFT
  EVALUATION_WINDOW:
    USE_SMALLER_WINDOW: False
    WINDOW_SIZE: 10
  MODEL_PATH:
LOG:
  PATH: runs/exp\
MODEL:
  BIGSMALL:
    FRAME_DEPTH: 3
  DROP_RATE: 0.2
  EFFICIENTPHYS:
    FRAME_DEPTH: 10
  MODEL_DIR: PreTrainedModels\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
  NAME: Physnet
  PHYSNET:
    FRAME_NUM: 128
  RESUME:
  TSCAN:
    FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
    DATASET: UBFC-rPPG
    DATA_FORMAT: NCDHW
    DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\UBFC
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
    FILE_LIST_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 128
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 32
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False
  METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR']
  USE_LAST_EPOCH: True
TOOLBOX_MODE: train_and_test
TRAIN:
  BATCH_SIZE: 1
  DATA:
    BEGIN: 0.0
    CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
    DATASET: SCAMPS
    DATA_FORMAT: NCDHW
    DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\SCAMPS\Train
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
    FILE_LIST_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 128
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 32
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False
  EPOCHS: 30
  LR: 0.009
  MODEL_FILE_NAME: SCAMPS_SCAMPS_UBFC_physnet
  OPTIMIZER:
    BETAS: (0.9, 0.999)
    EPS: 0.0001
    MOMENTUM: 0.9
UNSUPERVISED:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: PreprocessedData\_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    DATASET:
    DATA_FORMAT: NDCHW
    DATA_PATH:
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
    FILE_LIST_PATH: PreprocessedData\DataFileLists\_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 0
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      CHUNK_LENGTH: 180
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 30
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['']
      DO_CHUNK: True
      LABEL_TYPE:
      RESIZE:
        H: 128
        W: 128
      USE_PSUEDO_PPG_LABEL: False
  METHOD: []
  METRICS: []
VALID:
  DATA:
    BEGIN: 0.0
    CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache
    DATASET: SCAMPS
    DATA_FORMAT: NCDHW
    DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\SCAMPS\Val
    DO_PREPROCESS: False
    END: 1.0
    EXP_DATA_NAME:
    FILE_LIST_PATH: PreprocessedData\DataFileLists
    FILTERING:
      EXCLUSION_LIST: ['']
      SELECT_TASKS: False
      TASK_LIST: ['']
      USE_EXCLUSION_LIST: False
    FOLD:
      FOLD_NAME:
      FOLD_PATH:
    FS: 30
    INFO:
      EXERCISE: [True]
      GENDER: ['']
      GLASSER: [True]
      HAIR_COVER: [True]
      LIGHT: ['']
      MAKEUP: [True]
      MOTION: ['']
      SKIN_COLOR: [1]
    PREPROCESS:
      BIGSMALL:
        BIG_DATA_TYPE: ['']
        RESIZE:
          BIG_H: 144
          BIG_W: 144
          SMALL_H: 9
          SMALL_W: 9
        SMALL_DATA_TYPE: ['']
      CHUNK_LENGTH: 128
      CROP_FACE:
        DETECTION:
          DO_DYNAMIC_DETECTION: False
          DYNAMIC_DETECTION_FREQUENCY: 32
          USE_MEDIAN_FACE_BOX: False
        DO_CROP_FACE: True
        LARGE_BOX_COEF: 1.5
        USE_LARGE_FACE_BOX: True
      DATA_AUG: ['None']
      DATA_TYPE: ['DiffNormalized']
      DO_CHUNK: True
      LABEL_TYPE: DiffNormalized
      RESIZE:
        H: 72
        W: 72
      USE_PSUEDO_PPG_LABEL: False

Cached Data Path C:\Users\bsing\rPPG-Toolbox-main\cache\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse

File List Path C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
 train Preprocessed Dataset Length: 32

Testing uses last epoch, validation dataset is not required.

Cached Data Path C:\Users\bsing\rPPG-Toolbox-main\cache\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse

File List Path C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
 test Preprocessed Dataset Length: 51


====Training Epoch: 0====
Train epoch 0: 100%|███████████████| 32/32 [00:44<00:00,  1.38s/it, loss=0.0573]
Traceback (most recent call last):
  File "main.py", line 287, in <module>
    train_and_test(config, data_loader_dict)
  File "main.py", line 78, in train_and_test
    model_trainer.train(data_loader_dict)
  File "C:\Users\bsing\rPPG-Toolbox-main\neural_methods\trainer\PhysnetTrainer.py", line 81, in train
    self.save_model(epoch)
  File "C:\Users\bsing\rPPG-Toolbox-main\neural_methods\trainer\PhysnetTrainer.py", line 180, in save_model
    torch.save(self.model.state_dict(), model_path)
  File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 376, in save
    with _open_file_like(f, 'wb') as opened_file:
  File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 230, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 211, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'PreTrainedModels\\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse\\SCAMPS_SCAMPS_UBFC_physnet_Epoch0.pth'
(rppg-toolbox) PS C:\Users\bsing\rPPG-Toolbox-main>

Real time rPPG extraction through camera feed.

Thanks for this repository. Are you planning to add rPPG extraction from real-time camera stream anytime soon ?

PURE Database and Data loader Issue

Hi, i am wondering how you set up the PURE database as it has a json file containing frame information and pgg data but the ppg timestamps dont correspond to image timestamps. So how do you get the coresponding ppg signal to the image frame.

Concern about the HR prediction

Hi,

I have a question regarding on the

def _calculate_fft_hr(ppg_signal, fs=60, low_pass=0.75, high_pass=2.5):
    """Calculate heart rate based on PPG using Fast Fourier transform (FFT)."""
    ppg_signal = np.expand_dims(ppg_signal, 0)
    N = _next_power_of_2(ppg_signal.shape[1])
    f_ppg, pxx_ppg = scipy.signal.periodogram(ppg_signal, fs=fs, nfft=N, detrend=False)
    fmask_ppg = np.argwhere((f_ppg >= low_pass) & (f_ppg <= high_pass))
    mask_ppg = np.take(f_ppg, fmask_ppg)
    mask_pxx = np.take(pxx_ppg, fmask_ppg)
    fft_hr = np.take(mask_ppg, np.argmax(mask_pxx, 0))[0] * 60
    return fft_hr

So here, I see the low_ pass is 0.75 and the high pass is 2.5 which means the lower bound of HR is 0.75 * 60 = 45 and the upper bound is 2.5 * 60 = 150. The (45,150) HR range is narrower than the possible range (40,220-age). Will that lead to a overfit based on the knowledge you know?

Best Regards,
Zechen Zhang

ask for help with adding resnet into TS-CAN

I want to replace the conv in TS-CAN with ResBlock, is there anyone could show me a detailed input and output shape of each moudle of TSCAN?

    def forward(self, inputs, params=None):
        diff_input = inputs[:, :3, :, :]
        raw_input = inputs[:, 3:, :, :]

        diff_input = self.TSM_1(diff_input)
        d1 = torch.tanh(self.motion_conv1(diff_input))
        d1 = self.TSM_2(d1)
        d2 = torch.tanh(self.motion_conv2(d1))

        r1 = torch.tanh(self.apperance_conv1(raw_input))
        r2 = torch.tanh(self.apperance_conv2(r1))

        g1 = torch.sigmoid(self.apperance_att_conv1(r2))
        g1 = self.attn_mask_1(g1)
        gated1 = d2 * g1

        d3 = self.avg_pooling_1(gated1)
        d4 = self.dropout_1(d3)

        r3 = self.avg_pooling_2(r2)
        r4 = self.dropout_2(r3)

        d4 = self.TSM_3(d4)
        d5 = torch.tanh(self.motion_conv3(d4))
        d5 = self.TSM_4(d5)
        d6 = torch.tanh(self.motion_conv4(d5))

        r5 = torch.tanh(self.apperance_conv3(r4))
        r6 = torch.tanh(self.apperance_conv4(r5))

        g2 = torch.sigmoid(self.apperance_att_conv2(r6))
        g2 = self.attn_mask_2(g2)
        gated2 = d6 * g2

        d7 = self.avg_pooling_3(gated2)
        d8 = self.dropout_3(d7)
        d9 = d8.view(d8.size(0), -1)
        d10 = torch.tanh(self.final_dense_1(d9))
        d11 = self.dropout_4(d10)
        out = self.final_dense_2(d11)

In different state of PURE, POS,ICA,...methods show strange results.

I followed the setting of yours and test POS, ICA, CHROM, GREEN on PURE, the overall results is similar to yours. But when I show the results of different states of PURE, such as steady(01), talking(02),...,medium rotation(06), the results are strange as following:
===Unsupervised Method ( ICA ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [02:19<00:00, 2.37s/it]
Used Unsupervised Method: ICA
State 01===>
Testing==> | me: -0.8789| std: 1.9653| mae: 1.0547| rmse: 2.1529| mer: 0.0138| p: 0.9963
State 02===>
Testing==> | me: -0.1953| std: 3.6067| mae: 2.3438| rmse: 3.6120| mer: 0.0404| p: 0.9900
State 03===>
Testing==> | me: -7.5586| std: 22.8666| mae: 8.3496| rmse: 24.0835| mer: 0.0715| p: 0.1075
State 04===>
Testing==> | me: -7.2070| std: 21.2272| mae: 8.7012| rmse: 22.4173| mer: 0.0827| p: 0.2836
State 05===>
Testing==> | me: -0.6152| std: 1.8457| mae: 0.6152| rmse: 1.9455| mer: 0.0045| p: 0.9995
State 06===>
Testing==> | me: -7.5586| std: 22.0926| mae: 7.5586| rmse: 23.3498| mer: 0.0585| p: 0.2988
State all===>
Testing==> | me: -4.0668| std: 16.2102| mae: 4.8116| rmse: 16.7125| mer: 0.0453| p: 0.7209
FFT MAE (FFT Label):4.811639300847458
FFT RMSE (FFT Label):16.71254207577304
FFT MAPE (FFT Label):4.534044786884735
FFT Pearson (FFT Label):0.7208838648526229
===Unsupervised Method ( POS ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [01:53<00:00, 1.93s/it]
Used Unsupervised Method: POS
State 01===>
Testing==> | me: 3.8672| std: 13.5089| mae: 5.4492| rmse: 14.0515| mer: 0.1107| p: 0.8198
State 02===>
Testing==> | me: 0.2930| std: 2.6204| mae: 1.2695| rmse: 2.6367| mer: 0.0241| p: 0.9948
State 03===>
Testing==> | me: 0.1758| std: 0.5273| mae: 0.1758| rmse: 0.5559| mer: 0.0031| p: 0.9997
State 04===>
Testing==> | me: 8.8330| std: 17.4286| mae: 8.8330| rmse: 19.5391| mer: 0.1773| p: 0.6721
State 05===>
Testing==> | me: 0.6152| std: 1.8457| mae: 0.6152| rmse: 1.9455| mer: 0.0096| p: 0.9971
State 06===>
Testing==> | me: 4.4824| std: 14.6408| mae: 5.1855| rmse: 15.3116| mer: 0.1013| p: 0.7947
State all===>
Testing==> | me: 3.0911| std: 11.4066| mae: 3.6274| rmse: 11.8180| mer: 0.0718| p: 0.8782
FFT MAE (FFT Label):3.627350370762712
FFT RMSE (FFT Label):11.817967292639915
FFT MAPE (FFT Label):7.180985775473761
FFT Pearson (FFT Label):0.878197263199433
===Unsupervised Method ( GREEN ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [01:20<00:00, 1.37s/it]
Used Unsupervised Method: GREEN
State 01===>
Testing==> | me: -5.0977| std: 13.3744| mae: 5.2734| rmse: 14.3130| mer: 0.0602| p: 0.8306
State 02===>
Testing==> | me: -9.5215| std: 21.2498| mae: 14.2090| rmse: 23.2855| mer: 0.1567| p: 0.5776
State 03===>
Testing==> | me: -7.9541| std: 23.8647| mae: 8.1299| rmse: 25.1554| mer: 0.0662| p: 0.0211
State 04===>
Testing==> | me: -9.5361| std: 20.6235| mae: 12.8760| rmse: 22.7215| mer: 0.1478| p: 0.3394
State 05===>
Testing==> | me: -6.6797| std: 26.9703| mae: 11.2500| rmse: 27.7851| mer: 0.1133| p: -0.1028
State 06===>
Testing==> | me: -8.2617| std: 26.0228| mae: 9.4922| rmse: 27.3027| mer: 0.0815| p: -0.0789
State all===>
Testing==> | me: -7.8133| std: 22.5463| mae: 10.1372| rmse: 23.8618| mer: 0.1034| p: 0.3410
FFT MAE (FFT Label):10.13721530720339
FFT RMSE (FFT Label):23.86176217201261
FFT MAPE (FFT Label):10.34061430399768
FFT Pearson (FFT Label):0.34103255024699397
===Unsupervised Method ( LGI ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [01:22<00:00, 1.40s/it]
Used Unsupervised Method: LGI
State 01===>
Testing==> | me: -5.1855| std: 13.3782| mae: 5.7129| rmse: 14.3480| mer: 0.0661| p: 0.8289
State 02===>
Testing==> | me: 0.5859| std: 2.9002| mae: 1.5625| rmse: 2.9588| mer: 0.0288| p: 0.9937
State 03===>
Testing==> | me: -7.1191| std: 22.0996| mae: 7.5586| rmse: 23.2179| mer: 0.0615| p: 0.1862
State 04===>
Testing==> | me: -9.6240| std: 22.8342| mae: 9.7119| rmse: 24.7795| mer: 0.0889| p: 0.1432
State 05===>
Testing==> | me: 1.5820| std: 4.7461| mae: 1.5820| rmse: 5.0028| mer: 0.0321| p: 0.9805
State 06===>
Testing==> | me: 0.0879| std: 1.9034| mae: 0.9668| rmse: 1.9054| mer: 0.0144| p: 0.9973
State all===>
Testing==> | me: -3.3443| std: 15.0113| mae: 4.5658| rmse: 15.3794| mer: 0.0490| p: 0.7719
FFT MAE (FFT Label):4.565843485169491
FFT RMSE (FFT Label):15.379369631243781
FFT MAPE (FFT Label):4.8959893981165665
FFT Pearson (FFT Label):0.7719467652076283
===Unsupervised Method ( PBV ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [01:20<00:00, 1.37s/it]
Used Unsupervised Method: PBV
State 01===>
Testing==> | me: -4.2188| std: 13.9012| mae: 5.9766| rmse: 14.5272| mer: 0.0720| p: 0.8161
State 02===>
Testing==> | me: -7.5195| std: 28.0459| mae: 13.7695| rmse: 29.0364| mer: 0.1468| p: 0.1160
State 03===>
Testing==> | me: 0.1318| std: 0.8578| mae: 0.4834| rmse: 0.8679| mer: 0.0081| p: 0.9992
State 04===>
Testing==> | me: 0.0439| std: 3.9475| mae: 1.8896| rmse: 3.9477| mer: 0.0322| p: 0.9836
State 05===>
Testing==> | me: 0.2637| std: 1.7600| mae: 0.7910| rmse: 1.7796| mer: 0.0158| p: 0.9974
State 06===>
Testing==> | me: -0.6152| std: 2.1185| mae: 1.3184| rmse: 2.2060| mer: 0.0216| p: 0.9958
State all===>
Testing==> | me: -1.8919| std: 12.8426| mae: 3.8731| rmse: 12.9812| mer: 0.0478| p: 0.8381
FFT MAE (FFT Label):3.8731461864406778
FFT RMSE (FFT Label):12.98118252450634
FFT MAPE (FFT Label):4.775171833897241
FFT Pearson (FFT Label):0.8381165109607617
===Unsupervised Method ( CHROM ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [01:20<00:00, 1.37s/it]
Used Unsupervised Method: CHROM
State 01===>
Testing==> | me: 8.3496| std: 18.1287| mae: 9.9316| rmse: 19.9591| mer: 0.2069| p: 0.6540
State 02===>
Testing==> | me: 1.8555| std: 5.2644| mae: 2.0508| rmse: 5.5818| mer: 0.0403| p: 0.9794
State 03===>
Testing==> | me: 0.3955| std: 0.7971| mae: 0.3955| rmse: 0.8898| mer: 0.0062| p: 0.9993
State 04===>
Testing==> | me: 8.7891| std: 17.4513| mae: 8.8770| rmse: 19.5396| mer: 0.1779| p: 0.6715
State 05===>
Testing==> | me: 4.1309| std: 14.5594| mae: 5.3613| rmse: 15.1340| mer: 0.1045| p: 0.8063
State 06===>
Testing==> | me: 6.5039| std: 15.2748| mae: 7.3828| rmse: 16.6018| mer: 0.1437| p: 0.7651
State all===>
Testing==> | me: 5.0574| std: 14.0384| mae: 5.7278| rmse: 14.9216| mer: 0.1145| p: 0.8091
FFT MAE (FFT Label):5.727787341101695
FFT RMSE (FFT Label):14.921567302513367
FFT MAPE (FFT Label):11.44912674073281
FFT Pearson (FFT Label):0.8090882471300135
###############################################################################
For different traditional methods, there are some states show very good results (bold ones). If that happened to you, or did I make a mistake?

some issues about train

Hello, I used PURE dataset to train and verify Physnet and UBFC dataset to test, and an error occurred during the training
Traceback (most recent call last):
File "/home/wnb/Public/rPPG-Toolbox-main/main.py", line 246, in
train_ and_ test(config, data_loader_dict)
File "/home/wnb/Public/rPPG-Toolbox-main/main.py", line 74, in train_ and_ test
model_ trainer.train(data_loader_dict)
File "/home/wnb/Public/rPPG-Toolbox-main/neural_methods/trainer/PhysnetTrainer.py", line 68, in train
loss.backward()
File "/home/sun/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/tensor.py", line 307, in backward
torch.autograd. backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/sun/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/autograd/init.py", line 150, in backward
grad tensors_ = _ make_ grads(tensors, grad_tensors_)
File "/home/sun/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/autograd/init.py", line 51, in _ make_ grads
raise RuntimeError("grad can be implicitly created only for scalar outputs")
RuntimeError: grad can be implicitly created only for scalar outputs
Then I checked the definition of loss and found that you are using Neg_ Pearson defines it as follows:
class Neg_ Pearson(nn.Module):
def __ init__ (self):
super(Neg_Pearson, self).__ init__ ()
return
def forward(self, preds, labels):
cos = nn. CosineSimilarity(dim=0, eps=1e-6)
pearson = cos(preds - preds.mean(dim=0, keepdim=True), labels - labels. mean(dim=0, keepdim=True))
return 1 - pearson
Different from the definition of negative correlation coefficient mentioned in the previous article, I don't know whether it will affect the experimental results.
Would you please tell me how to solve this problem? Thank you very much

Restruct codes for neural methods

Add trainer class and simplify main_neuralmethods.py
Add comments and docstrings

About the blood pressure

Hi guys,

You guys have done great work in predicting HR and RR. I am working on using the rPPG signal predicted to blood pressure. Do you have some recommendations regarding this?