l-yezhu / cdcd Goto Github PK

Hi, I was wondering if you have released the implementation for the genre accuracy metric.

Hi, I have some doubts about the usage of the beat coverage rate. As you mentioned in the D2M-GAN paper, the beat coverage rate is computed as the division of the generated beat number by the original beat number. From my point of view, this restricts the generated music from including too many beat keypoints (which has a high beat hit rate but results in poor performance). Under these preliminaries, I have two main questions.

1. The beat coverage rate can easily exceed 1 if the generated music has more beat keypoints than the original music, but I cannot find any beat coverage rate that is bigger than 1 both in the D2M-GAN and CDCD papers and I wonder the reason. By the way, in this case, is it true that the performance is better if the beat coverage rate is closer to 1 rather than bigger?
2. It seems that the final beat coverage rate is computed as the average of all samples. However, if a test set has two samples, of which the beat coverage rate is 0.5 and 1.5, respectively. In this case, the final beat coverage rate is 1, which seems to be perfect under this evaluation metric yet each sample performs unsatisfactorily. How to prevent such a phenomenon when using this evaluation metric?

I would appreciate it if you could figure out my misunderstanding regarding the beat coverage rate, and I'd like to have a further discussion about the evaluation of music beats/rhythms.

Hello,Do you have inference cub.py code?

I load the genre label.npy in the data folder, the shape is (6632,10), while the aist train dataset only has 2581 audios and motions. why?

extra_file_path in code1 is None. This results in the __getitem__() returns NoneType for negative_music in code2.

No Module named 'image_synthesis'

Hello,do you have the preprocessed AIST++ dataset?

Hi,
Thanks for your excellent work. I wonder how you processed the aist_s6 data? Do you plan to release it?
Thanks!

Hi, I am curious about the computation procedure of beat detection. It seems that the beats are computed by extracting the local maximums of the onset envelopes via librosa, which is more accurate to be regarded as the auditory rhythms from my own perspective. Since the librosa library also includes the official implementation of beat detection (librosa.beat.beat_track), which picks peaks in onset strength approximately consistent with estimated tempo, I wonder if the beat detection methods in the paper have a certain rationale for the dance-to-music scenarios, or if computing the hit rate of the onset maximums can reflect the performance of music generation more precisely? Thanks.

Dear Ms.Zhu:
I hope this email finds you well. My name is zhaoyang Zhang, and I am a fellow researcher in the field of generating music through dance, much like yourself. I recently came across your paper titled "Quantized GAN for Complex Music Generation from Dance Videos" and was particularly intrigued by the evaluation metric you proposed, specifically the genre accuracy metric.
I am currently in the process of conducting experiments with my own models, and I believe that testing the code implementation of your genre accuracy metric would greatly benefit my research. I am writing to kindly request access to the code used to execute this metric in your paper, as it would provide invaluable assistance in ensuring the robustness and accuracy of my own experiments.
I understand that sharing code can be sensitive, but I assure you that I will utilize it solely for academic purposes and will not distribute it without your explicit permission. Any assistance you could provide would be immensely appreciated and duly acknowledged in my work.

Thank you very much for considering my request. I look forward to hearing from you at your earliest convenience.

Warm regards,
Zhaoyang Zhang CUC
[email protected]

Hi Ye,

Thank you for this great work.

When I followed the installation instruction and tried to setup the env with

cd ./synthesis/modeling/models/
pip install -e .

the error as follow:

But I can find the requirements.txt file at that path. Did I miss anything?
Thanks.