jingkang50 / funqa Goto Github PK

Hi, I'm confused about the purpose of this file, looking forward to your reply~

Where are the original videos and extended datasets mentioned in Part 2 that I didn't find in the relevant links?

hi ~thanks for releasing this excellent work to the public! But it seems that there is only annotations json file released, how can i donwload the videos in train/val/test sets.

Hi @Nicous20 and everyone,
While going through the test sample and annotations, I found that the ground truth start and end time stamps are not matching with the test sample duration.

Can you please provide us with the annotations aligned within the sample video durations? It would be helpful to calculate IoU scores in localization tasks over this valuable dataset. Otherwise, we cannot match the ground truth time span with the prediction time span.

Thanks.

Hi! Thanks for the exciting jobs! I have noticed another multi-choice version dataset but can not find the annotation files. Will it be released? 😁

chk_answer = []
    for data in answer:
        chk_answer.append({
            'task': data['task'],
            'output': data['output'],
            'instruction': data['instruction'],
            'ID': data['ID']
        })

    diff = False
    for data in submission:
        if {
                'task': data['task'],
                'output': data['output'],
                'instruction': data['instruction'],
                'ID': data['ID']
        } not in chk_answer:
            diff = True
            break

    assert diff == False, 'Submission file is not valid'
    print('File is valid! Loading File...')

The 'output' value of submission and gt answer is probably different. So there will definitely be an AssertionError.

Thanks for your awesome work! I have downloaded the dataset but find that there is no samples with task type of M4.