Comments (1)
Following the corresponding papers, they both use Visual Genome for data augmentation (bottom-up) or feature learning (both).
In Section 4.1.1 of Anderson et al. (CVPR 2018), they mentioned as:
When training the VQA model, we augment the VQA v2.0 training data with Visual Genome question and answer pairs provided the correct answer is present in modelโs answer vocabulary. This represents about 30% of the available data, or 485K questions.
For the image feature learning, both approaches use bottom-up attention image features which are pretrained using Visual Genome (as we did). Bottom-up (Anderson et al., 2018) uses Visual Genome for feature learning and training, while Counter is for feature learning. Therefore we cannot say that Counter is not available to the external data, Visual Genome. Besides this aspect, the Counter model is mainly to compare with Number score (Although we can show that BAN with Counter outperforms Original Counter without the augmentation of Visual Genome. See the table below).
In the 30% of Visual Genome, the ratio of counting question is approximately 11.5%.
Without Visual Genome augmentation in training and evaluate on test-dev split,
BAN (69.52 -> 69.14), BAN+Glove (69.66 -> 69.27), BAN+Glove+Counter (70.04 -> 69.58).
VQA 2.0 Test-dev results without Visual Genome augmentation:
| Overall | Yes/no | Number | Other
------------------+---------+---------+---------+--------
BAN | 69.14 | 85.03 | 49.80 | 59.97
BAN+Glove | 69.27 | 85.19 | 49.94 | 60.08
BAN+Glove+Counter | 69.58 | 85.18 | 52.76 | 60.06
from ban-vqa.
Related Issues (20)
- Why the learning rate is different with or without evalLoader ? HOT 3
- What does the 'xhyk,bvk,bqk->bhvq' mean??? HOT 6
- Can not download the image feature HOT 2
- train36_imgid2idx.pkl file HOT 1
- Annotation and Sentence dataset download
- How to get labels for objects? HOT 2
- can not found "Annotation and Sentence files to data/flickr30k/Flickr30kEntities.tar.gz. " HOT 1
- error when using adaptive_detection_features_converter.py HOT 2
- Evaluate.py HOT 1
- test.py HOT 1
- tar cache.pkl.tgz error, when downloading Pickle caches for the pretrained model HOT 1
- Attention Visualization HOT 1
- where to find "val_flickr30k_resnet101_faster_rcnn_genome.tsv.3"? HOT 1
- flickr 30k features download HOT 4
- how to get the files HOT 1
- Download Flickr30k features HOT 3
- Pretrained model for Flickr30k HOT 2
- Error in Flickr30k features HOT 2
- link no longer works HOT 2
- Trouble creating ID.pkls HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ban-vqa.