Part of LLMETA project, we implement Relation Transformer which can classify even deeper. LLMETA project is for metadata extraction from text (research paper), also in image too. But, as you can see, we didn't extract any metadata like file size or things like that we normally know about.

We propose a new metadata for image as scene graph, which can use various research field. But, our RelTR-based model can classify human emotions, age, races and also various breeds based on ImageNet classes. More details can be found in our brief paper.

Network of Deeper RelTR Model

Download Deeper_RelTR Repo with:

git clone https://github.com/KBH00/Deeper_RelTR.git
cd Deeper_RelTR

# create a conda environment
conda create -n drm python=3.11
conda activate drm

# install packages
# for windows
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121conda install matplotlib

# for linux
pip3 install torch torchvision torchaudio

# other pakages
pip install deepface
pip install tf-keras
pip install transformers
pip install scipy
pip install matplotlib

conda env create --file environment.yml

pip install -r requirements.txt

a) Download RelTR model pretrained on the Visual Genome dataset and put it under

ckpt/checkpoint0149.pth

b) Infer the relationships in an image with the command:

python img_inference.py --img_path $IMAGE_PATH --resume $MODEL_PATH 

We attached some images from internet. You can also test with your customized image.

you can see the results that can classify man to 22 year old happy latino man.

also even in animal breeds, but sometimes the accuracy of human emotions is lower than expected. which can turn on or off from our system.

number of classifiable labels

Comparsion on V6 and detailed test set

https://github.com/yrcong/RelTR

https://github.com/serengil/deepface