phellonchen / x-llm Goto Github PK

View Code? Open in Web Editor NEW

300.0 300.0 16.0 6.18 MB

X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languages

Home Page: https://x-llm.github.io

License: Apache License 2.0

Python 98.63% Shell 0.12% Jupyter Notebook 1.25%

x-llm's People

Contributors

Stargazers

Watchers

Forkers

cicichang apollohuang1 rayjue 2132660698 gongyong728125 dx2048 jjhw techthiyanes xsean2020 neverstoplearn whuhxb xmu-xiaoma666 zhikangniu brunoscaglione cillinzhang itsvisky

x-llm's Issues

No way to train speech interface!

Hello, the speech datasets' links for json and features all link back to the xllm repo. The speech encoder link also links back to the main page of xllm. Where do we find the blip2 checkpoints? Do we use their Q former only?

Wrong link for video Chinese json file

It seems that the Chinese json files for video datasets have wrong links. Could you please correct them?

how to link the image id in DATA JSON to the image in IMAGE URLS for WuKong

When the code will be released?

The url of "data json" for Visual Genome is incorrect

how to link the image id

I have a graphic dataset similar to coco's Chinese description, and I can directly run image_interface_stage2.sh on my data

instruction dataset

When will the instruction dataset be published?

Wukong captions' image datasets url is error.

Hi, handsome boy:
your Wukong captions' image datasets url is error, you can correct.

I-Adapter, V-Adapter and S-Adapter

The paper was unclear about how the encoded modalities are adapted through these adapters. Can you explain and point to the code where I-Adapter, V-Adapter, and S-Adapter are used and information about them, like the LoRA adapters or other networks?

Data

Hi, Thanks for your amazing work! I wonder when will you release your code and data? Thanks!

when will release the pretrained model?

hello, when will the pretrained model be released?

How can I try the llm after install?

Release Code & Data