• 📊 A benchmark focused on cross-dataset generalizability in abdominal multi-organ segmentation.
• 🧠 In-depth analysis on model generalizability across different data usage scenarios and the role of model size.

We train models on the official sets of FLARE22, AMOS, WORD, and TotalSegmentator, and evaluate them using their official validation sets as well as BTCV's official training set.

Note: While these datasets do have test sets, FLARE22, AMOS, and BTCV do not make their test labels publicly available. Therefore, for consistent evaluation, we use validation sets instead of test sets in A-Eval, regardless of label availability.

To ensure a meaningful and fair comparison across datasets, we evaluate the models’ performance based on a set of eight organ classes shared by all five datasets. We unify these labels using an overlapped label system. The corresponding code for label systems and label conversion can be found in the repository: label_systems.py and convert_label_2_overlap_label.py.

This project is released under the Apache 2.0 license.

• Special thanks go to the creators and maintainers of the public datasets that made our research possible:
  • FLARE22
  • AMOS
  • WORD
  • TotalSegmentator
  • BTCV
• Thanks to the SOTA framework of: nnUNet

• Hiring: We are hiring researchers, engineers, and interns in General Vision Group, Shanghai AI Lab. If you are interested in Medical Foundation Models and General Medical AI, including designing benchmark datasets, general models, evaluation systems, and efficient tools, please contact us.
• Global Collaboration: We're on a mission to redefine medical research, aiming for a more universally adaptable model. Our passionate team is delving into foundational healthcare models, promoting the development of the medical community. Collaborate with us to increase competitiveness, reduce risk, and expand markets.
• Contact: Junjun He([email protected]), Jin Ye([email protected]), and Tianbin Li ([email protected]).