Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model:
|
| 3 |
+
- Qwen/Qwen2.5-VL-72B-Instruct
|
| 4 |
+
language:
|
| 5 |
+
- en
|
| 6 |
+
license: apache-2.0
|
| 7 |
+
tags:
|
| 8 |
+
- transformers
|
| 9 |
+
- multimodal
|
| 10 |
+
pipeline_tag: visual-question-answering
|
| 11 |
+
---
|
| 12 |
+
# INFRL-Qwen2.5-VL-72B-Preview
|
| 13 |
+
## Model Overview
|
| 14 |
+
- **INFRL-Qwen2.5-VL-72B-Preview** improves visual reasoning upon [Qwen2.5-VL-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct) model.
|
| 15 |
+
|
| 16 |
+
- As of March 25th, 2025, **INFRL-Qwen2.5-VL-72B-Preview** is the best-performing open-sourced VL model on various visual reasoning benchmarks ([MathVision](https://mathllm.github.io/mathvision/),[MathVista](https://mathvista.github.io/), [MathVerse](https://mathverse-cuhk.github.io/)).
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
## Evaluation
|
| 20 |
+
|
| 21 |
+
| Models | MathVision (test) | MathVista (testmini) | MathVerse (testmini) |
|
| 22 |
+
|-------------------|-------------------|----------------------|----------------------|
|
| 23 |
+
| GPT4o | 30.6 | 60 | 41.2 |
|
| 24 |
+
| Gemini-2.0-Flash | 41.3 | 70.1 | 50.6 |
|
| 25 |
+
| Claude 3.5 Sonnet | 33.5 | 67.7 | 47.8 |
|
| 26 |
+
| QvQ-72B | 35.9 | 71.4 | 48.6 |
|
| 27 |
+
| InternVL2.5-78B | 34.9 | 72.3 | 51.7 |
|
| 28 |
+
| Qwen-VL-2.5-72B | 38.1 | 74.8 | 57.18 |
|
| 29 |
+
| INFRL-VL-Preview | 41.9 | 77.8 | 58.84 |
|
| 30 |
+
|
| 31 |
+
We will release a code repository for VLM evaluation. It supports RL training with simple rule-based rewards, meanwhile aligning with LLM-Judge results.
|
| 32 |
+
|
| 33 |
+
Stay tuned!
|
| 34 |
+
|
| 35 |
+
## Contributors
|
| 36 |
+
### Supervisors
|
| 37 |
+
Wei Chu • Yuan Qi
|
| 38 |
+
|
| 39 |
+
### VL Team
|
| 40 |
+
Haozhe Wang • Zuming Huang
|
| 41 |
+
|
| 42 |
+
### RL Team
|
| 43 |
+
Haozhe Wang • Chao Qu • Long Li
|
| 44 |
+
|
| 45 |
+
## Thanks
|
| 46 |
+
Thanks to Jiaran Hao, Liuyihan Song for supports in the RL infrastructure.
|
| 47 |
+
|
| 48 |
+
## Citation
|
| 49 |
+
If you find our model useful, please consider citing:
|
| 50 |
+
|
| 51 |
+
```
|
| 52 |
+
@misc {INFRL_VL_Preview,
|
| 53 |
+
author = { {Wang, Haozhe and Huang, Zuming and Qu, Chao and Chu, Wei and Qi, Yuan} },
|
| 54 |
+
title = { INFRL-Qwen2.5-VL-72B-Preview },
|
| 55 |
+
year = 2025,
|
| 56 |
+
url = { https://huggingface.co/infly/INFRL-Qwen2.5-VL-72B-Preview},
|
| 57 |
+
publisher = { Hugging Face }
|
| 58 |
+
}
|
| 59 |
+
```
|