Update README.md
Browse files
README.md
CHANGED
|
@@ -12,15 +12,15 @@ base_model:
|
|
| 12 |
base_model_relation: quantized
|
| 13 |
---
|
| 14 |
# Qwen3-Coder-30B-A3B-Instruct-AWQ
|
| 15 |
-
|
| 16 |
|
| 17 |
|
| 18 |
-
###
|
| 19 |
-
|
| 20 |
|
| 21 |
|
| 22 |
-
### 【
|
| 23 |
-
<i
|
| 24 |
```
|
| 25 |
CONTEXT_LENGTH=32768
|
| 26 |
|
|
@@ -40,33 +40,35 @@ vllm serve \
|
|
| 40 |
--port 8000
|
| 41 |
```
|
| 42 |
|
| 43 |
-
###
|
| 44 |
|
| 45 |
```
|
| 46 |
vllm==0.10.0
|
| 47 |
```
|
| 48 |
|
| 49 |
-
###
|
| 50 |
-
```
|
|
|
|
|
|
|
| 51 |
2025-08-01
|
| 52 |
1. 首次commit
|
| 53 |
```
|
| 54 |
|
| 55 |
-
###
|
| 56 |
|
| 57 |
| 文件大小 | 最近更新时间 |
|
| 58 |
|--------|--------------|
|
| 59 |
| `16GB` | `2025-08-01` |
|
| 60 |
|
| 61 |
|
| 62 |
-
###
|
| 63 |
|
| 64 |
```python
|
| 65 |
from modelscope import snapshot_download
|
| 66 |
snapshot_download('tclf90/Qwen3-Coder-30B-A3B-Instruct-AWQ', cache_dir="本地路径")
|
| 67 |
```
|
| 68 |
|
| 69 |
-
###
|
| 70 |
# Qwen3-Coder-30B-A3B-Instruct
|
| 71 |
<a href="https://chat.qwen.ai/" target="_blank" style="margin: 2px;">
|
| 72 |
<img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
|
|
|
|
| 12 |
base_model_relation: quantized
|
| 13 |
---
|
| 14 |
# Qwen3-Coder-30B-A3B-Instruct-AWQ
|
| 15 |
+
Base model [Qwen3-Coder-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct)
|
| 16 |
|
| 17 |
|
| 18 |
+
### ❗❗Reminder❗❗
|
| 19 |
+
This model suffers from significant loss under 4-bit quantization, please use with caution.
|
| 20 |
|
| 21 |
|
| 22 |
+
### 【vLLM Single Node with 4 GPUs Startup Command】
|
| 23 |
+
<i>Note: You must use `--enable-expert-parallel` to start this model, otherwise the expert tensor TP will not divide evenly. This is required even for 2 GPUs.</i>
|
| 24 |
```
|
| 25 |
CONTEXT_LENGTH=32768
|
| 26 |
|
|
|
|
| 40 |
--port 8000
|
| 41 |
```
|
| 42 |
|
| 43 |
+
### 【Dependencies】
|
| 44 |
|
| 45 |
```
|
| 46 |
vllm==0.10.0
|
| 47 |
```
|
| 48 |
|
| 49 |
+
### 【Model Update Date】
|
| 50 |
+
```
|
| 51 |
+
2025-08-19
|
| 52 |
+
1.[BugFix] Fix compatibility issues with vLLM 0.10.1
|
| 53 |
2025-08-01
|
| 54 |
1. 首次commit
|
| 55 |
```
|
| 56 |
|
| 57 |
+
### 【Model Files】
|
| 58 |
|
| 59 |
| 文件大小 | 最近更新时间 |
|
| 60 |
|--------|--------------|
|
| 61 |
| `16GB` | `2025-08-01` |
|
| 62 |
|
| 63 |
|
| 64 |
+
### 【Model Download】
|
| 65 |
|
| 66 |
```python
|
| 67 |
from modelscope import snapshot_download
|
| 68 |
snapshot_download('tclf90/Qwen3-Coder-30B-A3B-Instruct-AWQ', cache_dir="本地路径")
|
| 69 |
```
|
| 70 |
|
| 71 |
+
### 【Overview】
|
| 72 |
# Qwen3-Coder-30B-A3B-Instruct
|
| 73 |
<a href="https://chat.qwen.ai/" target="_blank" style="margin: 2px;">
|
| 74 |
<img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
|