ulmentflam commited on
Commit
3a374d4
·
verified ·
1 Parent(s): 2d4bf89

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -16,7 +16,7 @@ pipeline_tag: text-generation
16
 
17
  # QwerkyLlamaMambaHybrid
18
 
19
- This is a hybrid Mamba-Transformer model based on the Llama 3.2 architecture, distilled from Llama 3.3 70B into a 8B parameter model using Qwerky's proprietary distillation method. The model uses MAMBA layers interleaved with attention layers for efficient sequence modeling. The results are a 3B parameter model comparable in quality to Llama's 3.2 3B but running at speeds as fast or faster than Llama's 3.2 1B model.
20
 
21
  **Model Developer**: Qwerky AI
22
 
 
16
 
17
  # QwerkyLlamaMambaHybrid
18
 
19
+ This is a hybrid Mamba-Transformer model based on the Llama 3.2 architecture, distilled from Llama 3.3 70B into a 8B parameter model using Qwerky's proprietary distillation method. The model uses MAMBA layers interleaved with attention layers for efficient sequence modeling. The results are a 8B parameter model comparable in quality to Llama's 3.2 8B but running at speeds as fast or faster than Llama's 3.2 3B model.
20
 
21
  **Model Developer**: Qwerky AI
22