File size: 8,282 Bytes
0a52369 1cac55b 0a52369 f8e0289 0a52369 f8e0289 0a52369 f8e0289 0a52369 f8e0289 0a52369 f8e0289 1cac55b f8e0289 0a52369 f8e0289 0a52369 f8e0289 729869e f8e0289 729869e f8e0289 729869e f8e0289 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 |
---
license: cc-by-nc-nd-4.0
tags:
- mass-spectrometry
- molecular-formula
- dissolved-organic-matter
- knn
- scikit-learn
library_name: sklearn
pipeline_tag: feature-extraction
---
# DOM Formula Assignment using K-Nearest Neighbors
[](https://github.com/pcdslab/dom-formula-assignment-using-ml)
**A Machine Learning Approach to Enhanced Molecular Formula Assignment in Fulvic Acid DOM Mass Spectra**
> **Paper**: Under review
---
## Abstract
Dissolved organic matter (DOM) is a critical component of aquatic ecosystems, with the fulvic acid fraction (FA-DOM) exhibiting high mobility and ready bioavailability to microbial communities. While understanding the molecular composition is a vital area of study, the heterogeneity of the material, with a vast number of diverse compounds, makes this task challenging. Existing methods often struggle with incomplete formula assignment or reduced coverage highlighting the need for a better approach. In this study, we developed a machine learning approach using the k-nearest neighbors (KNN) algorithm to predict molecular formulas from ultra-high-resolution mass spectrometry data. The model was trained on chemical formulas assigned to multiple DOM samples using 7 Tesla(7T) and a 21 Tesla(21T) Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR MS) system, and tested on an independent 9.4 T FT-ICR MS Fulvic Acid dataset. A synthetic dataset of plausible elemental combinations (C, H, O, N, S) was also generated to enhance generalization. Our approach achieved a 99.9% assignment rate on the labeled test set and assigned a total of 13,605 formulas for unlabeled peaks compared to the existing approach, which assigned 5914 formulas, achieving up to a 2.3X improvement in formula assignment coverage compared to existing methods.
---
## Model Variants
### Single Models (8 variants)
Trained on individual datasets (7T or 21T FT-ICR MS data):
| Data Source | K | Metric | Variant Name |
|-------------|---|--------|--------------|
| 7T | 1 | Euclidean | `knn_7T_k1_euclidean` |
| 7T | 1 | Manhattan | `knn_7T_k1_manhattan` |
| 7T | 3 | Euclidean | `knn_7T_k3_euclidean` |
| 7T | 3 | Manhattan | `knn_7T_k3_manhattan` |
| 21T | 1 | Euclidean | `knn_21T_k1_euclidean` |
| 21T | 1 | Manhattan | `knn_21T_k1_manhattan` |
| 21T | 3 | Euclidean | `knn_21T_k3_euclidean` |
| 21T | 3 | Manhattan | `knn_21T_k3_manhattan` |
### Ensemble Models (8 variants)
Each combines multiple sub-models trained on different data versions:
| Data Source | K | Metric | Variant Name | Sub-models |
|-------------|---|--------|--------------|------------|
| **7T-21T** | 1 | Euclidean | `knn_7T21T_k1_euclidean_ensemble` | 2 (ver2+ver3) |
| **7T-21T** | 1 | Manhattan | `knn_7T21T_k1_manhattan_ensemble` | 2 (ver2+ver3) |
| **7T-21T** | 3 | Euclidean | `knn_7T21T_k3_euclidean_ensemble` | 2 (ver2+ver3) |
| **7T-21T** | 3 | Manhattan | `knn_7T21T_k3_manhattan_ensemble` | 2 (ver2+ver3) |
| **Synthetic** | 1 | Euclidean | `knn_Synthetic_k1_euclidean_ensemble` | 3 (ver2+ver3+synth) |
| **Synthetic** | 1 | Manhattan | `knn_Synthetic_k1_manhattan_ensemble` | 3 (ver2+ver3+synth) |
| **Synthetic** | 3 | Euclidean | `knn_Synthetic_k3_euclidean_ensemble` | 3 (ver2+ver3+synth) |
| **Synthetic** | 3 | Manhattan | `knn_Synthetic_k3_manhattan_ensemble` | 3 (ver2+ver3+synth) |
---
## Performance
Results on combined test sets (Suwannee River Fulvic Acid + Pahokee River Fulvic Acid + others):
| Model | True Predictions | New Assignments | False Predictions | **Assignment Rate** |
|-------|-----------------|-----------------|-------------------|---------------------|
| **Synthetic (K=1, Euclidean)** | 2,623 | 1,423 | 1 | **99.975%** |
| **Synthetic (K=1, Manhattan)** | 2,623 | 1,423 | 1 | **99.975%** |
| **Synthetic (K=3, Euclidean)** | 2,631 | 1,415 | 1 | **99.975%** |
| **Synthetic (K=3, Manhattan)** | 2,631 | 1,415 | 1 | **99.975%** |
| **7T-21T (K=1, Euclidean)** | 3,851 | 8 | 188 | **95.355%** |
| **7T-21T (K=1, Manhattan)** | 3,851 | 8 | 188 | **95.355%** |
| **7T-21T (K=3, Euclidean)** | 3,846 | 10 | 191 | **95.280%** |
| **7T-21T (K=3, Manhattan)** | 3,846 | 10 | 191 | **95.280%** |
| 21T (K=1, Euclidean) | 3,835 | 10 | 202 | 95.009% |
| 21T (K=1, Manhattan) | 3,835 | 10 | 202 | 95.009% |
| 21T (K=3, Euclidean) | 3,831 | 11 | 205 | 94.935% |
| 21T (K=3, Manhattan) | 3,831 | 11 | 205 | 94.935% |
| 7T (K=1, Euclidean) | 3,201 | 6 | 840 | 79.244% |
| 7T (K=1, Manhattan) | 3,201 | 6 | 840 | 79.244% |
| 7T (K=3, Euclidean) | 3,201 | 6 | 840 | 79.244% |
| 7T (K=3, Manhattan) | 3,201 | 6 | 840 | 79.244% |
**Key Findings**:
- **Synthetic models** achieve highest assignment rate (99.975%) and make many new predictions (1,423 novel formulas)
- **7T-21T ensemble models** provide best performance for real DOM samples (95.4% with only 8 new assignments)
- **Recommended for most users**: 7T-21T ensemble (K=1) - optimal balance of accuracy and confidence
---
## Quick Start
### Installation
```bash
pip install transformers huggingface_hub joblib scikit-learn
```
### Load Default Model
```python
from transformers import AutoModel
import numpy as np
# Load best model (7T-21T, K=1, Euclidean)
model = AutoModel.from_pretrained(
"SaeedLab/dom-formula-assignment-using-knn",
trust_remote_code=True
)
# Prepare mass data
masses = np.array([[245.1234], [387.2156], [512.3478]])
# Get formula predictions
predictions = model(masses)
print(predictions)
# Output: ['C12H15O6' 'C20H31O8' 'C28H48O9']
```
### Load Specific Variant
```python
# Load 21T model with K=1 and Euclidean distance
model = AutoModel.from_pretrained(
"SaeedLab/dom-formula-assignment-using-knn",
data_source="21T",
k_neighbors=1,
metric="euclidean",
trust_remote_code=True
)
# Load 7T-21T ensemble (automatically loads 2 sub-models)
model = AutoModel.from_pretrained(
"SaeedLab/dom-formula-assignment-using-knn",
data_source="7T-21T",
k_neighbors=1,
metric="euclidean",
trust_remote_code=True
)
```
### Batch Prediction
```python
import pandas as pd
# Load your peak list
peaks = pd.read_csv("my_peaks.csv")
masses = peaks['m/z'].values.reshape(-1, 1)
# Predict formulas
formulas = model(masses)
# Add to dataframe
peaks['formula'] = formulas
peaks.to_csv("annotated_peaks.csv", index=False)
```
---
## Model Selection Guide
| Use Case | Recommended Model | Why? |
|----------|-------------------|------|
| **Real DOM samples (best overall)** | 7T-21T ensemble (K=1) | Highest verified accuracy (95.4%), minimal new assignments |
| **Maximum assignment rate** | Synthetic ensemble (K=1) | 99.98% assignment rate (note: makes many novel predictions) |
| **21T data only** | 21T (K=1, Euclidean) | Optimized for 21T instrument data |
| **7T data only** | 7T (K=1, Euclidean) | Optimized for 7T instrument data |
| **Synthetic/simulated data** | Synthetic ensemble | Trained on computationally generated formulas |
## License
This model and associated code are released under the CC-BY-NC-ND 4.0 license and may only be used for non-commercial, academic research purposes with proper attribution. Any commercial use, sale, or other monetization of this model and its derivatives, which include models trained on outputs from the model or datasets created from the model, is prohibited and requires prior approval. Downloading the model requires prior registration on Hugging Face and agreeing to the terms of use. By downloading this model, you agree not to distribute, publish or reproduce a copy of the model. If another user within your organization wishes to use the model, they must register as an individual user and agree to comply with the terms of use. Users may not attempt to re-identify the deidentified data used to develop the underlying model. If you are a commercial entity, please contact the corresponding author.
---
## Contact
For any additional questions or comments, contact Fahad Saeed (fsaeed@fiu.edu).
---
|