satyaki-mitra
commited on
Commit
ยท
aaa179d
1
Parent(s):
edf1149
README uopdated
Browse files- README.md +382 -1087
- docs/BLOGPOST.md +182 -0
README.md
CHANGED
|
@@ -1,643 +1,394 @@
|
|
|
|
|
| 1 |
# ๐ AI Text Authentication Platform
|
| 2 |
-
## Enterprise
|
| 3 |
|
| 4 |

|
| 5 |

|
| 6 |
-

|
| 8 |
-

|
| 15 |
- [Key Differentiators](#-key-differentiators)
|
| 16 |
- [System Architecture](#-system-architecture)
|
|
|
|
| 17 |
- [Detection Metrics & Mathematical Foundation](#-detection-metrics--mathematical-foundation)
|
| 18 |
- [Ensemble Methodology](#-ensemble-methodology)
|
|
|
|
|
|
|
| 19 |
- [Project Structure](#-project-structure)
|
| 20 |
- [API Endpoints](#-api-endpoints)
|
| 21 |
-
- [Domain-Aware Detection](#-domain-aware-detection)
|
| 22 |
-
- [Performance Characteristics](#-performance-characteristics)
|
| 23 |
- [Installation & Setup](#-installation--setup)
|
| 24 |
-
- [
|
| 25 |
-
- [Accuracy & Validation](#-accuracy--validation)
|
| 26 |
- [Frontend Features](#-frontend-features)
|
|
|
|
| 27 |
- [Business Model & Market Analysis](#-business-model--market-analysis)
|
| 28 |
-
- [Future
|
| 29 |
-
- [
|
|
|
|
|
|
|
| 30 |
|
| 31 |
---
|
| 32 |
|
| 33 |
-
##
|
| 34 |
-
|
| 35 |
-
The **AI Text Authentication Platform** is a system designed to identify AI-generated content across multiple domains with exceptional accuracy. The platform addresses the growing challenge of content authenticity in education, publishing, hiring, and research sectors.
|
| 36 |
-
|
| 37 |
-
### What Makes This Platform Unique?
|
| 38 |
-
|
| 39 |
-
The system employs a **sophisticated ensemble of 6 complementary detection metrics** with **domain-aware calibration**, achieving **~90% accuracy** while maintaining computational efficiency, real-time performance, and complete explainability. Unlike traditional single-metric detectors, our platform analyzes text through multiple independent lenses to capture orthogonal signals that AI-generated content exhibits.
|
| 40 |
-
|
| 41 |
-
### Core Capabilities
|
| 42 |
|
| 43 |
-
**
|
| 44 |
-
- **Academic Domain**: Optimized for essays, research papers, and scholarly writing with specialized linguistic pattern recognition
|
| 45 |
-
- **Technical Documentation**: Calibrated for medical papers, technical manuals, and documentation with high-precision thresholds
|
| 46 |
-
- **Creative Writing**: Tuned for stories, narratives, and creative content with burstiness detection
|
| 47 |
-
- **Social Media**: Adapted for informal writing, blogs, and conversational text with relaxed linguistic requirements
|
| 48 |
|
| 49 |
-
|
| 50 |
-
1. **Automatic Domain Classification**: Intelligent identification of content type to apply appropriate detection parameters
|
| 51 |
-
2. **Multi-Metric Analysis**: Parallel execution of 6 independent metrics capturing different aspects of text generation
|
| 52 |
-
3. **Ensemble Aggregation**: Confidence-calibrated weighted voting with uncertainty quantification
|
| 53 |
-
4. **Model Attribution**: Identifies specific AI models (GPT-4, Claude, Gemini, LLaMA, etc.) with confidence scores
|
| 54 |
-
5. **Explainable Results**: Sentence-level highlighting with detailed reasoning and evidence presentation
|
| 55 |
|
| 56 |
-
|
| 57 |
-
- **High Performance**: Analyzes 100-500 word texts in 1.2 seconds with parallel computation
|
| 58 |
-
- **Scalable Architecture**: Auto-scaling infrastructure supporting batch processing and high-volume requests
|
| 59 |
-
- **Multi-Format Support**: Handles PDF, DOCX, TXT, DOC, and MD files with automatic text extraction
|
| 60 |
-
- **RESTful API**: Comprehensive API with authentication, rate limiting, and detailed documentation
|
| 61 |
-
- **Real-Time Dashboard**: Interactive web interface with dual-panel design and live analysis
|
| 62 |
-
- **Comprehensive Reporting**: Downloadable JSON and PDF reports with complete analysis breakdown
|
| 63 |
-
|
| 64 |
-
### Problem Statement & Market Context
|
| 65 |
|
| 66 |
-
|
| 67 |
-
- 60% of students regularly use AI tools for assignments
|
| 68 |
-
- 89% of teachers report AI-written submissions
|
| 69 |
-
- Traditional assessment methods becoming obsolete
|
| 70 |
|
| 71 |
-
**
|
| 72 |
-
- AI-generated applications masking true candidate qualifications
|
| 73 |
-
- Remote hiring amplifying verification challenges
|
| 74 |
|
| 75 |
-
**
|
| 76 |
-
- AI-generated articles flooding publishing platforms
|
| 77 |
-
- SEO manipulation through AI content farms
|
| 78 |
-
- Trust erosion in digital content ecosystems
|
| 79 |
|
| 80 |
-
**
|
| 81 |
-
- **Total Addressable Market**: $20B with 42% YoY growth
|
| 82 |
-
- **Education Sector**: $12B (45% growth rate)
|
| 83 |
-
- **Enterprise Hiring**: $5B (30% growth rate)
|
| 84 |
-
- **Content Publishing**: $3B (60% growth rate)
|
| 85 |
|
| 86 |
---
|
| 87 |
|
| 88 |
## ๐ฏ Key Differentiators
|
| 89 |
|
| 90 |
| Feature | Description | Impact |
|
| 91 |
-
|
| 92 |
-
|
|
| 93 |
-
|
|
| 94 |
-
|
|
| 95 |
-
|
|
| 96 |
-
|
|
| 97 |
-
|
|
| 98 |
|
| 99 |
---
|
| 100 |
|
| 101 |
## ๐๏ธ System Architecture
|
| 102 |
|
| 103 |
-
###
|
| 104 |
-
|
| 105 |
-
```
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
Model Attribution โ Sentence Highlighting
|
| 150 |
-
โ
|
| 151 |
-
Reasoning Generation โ Report Creation
|
| 152 |
-
โ
|
| 153 |
-
API Response (JSON/PDF)
|
| 154 |
-
```
|
| 155 |
|
| 156 |
---
|
| 157 |
|
| 158 |
-
##
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
|
| 179 |
-
|
| 180 |
-
**What It Captures**:
|
| 181 |
-
- **Perplexity**: Measures how surprised a language model is by the text. AI-generated text follows learned probability distributions closely, resulting in lower perplexity (15-30), while human writing exhibits creative unpredictability with higher perplexity (40-80).
|
| 182 |
-
- **Entropy**: Quantifies token-level randomness and vocabulary diversity. AI models tend toward repetitive token selection patterns (2.8-3.8 bits/token), whereas humans use more varied vocabulary (4.2-5.5 bits/token).
|
| 183 |
-
|
| 184 |
-
**Domain Manifestations**:
|
| 185 |
-
- **Academic**: Human papers show higher entropy in technical terminology selection, varied sentence starters
|
| 186 |
-
- **Technical**: AI documentation exhibits predictable term sequences; humans show domain expertise through unexpected connections
|
| 187 |
-
- **Creative**: Human creativity produces higher entropy in word choice; AI follows genre conventions rigidly
|
| 188 |
-
- **Social Media**: Humans use slang, abbreviations unpredictably; AI maintains consistent formality
|
| 189 |
-
|
| 190 |
-
#### Dimension 2: Structural & Syntactic Patterns
|
| 191 |
-
**Metrics Involved**: Structural (15%), Linguistic (15%)
|
| 192 |
-
|
| 193 |
-
**What It Captures**:
|
| 194 |
-
- **Structural**: Analyzes sentence length variance (burstiness), paragraph uniformity, and formatting consistency. AI generates overly uniform structures, while humans naturally vary their writing rhythm.
|
| 195 |
-
- **Linguistic**: Evaluates POS tag diversity, parse tree depth, and grammatical sophistication. AI models produce predictable syntactic patterns, whereas humans exhibit more complex and varied grammatical structures.
|
| 196 |
-
|
| 197 |
-
**Domain Manifestations**:
|
| 198 |
-
- **Academic**: AI papers show uniform paragraph lengths; humans vary based on argument complexity
|
| 199 |
-
- **Technical**: AI maintains consistent sentence structure in procedures; humans adjust complexity for concept difficulty
|
| 200 |
-
- **Creative**: Humans use burstiness for dramatic effect (short sentences in action, longer in description); AI averages out
|
| 201 |
-
- **Social Media**: Human posts vary wildly in length/structure; AI maintains unnatural consistency
|
| 202 |
-
|
| 203 |
-
#### Dimension 3: Semantic Coherence & Content Stability
|
| 204 |
-
**Metrics Involved**: Semantic Analysis (15%), DetectGPT (10%)
|
| 205 |
-
|
| 206 |
-
**What It Captures**:
|
| 207 |
-
- **Semantic Analysis**: Measures sentence-to-sentence coherence, n-gram repetition patterns, and contextual consistency. AI sometimes produces semantically coherent but contextually shallow connections.
|
| 208 |
-
- **DetectGPT**: Tests text stability under perturbation. AI-generated text sits at probability peaks in the model's output space, making it more sensitive to small changes, while human text is more robust to minor modifications.
|
| 209 |
-
|
| 210 |
-
**Domain Manifestations**:
|
| 211 |
-
- **Academic**: AI arguments show surface-level coherence but lack deep logical progression; humans build cumulative reasoning
|
| 212 |
-
- **Technical**: AI procedures are coherent but may miss implicit expert knowledge; humans include domain-specific nuances
|
| 213 |
-
- **Creative**: AI narratives maintain consistency but lack subtle foreshadowing; humans plant intentional inconsistencies for plot
|
| 214 |
-
- **Social Media**: AI maintains topic focus rigidly; humans naturally digress and return to main points
|
| 215 |
-
|
| 216 |
-
### Cross-Dimensional Detection Power
|
| 217 |
-
|
| 218 |
-
The ensemble's strength lies in capturing **multi-dimensional anomalies** simultaneously:
|
| 219 |
-
|
| 220 |
-
**Example 1: Sophisticated GPT-4 Academic Essay**
|
| 221 |
-
- Dimension 1 (Statistical): Low perplexity (22) + low entropy (3.2) โ **AI signal**
|
| 222 |
-
- Dimension 2 (Structural): High sentence uniformity (burstiness: 0.15) โ **AI signal**
|
| 223 |
-
- Dimension 3 (Semantic): High coherence but low perturbation stability โ **AI signal**
|
| 224 |
-
- **Result**: High-confidence AI detection (92% probability)
|
| 225 |
-
|
| 226 |
-
**Example 2: Human Technical Documentation**
|
| 227 |
-
- Dimension 1 (Statistical): Moderate perplexity (35) + moderate entropy (4.0) โ **Human signal**
|
| 228 |
-
- Dimension 2 (Structural): Varied structure with intentional consistency in procedures โ **Mixed signal**
|
| 229 |
-
- Dimension 3 (Semantic): Deep coherence + high perturbation stability โ **Human signal**
|
| 230 |
-
- **Result**: High-confidence human detection (88% human probability)
|
| 231 |
-
|
| 232 |
-
**Example 3: Human-Edited AI Content (Mixed)**
|
| 233 |
-
- Dimension 1 (Statistical): Low perplexity core with high-entropy edits โ **Mixed signal**
|
| 234 |
-
- Dimension 2 (Structural): Sections of uniformity interrupted by varied structures โ **Mixed signal**
|
| 235 |
-
- Dimension 3 (Semantic): Stable AI sections + unstable human additions โ **Mixed signal**
|
| 236 |
-
- **Result**: Mixed content detection with section-level attribution
|
| 237 |
|
| 238 |
---
|
| 239 |
|
| 240 |
-
##
|
| 241 |
|
| 242 |
-
|
| 243 |
|
| 244 |
-
|
| 245 |
-
|
| 246 |
-
|
| 247 |
-
|
| 248 |
-
|
| 249 |
-
|
| 250 |
-
-
|
| 251 |
-
- `P(w_i | context)` = conditional probability from GPT-2 XL
|
| 252 |
-
- `k` = context window size
|
| 253 |
|
| 254 |
-
|
| 255 |
-
- **AI text**: Lower perplexity (15-30) - more predictable to language models
|
| 256 |
-
- **Human text**: Higher perplexity (40-80) - more creative and unpredictable
|
| 257 |
|
| 258 |
-
**
|
| 259 |
-
|
| 260 |
-
# Academic texts naturally have lower perplexity
|
| 261 |
-
if domain == Domain.ACADEMIC:
|
| 262 |
-
perplexity_threshold *= 1.2
|
| 263 |
-
elif domain == Domain.SOCIAL_MEDIA:
|
| 264 |
-
perplexity_threshold *= 0.8
|
| 265 |
-
```
|
| 266 |
|
| 267 |
-
**Implementation
|
| 268 |
```python
|
| 269 |
-
def calculate_perplexity(text, model):
|
| 270 |
tokens = tokenize(text)
|
| 271 |
log_probs = []
|
| 272 |
-
|
| 273 |
for i in range(len(tokens)):
|
| 274 |
context = tokens[max(0, i-k):i]
|
| 275 |
prob = model.get_probability(tokens[i], context)
|
| 276 |
log_probs.append(math.log(prob))
|
| 277 |
-
|
| 278 |
-
return math.exp(-sum(log_probs) / len(tokens))
|
| 279 |
```
|
| 280 |
|
| 281 |
-
|
| 282 |
-
|
| 283 |
-
### 2. Entropy Metric (20% Weight)
|
| 284 |
-
|
| 285 |
-
**Shannon Entropy**:
|
| 286 |
```python
|
| 287 |
-
|
|
|
|
|
|
|
|
|
|
| 288 |
```
|
| 289 |
|
| 290 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 291 |
```python
|
|
|
|
| 292 |
def calculate_text_entropy(text):
|
| 293 |
tokens = text.split()
|
| 294 |
token_freq = Counter(tokens)
|
| 295 |
-
|
| 296 |
-
|
| 297 |
-
entropy = 0
|
| 298 |
-
for token, freq in token_freq.items():
|
| 299 |
-
probability = freq / total_tokens
|
| 300 |
-
entropy -= probability * math.log2(probability)
|
| 301 |
-
|
| 302 |
return entropy
|
| 303 |
```
|
| 304 |
|
| 305 |
-
|
| 306 |
-
- **AI text**: Lower entropy (2.8-3.8 bits/token) - repetitive patterns
|
| 307 |
-
- **Human text**: Higher entropy (4.2-5.5 bits/token) - diverse vocabulary
|
| 308 |
-
|
| 309 |
-
**Advanced Features**:
|
| 310 |
-
- N-gram entropy analysis (bigrams, trigrams)
|
| 311 |
-
- Contextual entropy using sliding windows
|
| 312 |
-
- Conditional entropy between adjacent sentences
|
| 313 |
-
|
| 314 |
-
---
|
| 315 |
-
|
| 316 |
-
### 3. Structural Metric (15% Weight)
|
| 317 |
-
|
| 318 |
-
**Burstiness Score**:
|
| 319 |
-
```python
|
| 320 |
-
Burstiness = (ฯ - ฮผ) / (ฯ + ฮผ)
|
| 321 |
-
```
|
| 322 |
-
|
| 323 |
-
**Where**:
|
| 324 |
-
- `ฯ` = standard deviation of sentence lengths
|
| 325 |
-
- `ฮผ` = mean sentence length
|
| 326 |
|
| 327 |
-
**
|
| 328 |
-
|
| 329 |
-
Uniformity = 1 - (std_dev / mean_length)
|
| 330 |
-
```
|
| 331 |
|
| 332 |
-
**
|
| 333 |
-
|
| 334 |
-
- Predictable paragraph structures
|
| 335 |
-
- Limited structural variation
|
| 336 |
-
- Uniform punctuation usage
|
| 337 |
|
| 338 |
-
**
|
| 339 |
```python
|
| 340 |
def calculate_burstiness(text):
|
| 341 |
sentences = split_sentences(text)
|
| 342 |
lengths = [len(s.split()) for s in sentences]
|
| 343 |
-
|
| 344 |
mean_len = np.mean(lengths)
|
| 345 |
std_len = np.std(lengths)
|
| 346 |
-
|
| 347 |
burstiness = (std_len - mean_len) / (std_len + mean_len)
|
| 348 |
-
uniformity = 1 - (std_len
|
| 349 |
-
|
| 350 |
-
return {
|
| 351 |
-
'burstiness': burstiness,
|
| 352 |
-
'uniformity': uniformity,
|
| 353 |
-
'mean_length': mean_len,
|
| 354 |
-
'std_length': std_len
|
| 355 |
-
}
|
| 356 |
-
```
|
| 357 |
-
|
| 358 |
-
---
|
| 359 |
-
|
| 360 |
-
### 4. Semantic Analysis Metric (15% Weight)
|
| 361 |
-
|
| 362 |
-
**Coherence Scoring**:
|
| 363 |
-
```python
|
| 364 |
-
Coherence = 1/n * ฮฃ cosine_similarity(sentence_i, sentence_{i+1})
|
| 365 |
```
|
| 366 |
|
| 367 |
-
|
| 368 |
-
```python
|
| 369 |
-
Repetition_Score = count_ngram_repeats(text, n=3) / total_ngrams
|
| 370 |
-
```
|
| 371 |
|
| 372 |
-
**
|
| 373 |
-
-
|
| 374 |
-
- Topic consistency across paragraphs
|
| 375 |
-
- Logical flow assessment
|
| 376 |
-
- Redundancy pattern detection
|
| 377 |
|
| 378 |
-
**
|
| 379 |
```python
|
| 380 |
-
def calculate_semantic_coherence(text,
|
| 381 |
sentences = split_sentences(text)
|
| 382 |
-
embeddings = [
|
| 383 |
-
|
| 384 |
-
|
| 385 |
-
for i in range(len(embeddings) - 1):
|
| 386 |
-
similarity = cosine_similarity(embeddings[i], embeddings[i+1])
|
| 387 |
-
coherence_scores.append(similarity)
|
| 388 |
-
|
| 389 |
-
return {
|
| 390 |
-
'mean_coherence': np.mean(coherence_scores),
|
| 391 |
-
'coherence_variance': np.var(coherence_scores),
|
| 392 |
-
'coherence_scores': coherence_scores
|
| 393 |
-
}
|
| 394 |
```
|
| 395 |
|
| 396 |
-
|
| 397 |
-
|
| 398 |
-
### 5. Linguistic Metric (15% Weight)
|
| 399 |
-
|
| 400 |
-
**POS Tag Diversity**:
|
| 401 |
-
```python
|
| 402 |
-
POS_Diversity = unique_POS_tags / total_tokens
|
| 403 |
-
```
|
| 404 |
-
|
| 405 |
-
**Syntactic Complexity**:
|
| 406 |
-
```python
|
| 407 |
-
Complexity = average_parse_tree_depth(sentences)
|
| 408 |
-
```
|
| 409 |
|
| 410 |
-
**
|
| 411 |
-
- Part-of-speech tag distribution
|
| 412 |
-
- Dependency parse tree depth and structure
|
| 413 |
-
- Syntactic variety across sentences
|
| 414 |
-
- Grammatical sophistication indicators
|
| 415 |
|
| 416 |
-
**Implementation**:
|
| 417 |
```python
|
| 418 |
def calculate_linguistic_features(text, nlp_model):
|
| 419 |
doc = nlp_model(text)
|
| 420 |
-
|
| 421 |
-
# POS diversity
|
| 422 |
pos_tags = [token.pos_ for token in doc]
|
| 423 |
-
pos_diversity = len(set(pos_tags))
|
| 424 |
-
|
| 425 |
-
|
| 426 |
-
depths = []
|
| 427 |
-
for sent in doc.sents:
|
| 428 |
-
depth = max(get_tree_depth(token) for token in sent)
|
| 429 |
-
depths.append(depth)
|
| 430 |
-
|
| 431 |
-
return {
|
| 432 |
-
'pos_diversity': pos_diversity,
|
| 433 |
-
'mean_tree_depth': np.mean(depths),
|
| 434 |
-
'complexity_variance': np.var(depths)
|
| 435 |
-
}
|
| 436 |
```
|
| 437 |
|
| 438 |
-
|
| 439 |
|
| 440 |
-
|
|
|
|
| 441 |
|
| 442 |
-
**Curvature Principle**:
|
| 443 |
-
```python
|
| 444 |
-
Stability_Score = 1/n * ฮฃ |log P(x) - log P(x_perturbed)|
|
| 445 |
-
```
|
| 446 |
-
|
| 447 |
-
Where `x_perturbed` are minor modifications of the original text.
|
| 448 |
-
|
| 449 |
-
**Perturbation Strategy**:
|
| 450 |
-
- Random word substitutions with synonyms
|
| 451 |
-
- Minor grammatical alterations
|
| 452 |
-
- Punctuation modifications
|
| 453 |
-
- Word order variations in non-critical positions
|
| 454 |
-
|
| 455 |
-
**Theory**:
|
| 456 |
-
AI-generated text sits at local maxima in the model's probability distribution. Small perturbations cause larger probability drops for AI text than for human text.
|
| 457 |
-
|
| 458 |
-
**Implementation**:
|
| 459 |
```python
|
| 460 |
def detect_gpt_score(text, model, num_perturbations=20):
|
| 461 |
-
|
| 462 |
-
|
| 463 |
-
perturbation_diffs = []
|
| 464 |
for _ in range(num_perturbations):
|
| 465 |
perturbed = generate_perturbation(text)
|
| 466 |
-
|
| 467 |
-
|
| 468 |
-
perturbation_diffs.append(diff)
|
| 469 |
-
|
| 470 |
-
stability_score = np.mean(perturbation_diffs)
|
| 471 |
-
return stability_score
|
| 472 |
```
|
| 473 |
|
| 474 |
---
|
| 475 |
|
| 476 |
## ๐๏ธ Ensemble Methodology
|
| 477 |
|
| 478 |
-
### Confidence
|
| 479 |
-
|
| 480 |
-
|
|
|
|
|
|
|
| 481 |
|
| 482 |
```python
|
| 483 |
def ensemble_aggregation(metric_results, domain):
|
| 484 |
-
|
| 485 |
-
|
| 486 |
-
|
| 487 |
-
|
| 488 |
-
confidence_weights = {}
|
| 489 |
-
for metric, result in metric_results.items():
|
| 490 |
-
confidence_factor = sigmoid_confidence_adjustment(result.confidence)
|
| 491 |
-
confidence_weights[metric] = base_weights[metric] * confidence_factor
|
| 492 |
-
|
| 493 |
-
# Normalize and aggregate
|
| 494 |
-
total_weight = sum(confidence_weights.values())
|
| 495 |
-
final_weights = {k: v/total_weight for k, v in confidence_weights.items()}
|
| 496 |
-
|
| 497 |
return weighted_aggregate(metric_results, final_weights)
|
| 498 |
```
|
| 499 |
|
| 500 |
### Uncertainty Quantification
|
| 501 |
-
|
| 502 |
```python
|
| 503 |
def calculate_uncertainty(metric_results, ensemble_result):
|
| 504 |
-
|
| 505 |
-
|
| 506 |
-
|
| 507 |
-
|
| 508 |
-
confidence_uncertainty = 1 - np.mean([r.confidence for r in metric_results.values()])
|
| 509 |
-
|
| 510 |
-
# Decision uncertainty (distance from 0.5)
|
| 511 |
-
decision_uncertainty = 1 - 2 * abs(ensemble_result.ai_probability - 0.5)
|
| 512 |
-
|
| 513 |
-
return (variance_uncertainty * 0.4 +
|
| 514 |
-
confidence_uncertainty * 0.3 +
|
| 515 |
-
decision_uncertainty * 0.3)
|
| 516 |
```
|
| 517 |
|
| 518 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 519 |
|
| 520 |
```python
|
| 521 |
DOMAIN_WEIGHTS = {
|
| 522 |
-
|
| 523 |
-
|
| 524 |
-
|
| 525 |
-
|
| 526 |
-
'linguistic': 0.20, # Increased for academic rigor
|
| 527 |
-
'semantic': 0.15,
|
| 528 |
-
'detect_gpt': 0.10
|
| 529 |
-
},
|
| 530 |
-
Domain.TECHNICAL: {
|
| 531 |
-
'perplexity': 0.20,
|
| 532 |
-
'entropy': 0.18,
|
| 533 |
-
'structural': 0.12,
|
| 534 |
-
'linguistic': 0.18,
|
| 535 |
-
'semantic': 0.22, # Increased for logical consistency
|
| 536 |
-
'detect_gpt': 0.10
|
| 537 |
-
},
|
| 538 |
-
Domain.CREATIVE: {
|
| 539 |
-
'perplexity': 0.25,
|
| 540 |
-
'entropy': 0.25, # Increased for vocabulary diversity
|
| 541 |
-
'structural': 0.20, # Increased for burstiness
|
| 542 |
-
'linguistic': 0.12,
|
| 543 |
-
'semantic': 0.10,
|
| 544 |
-
'detect_gpt': 0.08
|
| 545 |
-
},
|
| 546 |
-
Domain.SOCIAL_MEDIA: {
|
| 547 |
-
'perplexity': 0.30, # Highest weight for statistical patterns
|
| 548 |
-
'entropy': 0.22,
|
| 549 |
-
'structural': 0.15,
|
| 550 |
-
'linguistic': 0.10, # Relaxed for informal writing
|
| 551 |
-
'semantic': 0.13,
|
| 552 |
-
'detect_gpt': 0.10
|
| 553 |
-
}
|
| 554 |
}
|
| 555 |
```
|
| 556 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 557 |
---
|
| 558 |
|
| 559 |
-
## ๐ Project Structure
|
| 560 |
|
| 561 |
```text
|
| 562 |
text_auth/
|
| 563 |
โโโ config/
|
| 564 |
-
โ โโโ
|
| 565 |
-
โ โโโ
|
| 566 |
-
โ
|
| 567 |
-
โ โโโ threshold_config.py # Domain-aware thresholds
|
| 568 |
-
โ
|
| 569 |
โโโ data/
|
| 570 |
-
โ โโโ reports/
|
| 571 |
-
โ โโโ uploads/
|
| 572 |
-
โ
|
| 573 |
โโโ detector/
|
| 574 |
-
โ โโโ
|
| 575 |
-
โ โโโ
|
| 576 |
-
โ โโโ
|
| 577 |
-
โ
|
| 578 |
-
โ โโโ orchestrator.py # Main detection pipeline
|
| 579 |
-
โ
|
| 580 |
-
โโโ logs/ # Application logs
|
| 581 |
-
โ
|
| 582 |
โโโ metrics/
|
| 583 |
-
โ โโโ
|
| 584 |
-
โ โโโ
|
| 585 |
-
โ โโโ
|
| 586 |
-
โ โโโ
|
| 587 |
-
โ โโโ
|
| 588 |
-
โ โโโ
|
| 589 |
-
โ
|
| 590 |
-
โ โโโ structural.py # Structural patterns
|
| 591 |
-
โ
|
| 592 |
โโโ models/
|
| 593 |
-
โ โโโ
|
| 594 |
-
โ
|
| 595 |
-
โ โโโ model_registry.py # Model version registry
|
| 596 |
-
โ
|
| 597 |
โโโ processors/
|
| 598 |
-
โ โโโ
|
| 599 |
-
โ โโโ
|
| 600 |
-
โ โโโ
|
| 601 |
-
โ
|
| 602 |
-
โ โโโ text_processor.py # Text preprocessing
|
| 603 |
-
โ
|
| 604 |
โโโ reporter/
|
| 605 |
-
โ โโโ
|
| 606 |
-
โ
|
| 607 |
-
โ โโโ report_generator.py # JSON/PDF report generation
|
| 608 |
-
โ
|
| 609 |
โโโ ui/
|
| 610 |
-
โ
|
| 611 |
-
โ โโโ static/
|
| 612 |
-
โ โโโ index.html # Web interface
|
| 613 |
-
โ
|
| 614 |
โโโ utils/
|
| 615 |
-
โ
|
| 616 |
-
|
| 617 |
-
|
| 618 |
-
โโโ
|
| 619 |
-
|
| 620 |
-
โโโ requirements.txt # Python dependencies
|
| 621 |
-
โโโ run.sh # Application launcher
|
| 622 |
-
โโโ text_auth_app.py # FastAPI application entry
|
| 623 |
```
|
| 624 |
|
| 625 |
---
|
| 626 |
|
| 627 |
-
## ๐ API Endpoints
|
| 628 |
-
|
| 629 |
-
### Core Analysis Endpoints
|
| 630 |
|
| 631 |
-
|
| 632 |
-
|
| 633 |
|
| 634 |
-
|
| 635 |
-
|
| 636 |
-
**Request**:
|
| 637 |
```json
|
| 638 |
{
|
| 639 |
-
"text":
|
| 640 |
-
"domain":
|
| 641 |
"enable_attribution": true,
|
| 642 |
"enable_highlighting": true,
|
| 643 |
"use_sentence_level": true,
|
|
@@ -645,666 +396,210 @@ Analyze pasted text for AI generation.
|
|
| 645 |
}
|
| 646 |
```
|
| 647 |
|
| 648 |
-
**Response
|
| 649 |
```json
|
| 650 |
{
|
| 651 |
-
"status":
|
| 652 |
-
"analysis_id":
|
| 653 |
-
"detection_result":
|
| 654 |
-
"ensemble_result":
|
| 655 |
-
|
| 656 |
-
"ai_probability": 0.8943,
|
| 657 |
-
"human_probability": 0.0957,
|
| 658 |
-
"mixed_probability": 0.0100,
|
| 659 |
-
"overall_confidence": 0.8721,
|
| 660 |
-
"uncertainty_score": 0.2345,
|
| 661 |
-
"consensus_level": 0.8123
|
| 662 |
-
},
|
| 663 |
-
"metric_results": {
|
| 664 |
-
"structural": {
|
| 665 |
-
"ai_probability": 0.85,
|
| 666 |
-
"confidence": 0.78,
|
| 667 |
-
"burstiness": 0.15,
|
| 668 |
-
"uniformity": 0.82
|
| 669 |
-
},
|
| 670 |
-
"perplexity": {
|
| 671 |
-
"ai_probability": 0.92,
|
| 672 |
-
"confidence": 0.89,
|
| 673 |
-
"score": 22.5
|
| 674 |
-
},
|
| 675 |
-
"entropy": {
|
| 676 |
-
"ai_probability": 0.88,
|
| 677 |
-
"confidence": 0.85,
|
| 678 |
-
"score": 3.2
|
| 679 |
-
},
|
| 680 |
-
"linguistic": {
|
| 681 |
-
"ai_probability": 0.87,
|
| 682 |
-
"confidence": 0.79,
|
| 683 |
-
"pos_diversity": 0.65
|
| 684 |
-
},
|
| 685 |
-
"semantic": {
|
| 686 |
-
"ai_probability": 0.89,
|
| 687 |
-
"confidence": 0.81,
|
| 688 |
-
"coherence": 0.78
|
| 689 |
-
},
|
| 690 |
-
"detect_gpt": {
|
| 691 |
-
"ai_probability": 0.84,
|
| 692 |
-
"confidence": 0.76,
|
| 693 |
-
"stability_score": 0.25
|
| 694 |
-
}
|
| 695 |
-
}
|
| 696 |
-
},
|
| 697 |
-
"attribution": {
|
| 698 |
-
"predicted_model": "gpt-4",
|
| 699 |
-
"confidence": 0.7632,
|
| 700 |
-
"model_probabilities": {
|
| 701 |
-
"gpt-4": 0.76,
|
| 702 |
-
"claude-3-opus": 0.21,
|
| 703 |
-
"gemini-pro": 0.03
|
| 704 |
-
}
|
| 705 |
},
|
| 706 |
-
"
|
| 707 |
-
"
|
| 708 |
-
|
| 709 |
-
"key_indicators": [
|
| 710 |
-
"Low perplexity (22.5) suggests high predictability to language models",
|
| 711 |
-
"Uniform sentence structure (burstiness: 0.15) indicates AI generation",
|
| 712 |
-
"Low entropy (3.2 bits/token) reveals repetitive token patterns"
|
| 713 |
-
],
|
| 714 |
-
"confidence_explanation": "High confidence due to strong metric agreement (consensus: 81.2%)"
|
| 715 |
-
}
|
| 716 |
-
}
|
| 717 |
-
```
|
| 718 |
-
|
| 719 |
-
---
|
| 720 |
-
|
| 721 |
-
#### 2. File Analysis
|
| 722 |
-
**POST** `/api/analyze/file`
|
| 723 |
-
|
| 724 |
-
Analyze uploaded documents (PDF, DOCX, TXT, DOC, MD).
|
| 725 |
-
|
| 726 |
-
**Features**:
|
| 727 |
-
- Automatic text extraction from multiple formats
|
| 728 |
-
- Domain classification
|
| 729 |
-
- File size validation (10MB limit)
|
| 730 |
-
- Multi-page PDF support
|
| 731 |
-
|
| 732 |
-
**Request** (multipart/form-data):
|
| 733 |
-
```
|
| 734 |
-
file: <binary file data>
|
| 735 |
-
domain: "academic" (optional)
|
| 736 |
-
enable_attribution: true (optional)
|
| 737 |
-
```
|
| 738 |
-
|
| 739 |
-
**Response**: Same structure as text analysis endpoint
|
| 740 |
-
|
| 741 |
-
---
|
| 742 |
-
|
| 743 |
-
#### 3. Report Generation
|
| 744 |
-
**POST** `/api/report/generate`
|
| 745 |
-
|
| 746 |
-
Generate downloadable reports in JSON/PDF formats.
|
| 747 |
-
|
| 748 |
-
**Request**:
|
| 749 |
-
```json
|
| 750 |
-
{
|
| 751 |
-
"analysis_id": "analysis_1701234567890",
|
| 752 |
-
"format": "json|pdf",
|
| 753 |
-
"include_highlights": true,
|
| 754 |
-
"include_metrics_breakdown": true
|
| 755 |
-
}
|
| 756 |
-
```
|
| 757 |
-
|
| 758 |
-
**Supported Formats**:
|
| 759 |
-
- `json`: Complete structured data
|
| 760 |
-
- `pdf`: Printable professional reports
|
| 761 |
-
|
| 762 |
-
---
|
| 763 |
-
|
| 764 |
-
### Utility Endpoints
|
| 765 |
-
|
| 766 |
-
#### 4. Health Check
|
| 767 |
-
**GET** `/health`
|
| 768 |
-
|
| 769 |
-
```json
|
| 770 |
-
{
|
| 771 |
-
"status": "healthy",
|
| 772 |
-
"version": "2.0.0",
|
| 773 |
-
"uptime": 12345.67,
|
| 774 |
-
"models_loaded": {
|
| 775 |
-
"orchestrator": true,
|
| 776 |
-
"attributor": true,
|
| 777 |
-
"highlighter": true
|
| 778 |
-
}
|
| 779 |
}
|
| 780 |
```
|
| 781 |
|
| 782 |
-
|
| 783 |
-
|
| 784 |
-
#### 5. Domain Information
|
| 785 |
-
**GET** `/api/domains`
|
| 786 |
|
| 787 |
-
|
|
|
|
| 788 |
|
| 789 |
-
|
| 790 |
-
|
| 791 |
-
|
| 792 |
-
|
| 793 |
-
"id": "academic",
|
| 794 |
-
"name": "Academic Writing",
|
| 795 |
-
"description": "Essays, research papers, scholarly articles",
|
| 796 |
-
"ai_threshold": 0.88,
|
| 797 |
-
"human_threshold": 0.65
|
| 798 |
-
},
|
| 799 |
-
{
|
| 800 |
-
"id": "technical_doc",
|
| 801 |
-
"name": "Technical Documentation",
|
| 802 |
-
"description": "Technical manuals, medical papers, research documentation",
|
| 803 |
-
"ai_threshold": 0.92,
|
| 804 |
-
"human_threshold": 0.72
|
| 805 |
-
},
|
| 806 |
-
{
|
| 807 |
-
"id": "creative",
|
| 808 |
-
"name": "Creative Writing",
|
| 809 |
-
"description": "Stories, narratives, creative content",
|
| 810 |
-
"ai_threshold": 0.78,
|
| 811 |
-
"human_threshold": 0.55
|
| 812 |
-
},
|
| 813 |
-
{
|
| 814 |
-
"id": "social_media",
|
| 815 |
-
"name": "Social Media & Casual",
|
| 816 |
-
"description": "Blogs, social posts, informal writing",
|
| 817 |
-
"ai_threshold": 0.80,
|
| 818 |
-
"human_threshold": 0.50
|
| 819 |
-
}
|
| 820 |
-
]
|
| 821 |
-
}
|
| 822 |
-
```
|
| 823 |
|
| 824 |
---
|
| 825 |
|
| 826 |
-
|
| 827 |
-
**GET** `/api/models`
|
| 828 |
-
|
| 829 |
-
Returns detectable AI models for attribution.
|
| 830 |
-
|
| 831 |
-
```json
|
| 832 |
-
{
|
| 833 |
-
"models": [
|
| 834 |
-
{"id": "gpt-4", "name": "GPT-4", "provider": "OpenAI"},
|
| 835 |
-
{"id": "gpt-3.5-turbo", "name": "GPT-3.5 Turbo", "provider": "OpenAI"},
|
| 836 |
-
{"id": "claude-3-opus", "name": "Claude 3 Opus", "provider": "Anthropic"},
|
| 837 |
-
{"id": "claude-3-sonnet", "name": "Claude 3 Sonnet", "provider": "Anthropic"},
|
| 838 |
-
{"id": "gemini-pro", "name": "Gemini Pro", "provider": "Google"},
|
| 839 |
-
{"id": "llama-2-70b", "name": "LLaMA 2 70B", "provider": "Meta"},
|
| 840 |
-
{"id": "mixtral-8x7b", "name": "Mixtral 8x7B", "provider": "Mistral AI"}
|
| 841 |
-
]
|
| 842 |
-
}
|
| 843 |
-
```
|
| 844 |
-
|
| 845 |
-
---
|
| 846 |
-
|
| 847 |
-
## ๐ฏ Domain-Aware Detection
|
| 848 |
-
|
| 849 |
-
### Domain-Specific Thresholds
|
| 850 |
-
|
| 851 |
-
| Domain | AI Threshold | Human Threshold | Key Adjustments |
|
| 852 |
-
|--------|--------------|-----------------|-----------------|
|
| 853 |
-
| **Academic** | > 0.88 | < 0.65 | Higher linguistic weight, reduced perplexity sensitivity |
|
| 854 |
-
| **Technical/Medical** | > 0.92 | < 0.72 | Much higher thresholds, focus on semantic patterns |
|
| 855 |
-
| **Creative Writing** | > 0.78 | < 0.55 | Balanced weights, emphasis on burstiness detection |
|
| 856 |
-
| **Social Media** | > 0.80 | < 0.50 | Higher statistical weight, relaxed linguistic requirements |
|
| 857 |
-
|
| 858 |
-
### Performance by Domain
|
| 859 |
-
|
| 860 |
-
| Domain | Precision | Recall | F1-Score | False Positive Rate |
|
| 861 |
-
|--------|-----------|--------|----------|---------------------|
|
| 862 |
-
| **Academic Papers** | 96.2% | 93.8% | 95.0% | 1.8% |
|
| 863 |
-
| **Student Essays** | 94.5% | 92.1% | 93.3% | 2.5% |
|
| 864 |
-
| **Technical Documentation** | 92.8% | 90.5% | 91.6% | 3.1% |
|
| 865 |
-
| **Mixed Human-AI Content** | 88.7% | 85.3% | 87.0% | 4.2% |
|
| 866 |
-
|
| 867 |
-
### Domain Calibration Strategy
|
| 868 |
-
|
| 869 |
-
**Academic Domain**
|
| 870 |
-
- **Use Cases**: Essays, research papers, assignments
|
| 871 |
-
- **Adjustments**:
|
| 872 |
-
- Increased linguistic metric weight (20% vs 15% baseline)
|
| 873 |
-
- Higher perplexity threshold multiplier (1.2x)
|
| 874 |
-
- Stricter structural uniformity detection
|
| 875 |
-
- **Rationale**: Academic writing naturally has lower perplexity due to formal language, requiring calibrated thresholds
|
| 876 |
-
|
| 877 |
-
**Technical/Medical Domain**
|
| 878 |
-
- **Use Cases**: Research papers, documentation, technical reports
|
| 879 |
-
- **Adjustments**:
|
| 880 |
-
- Highest AI threshold (0.92) to minimize false positives
|
| 881 |
-
- Increased semantic analysis weight (22% vs 15%)
|
| 882 |
-
- Reduced linguistic weight for domain-specific terminology
|
| 883 |
-
- **Rationale**: Technical content has specialized vocabulary that may appear "unusual" to general language models
|
| 884 |
-
|
| 885 |
-
**Creative Writing Domain**
|
| 886 |
-
- **Use Cases**: Stories, creative essays, narratives, personal writing
|
| 887 |
-
- **Adjustments**:
|
| 888 |
-
- Highest entropy weight (25% vs 20%) for vocabulary diversity
|
| 889 |
-
- Increased structural weight (20% vs 15%) for burstiness detection
|
| 890 |
-
- Lower AI threshold (0.78) to catch creative AI content
|
| 891 |
-
- **Rationale**: Human creativity exhibits high burstiness and vocabulary diversity
|
| 892 |
-
|
| 893 |
-
**Social Media Domain**
|
| 894 |
-
- **Use Cases**: Blogs, social posts, informal writing, casual content
|
| 895 |
-
- **Adjustments**:
|
| 896 |
-
- Highest perplexity weight (30% vs 25%) for statistical patterns
|
| 897 |
-
- Relaxed linguistic requirements (10% vs 15%)
|
| 898 |
-
- Lower perplexity threshold multiplier (0.8x)
|
| 899 |
-
- **Rationale**: Informal writing naturally has grammatical flexibility and slang usage
|
| 900 |
-
|
| 901 |
-
---
|
| 902 |
-
|
| 903 |
-
## โก Performance Characteristics
|
| 904 |
-
|
| 905 |
-
### Processing Times
|
| 906 |
-
|
| 907 |
-
| Text Length | Processing Time | CPU Usage | Memory Usage |
|
| 908 |
-
|-------------|----------------|-----------|--------------|
|
| 909 |
-
| **Short** (100-500 words) | 1.2 seconds | 0.8 vCPU | 512 MB |
|
| 910 |
-
| **Medium** (500-2000 words) | 3.5 seconds | 1.2 vCPU | 1 GB |
|
| 911 |
-
| **Long** (2000+ words) | 7.8 seconds | 2.0 vCPU | 2 GB |
|
| 912 |
-
|
| 913 |
-
### Computational Optimization
|
| 914 |
-
|
| 915 |
-
**Parallel Metric Computation**
|
| 916 |
-
- Independent metrics run concurrently using thread pools
|
| 917 |
-
- 3-4x speedup compared to sequential execution
|
| 918 |
-
- Efficient resource utilization with async/await patterns
|
| 919 |
-
|
| 920 |
-
**Conditional Execution**
|
| 921 |
-
- Expensive metrics (DetectGPT) can be skipped for faster analysis
|
| 922 |
-
- Adaptive threshold early-exit when high confidence is achieved
|
| 923 |
-
- Progressive analysis with real-time confidence updates
|
| 924 |
-
|
| 925 |
-
**Model Caching**
|
| 926 |
-
- Pre-trained models loaded once at startup
|
| 927 |
-
- Shared model instances across requests
|
| 928 |
-
- Memory-efficient model storage with quantization
|
| 929 |
-
|
| 930 |
-
**Memory Management**
|
| 931 |
-
- Efficient text processing with streaming where possible
|
| 932 |
-
- Automatic garbage collection of analysis artifacts
|
| 933 |
-
- Bounded memory usage with configurable limits
|
| 934 |
-
|
| 935 |
-
### Cost Analysis
|
| 936 |
-
|
| 937 |
-
| Text Length | Processing Time | Cost per Analysis | Monthly Cost (1000 analyses) |
|
| 938 |
-
|-------------|----------------|-------------------|------------------------------|
|
| 939 |
-
| Short (100-500 words) | 1.2 sec | $0.0008 | $0.80 |
|
| 940 |
-
| Medium (500-2000 words) | 3.5 sec | $0.0025 | $2.50 |
|
| 941 |
-
| Long (2000+ words) | 7.8 sec | $0.0058 | $5.80 |
|
| 942 |
-
| Batch (100 documents) | 45 sec | $0.42 | N/A |
|
| 943 |
-
|
| 944 |
-
---
|
| 945 |
-
|
| 946 |
-
## ๐ง Installation & Setup
|
| 947 |
|
| 948 |
### Prerequisites
|
|
|
|
|
|
|
|
|
|
|
|
|
| 949 |
|
| 950 |
-
|
| 951 |
-
- **RAM**: 4GB minimum, 8GB recommended
|
| 952 |
-
- **Disk Space**: 2GB for models and dependencies
|
| 953 |
-
- **OS**: Linux, macOS, or Windows with WSL
|
| 954 |
-
|
| 955 |
-
### Quick Start
|
| 956 |
-
|
| 957 |
```bash
|
| 958 |
-
|
| 959 |
-
|
| 960 |
-
cd ai-text-detector
|
| 961 |
-
|
| 962 |
-
# Create virtual environment
|
| 963 |
python -m venv venv
|
| 964 |
-
source venv/bin/activate
|
| 965 |
-
|
| 966 |
-
# Install dependencies
|
| 967 |
pip install -r requirements.txt
|
| 968 |
-
|
| 969 |
-
|
| 970 |
-
./run.sh
|
| 971 |
-
# Or: python text_auth_app.py
|
| 972 |
```
|
| 973 |
|
| 974 |
-
|
| 975 |
-
-
|
| 976 |
-
-
|
| 977 |
-
- **Interactive API**: http://localhost:8000/api/redoc
|
| 978 |
|
| 979 |
-
|
| 980 |
|
| 981 |
-
|
| 982 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 983 |
```python
|
| 984 |
-
|
| 985 |
-
|
| 986 |
-
VERSION = "2.0.0"
|
| 987 |
-
DEBUG = False
|
| 988 |
-
|
| 989 |
-
# Server Configuration
|
| 990 |
-
HOST = "0.0.0.0"
|
| 991 |
-
PORT = 8000
|
| 992 |
-
WORKERS = 4
|
| 993 |
-
|
| 994 |
-
# Detection Settings
|
| 995 |
-
DEFAULT_DOMAIN = "academic"
|
| 996 |
-
ENABLE_ATTRIBUTION = True
|
| 997 |
-
ENABLE_HIGHLIGHTING = True
|
| 998 |
-
MAX_TEXT_LENGTH = 50000
|
| 999 |
-
|
| 1000 |
-
# File Upload Settings
|
| 1001 |
-
MAX_FILE_SIZE = 10 * 1024 * 1024 # 10MB
|
| 1002 |
-
ALLOWED_EXTENSIONS = [".pdf", ".docx", ".txt", ".doc", ".md"]
|
| 1003 |
-
|
| 1004 |
-
# Performance Settings
|
| 1005 |
-
METRIC_TIMEOUT = 30 # seconds
|
| 1006 |
-
ENABLE_PARALLEL_METRICS = True
|
| 1007 |
-
CACHE_MODELS = True
|
| 1008 |
```
|
|
|
|
| 1009 |
---
|
| 1010 |
|
| 1011 |
-
##
|
| 1012 |
|
| 1013 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1014 |
|
| 1015 |
-
|
| 1016 |
|
| 1017 |
-
|
| 1018 |
-
|---------------|---------|----------|-----------|--------|
|
| 1019 |
-
| **GPT-4 Generated Text** | 5,000 | 95.8% | 96.2% | 95.3% |
|
| 1020 |
-
| **Claude-3 Generated** | 3,000 | 94.2% | 94.8% | 93.5% |
|
| 1021 |
-
| **Gemini Pro Generated** | 2,500 | 93.6% | 94.1% | 93.0% |
|
| 1022 |
-
| **LLaMA 2 Generated** | 2,000 | 92.8% | 93.3% | 92.2% |
|
| 1023 |
-
| **Human Academic Writing** | 10,000 | 96.1% | 95.7% | 96.4% |
|
| 1024 |
-
| **Human Creative Writing** | 5,000 | 94.8% | 94.3% | 95.2% |
|
| 1025 |
-
| **Mixed Content** | 2,000 | 88.7% | 89.2% | 88.1% |
|
| 1026 |
-
| **Overall Weighted** | 29,500 | **94.3%** | **94.6%** | **94.1%** |
|
| 1027 |
|
| 1028 |
-
###
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1029 |
|
|
|
|
| 1030 |
```
|
| 1031 |
-
|
| 1032 |
-
|
| 1033 |
-
Actual
|
| 1034 |
-
|
| 1035 |
-
Mixed 420 580 1,000 (2,000 samples)
|
| 1036 |
```
|
| 1037 |
|
| 1038 |
-
|
| 1039 |
-
-
|
| 1040 |
-
-
|
| 1041 |
-
-
|
| 1042 |
-
-
|
| 1043 |
-
|
| 1044 |
-
### Cross-Domain Validation
|
| 1045 |
-
|
| 1046 |
-
| Domain | Dataset Size | Accuracy | Notes |
|
| 1047 |
-
|--------|--------------|----------|-------|
|
| 1048 |
-
| Academic Papers | 5,000 | 96.2% | High precision on scholarly content |
|
| 1049 |
-
| Student Essays | 10,000 | 94.5% | Robust across varying skill levels |
|
| 1050 |
-
| Technical Docs | 3,000 | 92.8% | Specialized terminology handled well |
|
| 1051 |
-
| Creative Writing | 5,000 | 93.7% | Excellent burstiness detection |
|
| 1052 |
-
| Social Media | 4,000 | 91.5% | Adapted to informal language |
|
| 1053 |
-
|
| 1054 |
-
### Continuous Improvement
|
| 1055 |
-
|
| 1056 |
-
**Model Update Pipeline**
|
| 1057 |
-
- Regular retraining on new AI model releases
|
| 1058 |
-
- Continuous validation against emerging patterns
|
| 1059 |
-
- Adaptive threshold calibration based on false positive feedback
|
| 1060 |
-
- A/B testing of metric weight adjustments
|
| 1061 |
-
|
| 1062 |
-
**Feedback Loop**
|
| 1063 |
-
- User-reported false positives integrated into training
|
| 1064 |
-
- Monthly accuracy audits
|
| 1065 |
-
- Quarterly model version updates
|
| 1066 |
-
- Real-time performance monitoring
|
| 1067 |
-
|
| 1068 |
-
**Research Validation**
|
| 1069 |
-
- Peer-reviewed methodology
|
| 1070 |
-
- Open benchmark participation
|
| 1071 |
-
- Academic collaboration program
|
| 1072 |
-
- Published accuracy reports
|
| 1073 |
-
|
| 1074 |
-
---
|
| 1075 |
-
|
| 1076 |
-
## ๐จ Frontend Features
|
| 1077 |
-
|
| 1078 |
-
### Real-Time Analysis Interface
|
| 1079 |
-
|
| 1080 |
-
**Dual-Panel Design**
|
| 1081 |
-
- **Left Panel**: Text input with file upload support
|
| 1082 |
-
- **Right Panel**: Live analysis results with progressive updates
|
| 1083 |
-
- Responsive layout adapting to screen size
|
| 1084 |
-
- Dark/light mode support
|
| 1085 |
-
|
| 1086 |
-
**Interactive Highlighting**
|
| 1087 |
-
- Sentence-level AI probability visualization
|
| 1088 |
-
- Color-coded confidence indicators:
|
| 1089 |
-
- ๐ด Red (90-100%): Very high AI probability
|
| 1090 |
-
- ๐ Orange (70-90%): High AI probability
|
| 1091 |
-
- ๐ก Yellow (50-70%): Moderate AI probability
|
| 1092 |
-
- ๐ข Green (0-50%): Low AI probability (likely human)
|
| 1093 |
-
- Hover tooltips with detailed metric breakdowns
|
| 1094 |
-
- Click-to-expand for sentence-specific analysis
|
| 1095 |
-
|
| 1096 |
-
**Comprehensive Reports**
|
| 1097 |
-
- **Summary View**: High-level verdict and confidence
|
| 1098 |
-
- **Highlights View**: Sentence-level color-coded analysis
|
| 1099 |
-
- **Metrics View**: Detailed breakdown of all 6 metrics
|
| 1100 |
-
- **Attribution View**: AI model identification with probabilities
|
| 1101 |
-
|
| 1102 |
-
**Download Options**
|
| 1103 |
-
- JSON format for programmatic access
|
| 1104 |
-
- PDF format for professional reports
|
| 1105 |
-
|
| 1106 |
-
### User Experience
|
| 1107 |
-
|
| 1108 |
-
**Responsive Design**
|
| 1109 |
-
- Works seamlessly on desktop and mobile devices
|
| 1110 |
-
- Touch-optimized controls for tablets
|
| 1111 |
-
- Adaptive layout for varying screen sizes
|
| 1112 |
-
- Progressive Web App (PWA) capabilities
|
| 1113 |
-
|
| 1114 |
-
**Progress Indicators**
|
| 1115 |
-
- Real-time analysis status updates
|
| 1116 |
-
- Animated loading states
|
| 1117 |
-
- Estimated completion time
|
| 1118 |
-
- Metric-by-metric progress visualization
|
| 1119 |
-
|
| 1120 |
-
**Error Handling**
|
| 1121 |
-
- User-friendly error messages
|
| 1122 |
-
- Helpful troubleshooting suggestions
|
| 1123 |
-
- Graceful degradation on metric failures
|
| 1124 |
-
- Retry mechanisms for transient errors
|
| 1125 |
|
| 1126 |
---
|
| 1127 |
|
| 1128 |
## ๐ผ Business Model & Market Analysis
|
| 1129 |
|
| 1130 |
-
|
|
|
|
| 1131 |
|
| 1132 |
-
**
|
| 1133 |
-
-
|
| 1134 |
-
- Enterprise Hiring: $5B (30% YoY growth)
|
| 1135 |
-
- Content Publishing: $3B (60% YoY growth)
|
| 1136 |
|
| 1137 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1138 |
|
| 1139 |
-
|
| 1140 |
-
- 60% of students regularly use AI tools for assignments
|
| 1141 |
-
- 89% of teachers report encountering AI-written submissions
|
| 1142 |
-
- Traditional assessment methods becoming obsolete
|
| 1143 |
-
- Urgent need for reliable detection tools
|
| 1144 |
-
|
| 1145 |
-
**Hiring Quality Degradation**
|
| 1146 |
-
- AI-generated applications masking true candidate qualifications
|
| 1147 |
-
- Remote hiring amplifying verification challenges
|
| 1148 |
-
- Resume screening becoming unreliable
|
| 1149 |
-
- Interview process contaminated by AI-prepared responses
|
| 1150 |
|
| 1151 |
-
|
| 1152 |
-
- AI-generated articles flooding publishing platforms
|
| 1153 |
-
- SEO manipulation through AI content farms
|
| 1154 |
-
- Trust erosion in digital content ecosystems
|
| 1155 |
-
- Advertising revenue impacted by low-quality AI content
|
| 1156 |
|
| 1157 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1158 |
|
| 1159 |
-
|
| 1160 |
-
|
| 1161 |
-
|
| 1162 |
-
|
| 1163 |
-
|
| 1164 |
-
| **Our Solution** | **~9%+** | Domain adaptation, explainability, attribution | $15/month | **Superior accuracy, lower false positives** |
|
| 1165 |
|
| 1166 |
---
|
| 1167 |
|
| 1168 |
-
##
|
| 1169 |
-
|
| 1170 |
-
### Planned Features (Q1-Q2 2026)
|
| 1171 |
-
|
| 1172 |
-
**Multi-Language Support**
|
| 1173 |
-
- Detection for Spanish, French, German, Chinese
|
| 1174 |
-
- Language-specific metric calibration
|
| 1175 |
-
- Cross-lingual attribution
|
| 1176 |
-
- Multilingual training datasets
|
| 1177 |
-
|
| 1178 |
-
**Real-Time API**
|
| 1179 |
-
- WebSocket support for streaming analysis
|
| 1180 |
-
- Progressive result updates
|
| 1181 |
-
- Live collaboration features
|
| 1182 |
-
- Real-time dashboard for educators
|
| 1183 |
-
|
| 1184 |
-
**Advanced Attribution**
|
| 1185 |
-
- Fine-grained model version detection (GPT-4-turbo vs GPT-4)
|
| 1186 |
-
- Training data epoch identification
|
| 1187 |
-
- Generation parameter estimation (temperature, top-p)
|
| 1188 |
-
- Prompt engineering pattern detection
|
| 1189 |
-
|
| 1190 |
-
**Custom Thresholds**
|
| 1191 |
-
- User-configurable sensitivity settings
|
| 1192 |
-
- Institution-specific calibration
|
| 1193 |
-
- Subject-matter specialized models
|
| 1194 |
-
- Adjustable false positive tolerance
|
| 1195 |
-
|
| 1196 |
-
### Research Directions
|
| 1197 |
-
|
| 1198 |
-
**Adversarial Robustness**
|
| 1199 |
-
- Defense against detection evasion techniques
|
| 1200 |
-
- Paraphrasing attack detection
|
| 1201 |
-
- Synonym substitution resilience
|
| 1202 |
-
- Steganographic AI content identification
|
| 1203 |
-
|
| 1204 |
-
**Cross-Model Generalization**
|
| 1205 |
-
- Improved detection of novel AI models
|
| 1206 |
-
- Zero-shot detection capabilities
|
| 1207 |
-
- Transfer learning across model families
|
| 1208 |
-
- Emerging model early warning system
|
| 1209 |
-
|
| 1210 |
-
**Explainable AI Enhancement**
|
| 1211 |
-
- Natural language reasoning generation
|
| 1212 |
-
- Visual explanation dashboards
|
| 1213 |
-
- Counterfactual examples
|
| 1214 |
-
- Feature importance visualization
|
| 1215 |
-
|
| 1216 |
-
**Hybrid Content Analysis**
|
| 1217 |
-
- Paragraph-level attribution
|
| 1218 |
-
- Human-AI collaboration detection
|
| 1219 |
-
- Edit pattern recognition
|
| 1220 |
-
- Content provenance tracking
|
| 1221 |
|
| 1222 |
-
|
| 1223 |
|
| 1224 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1225 |
|
| 1226 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1227 |
|
| 1228 |
-
|
| 1229 |
-
|----------|------------------|--------------|-------|
|
| 1230 |
-
| **Cloud Infrastructure** | AWS EC2, S3, RDS, CloudFront | $8,000 | Auto-scaling based on demand |
|
| 1231 |
-
| **ML Training** | AWS SageMaker, GPU instances | $12,000 | Spot instances for cost optimization |
|
| 1232 |
-
| **Monitoring & Analytics** | Datadog, Sentry, Mixpanel | $1,500 | Performance tracking and user analytics |
|
| 1233 |
-
| **Development Tools** | GitHub, Jira, Slack, Figma | $500 | Team collaboration and project management |
|
| 1234 |
-
| **Database** | PostgreSQL (RDS), Redis | Included | Primary and cache layers |
|
| 1235 |
-
| **CDN & Storage** | CloudFront, S3 | Included | Global content delivery |
|
| 1236 |
|
| 1237 |
-
|
| 1238 |
|
| 1239 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1240 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1241 |
```
|
| 1242 |
-
โโโโโโโโโโโโโโโโโโโ
|
| 1243 |
-
โ CloudFront โ
|
| 1244 |
-
โ (Global CDN) โ
|
| 1245 |
-
โโโโโโโโโโฌโโโโโโโโโ
|
| 1246 |
-
โ
|
| 1247 |
-
โโโโโโโโโโผโโโโโโโโโ
|
| 1248 |
-
โ Load Balancer โ
|
| 1249 |
-
โ (ALB/NLB) โ
|
| 1250 |
-
โโโโโโโโโโฌโโโโโโโโโ
|
| 1251 |
-
โ
|
| 1252 |
-
โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโ
|
| 1253 |
-
โ โ โ
|
| 1254 |
-
โโโโโโผโโโโโ โโโโโโผโโโโโ โโโโโโผโโโโโ
|
| 1255 |
-
โ API โ โ API โ โ API โ
|
| 1256 |
-
โ Server 1โ โ Server 2โ โ Server Nโ
|
| 1257 |
-
โโโโโโฌโโโโโ โโโโโโฌโโโโโ โโโโโโฌโโโโโ
|
| 1258 |
-
โ โ โ
|
| 1259 |
-
โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโ
|
| 1260 |
-
โ
|
| 1261 |
-
โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโ
|
| 1262 |
-
โ โ โ
|
| 1263 |
-
โโโโโโผโโโโโ โโโโโโผโโโโโ โโโโโโผโโโโโ
|
| 1264 |
-
โ Redis โ โPostgreSQL โ S3 โ
|
| 1265 |
-
โ Cache โ โ Database โ โ Storage โ
|
| 1266 |
-
โโโโโโโโโโโ โโโโโโโโโโโโ โโโโโโโโโโโ
|
| 1267 |
-
```
|
| 1268 |
-
|
| 1269 |
-
### Risk Assessment & Mitigation
|
| 1270 |
-
|
| 1271 |
-
| Risk | Probability | Impact | Mitigation Strategy | Contingency Plan |
|
| 1272 |
-
|------|-------------|--------|---------------------|------------------|
|
| 1273 |
-
| **Model Performance Degradation** | High | Critical | Continuous monitoring, automated retraining, ensemble diversity | Rapid model rollback, human review fallback |
|
| 1274 |
-
| **Adversarial Attacks** | Medium | High | Adversarial training, input sanitization, multiple detection layers | Rate limiting, manual review escalation |
|
| 1275 |
-
| **API Security Breaches** | Low | Critical | OAuth 2.0, API key rotation, request validation, DDoS protection | Immediate key revocation, traffic blocking |
|
| 1276 |
-
| **Infrastructure Scaling Issues** | Medium | High | Auto-scaling groups, load testing, geographic distribution | Traffic shaping, graceful degradation |
|
| 1277 |
-
| **False Positive Complaints** | High | Medium | Transparent confidence scores, appeals process, continuous calibration | Manual expert review, threshold adjustment |
|
| 1278 |
|
| 1279 |
---
|
| 1280 |
|
| 1281 |
-
## ๐ License
|
| 1282 |
-
|
| 1283 |
-
This project is licensed under the MIT License - see the `LICENSE` file for details.
|
| 1284 |
-
|
| 1285 |
-
---
|
| 1286 |
|
| 1287 |
-
|
| 1288 |
|
| 1289 |
-
|
| 1290 |
-
-
|
| 1291 |
-
-
|
| 1292 |
-
-
|
| 1293 |
|
| 1294 |
---
|
| 1295 |
|
| 1296 |
<div align="center">
|
| 1297 |
|
| 1298 |
-
**Built with โค๏ธ
|
| 1299 |
-
|
| 1300 |
-
*Advancing AI transparency and content authenticity*
|
| 1301 |
-
|
| 1302 |
-
[โญ Star us on GitHub](https://github.com/your-org/ai-text-detector) | [๐ Documentation](https://docs.textdetector.ai) | [๐ Report Bug](https://github.com/your-org/ai-text-detector/issues) | [๐ก Request Feature](https://github.com/your-org/ai-text-detector/issues)
|
| 1303 |
-
|
| 1304 |
-
---
|
| 1305 |
-
|
| 1306 |
-
**Version 2.0.0** | Last Updated: October 28, 2025
|
| 1307 |
|
| 1308 |
-
|
| 1309 |
|
| 1310 |
-
</div>
|
|
|
|
| 1 |
+
|
| 2 |
# ๐ AI Text Authentication Platform
|
| 3 |
+
## EnterpriseโGrade AI Content Authentication
|
| 4 |
|
| 5 |

|
| 6 |

|
| 7 |
+

|
| 8 |

|
| 9 |
+

|
| 10 |
|
| 11 |
---
|
| 12 |
|
| 13 |
## ๐ Table of Contents
|
| 14 |
|
| 15 |
+
- [Abstract](#-abstract)
|
| 16 |
- [Overview](#-overview)
|
| 17 |
- [Key Differentiators](#-key-differentiators)
|
| 18 |
- [System Architecture](#-system-architecture)
|
| 19 |
+
- [Workflow / Data Flow](#-workflow--data-flow)
|
| 20 |
- [Detection Metrics & Mathematical Foundation](#-detection-metrics--mathematical-foundation)
|
| 21 |
- [Ensemble Methodology](#-ensemble-methodology)
|
| 22 |
+
- [DomainโAware Detection](#-domain-aware-detection)
|
| 23 |
+
- [Performance & Cost Characteristics](#-performance--cost-characteristics)
|
| 24 |
- [Project Structure](#-project-structure)
|
| 25 |
- [API Endpoints](#-api-endpoints)
|
|
|
|
|
|
|
| 26 |
- [Installation & Setup](#-installation--setup)
|
| 27 |
+
- [Model Management & FirstโRun Behavior](#-model-management--first-run-behavior)
|
|
|
|
| 28 |
- [Frontend Features](#-frontend-features)
|
| 29 |
+
- [Accuracy, Validation & Continuous Improvement](#-accuracy-validation--continuous-improvement)
|
| 30 |
- [Business Model & Market Analysis](#-business-model--market-analysis)
|
| 31 |
+
- [Research Impact & Future Scope](#-research-impact--future-scope)
|
| 32 |
+
- [Infrastructure & Deployment](#-infrastructure--deployment)
|
| 33 |
+
- [Security & Risk Mitigation](#-security--risk-mitigation)
|
| 34 |
+
- [License & Acknowledgments](#-license--acknowledgments)
|
| 35 |
|
| 36 |
---
|
| 37 |
|
| 38 |
+
## ๐ Abstract
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
+
**AI Text Authentication Platform** is a researchโoriented, productionโminded MVP that detects and attributes AIโgenerated text across multiple domains using a multiโmetric, explainable ensemble approach. The platform is designed for reproducibility, extensibility, and realโworld deployment: model weights are autoโfetched from Hugging Face on first run and cached for offline reuse.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
+
This README is researchโgrade (detailed math, methodology, and benchmarks) while being approachable for recruiters and technical reviewers.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
|
| 44 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |
+
## ๐ Overview
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
+
**Problem.** AI generation tools increasingly produce publishable text, creating integrity and verification challenges in education, hiring, publishing, and enterprise content systems.
|
|
|
|
|
|
|
| 49 |
|
| 50 |
+
**Solution.** A domainโaware detector combining six orthogonal metrics (Perplexity, Entropy, Structural, Semantic, Linguistic, DetectGPT perturbation stability) into a confidenceโcalibrated ensemble. Outputs are explainable with sentenceโlevel highlighting, attribution probabilities, and downloadable reports (JSON/PDF).
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
+
**MVP Scope.** Endโtoโend FastAPI backend, lightweight HTML UI, modular metrics, Hugging Face model autoโdownload, and a prototype ensemble classifier. Model weights are not committed to the repo; they are fetched at first run.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
|
| 54 |
---
|
| 55 |
|
| 56 |
## ๐ฏ Key Differentiators
|
| 57 |
|
| 58 |
| Feature | Description | Impact |
|
| 59 |
+
|---|---:|---|
|
| 60 |
+
| **DomainโAware Detection** | Perโdomain thresholding and weight tuning (academic, technical, creative, social) | โ15โ20% accuracy vs generic detectors |
|
| 61 |
+
| **6โMetric Ensemble** | Orthogonal signals across statistical, syntactic and semantic dimensions | Low false positives (โ2โ3%) |
|
| 62 |
+
| **Explainability** | Sentenceโlevel scoring, highlights, and humanโreadable reasoning | Trust & auditability |
|
| 63 |
+
| **Model Attribution** | Likely model identification (GPTโ4, Claude, Gemini, LLaMA, etc.) | Forensic insights |
|
| 64 |
+
| **Auto Model Fetch** | Firstโrun download from Hugging Face, local cache, offline fallback | Lightweight repo & reproducible runs |
|
| 65 |
+
| **Extensible Design** | Plugโin metrics, model registry, and retraining pipeline hooks | Easy research iteration |
|
| 66 |
|
| 67 |
---
|
| 68 |
|
| 69 |
## ๐๏ธ System Architecture
|
| 70 |
|
| 71 |
+
### Architecture (Darkโthemed Mermaid)
|
| 72 |
+
|
| 73 |
+
```mermaid
|
| 74 |
+
%%{init: {'theme': 'dark'}}%%
|
| 75 |
+
flowchart LR
|
| 76 |
+
subgraph FE [Frontend Layer]
|
| 77 |
+
A[Web UI<br/>File Upload & Input]
|
| 78 |
+
B[Interactive Dashboard]
|
| 79 |
+
end
|
| 80 |
+
|
| 81 |
+
subgraph API [API & Gateway]
|
| 82 |
+
C[FastAPI<br/>Auth & Rate Limit]
|
| 83 |
+
end
|
| 84 |
+
|
| 85 |
+
subgraph ORCH [Detection Orchestrator]
|
| 86 |
+
D[Domain Classifier]
|
| 87 |
+
E[Preprocessor]
|
| 88 |
+
F[Metric Coordinator]
|
| 89 |
+
end
|
| 90 |
+
|
| 91 |
+
subgraph METRICS [Metrics Pool]
|
| 92 |
+
P1[Perplexity]
|
| 93 |
+
P2[Entropy]
|
| 94 |
+
P3[Structural]
|
| 95 |
+
P4[Linguistic]
|
| 96 |
+
P5[Semantic]
|
| 97 |
+
P6[DetectGPT]
|
| 98 |
+
end
|
| 99 |
+
|
| 100 |
+
G[Ensemble Classifier]
|
| 101 |
+
H[Postprocessing & Reporter]
|
| 102 |
+
I[Model Manager<br/>(HuggingFace Cache)]
|
| 103 |
+
J[Storage: Logs, Reports, Cache]
|
| 104 |
+
|
| 105 |
+
A --> C
|
| 106 |
+
B --> C
|
| 107 |
+
C --> ORCH
|
| 108 |
+
ORCH --> METRICS
|
| 109 |
+
METRICS --> G
|
| 110 |
+
G --> H
|
| 111 |
+
H --> C
|
| 112 |
+
I --> ORCH
|
| 113 |
+
C --> J
|
| 114 |
+
```
|
| 115 |
+
|
| 116 |
+
**Notes:** The orchestrator schedules parallel metric computation, handles timeouts, and coordinates with the model manager for model loading and caching.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 117 |
|
| 118 |
---
|
| 119 |
|
| 120 |
+
## ๐ Workflow / Data Flow
|
| 121 |
+
|
| 122 |
+
```mermaid
|
| 123 |
+
%%{init: {'theme': 'dark'}}%%
|
| 124 |
+
sequenceDiagram
|
| 125 |
+
participant U as User (UI/API)
|
| 126 |
+
participant API as FastAPI
|
| 127 |
+
participant O as Orchestrator
|
| 128 |
+
participant M as Metrics Pool
|
| 129 |
+
participant E as Ensemble
|
| 130 |
+
participant R as Reporter
|
| 131 |
+
|
| 132 |
+
U->>API: Submit text / upload file
|
| 133 |
+
API->>O: Validate & enqueue job
|
| 134 |
+
O->>M: Preprocess & dispatch metrics (parallel)
|
| 135 |
+
M-->>O: Metric results (async)
|
| 136 |
+
O->>E: Aggregate & calibrate
|
| 137 |
+
E-->>O: Final verdict + uncertainty
|
| 138 |
+
O->>R: Generate highlights & report
|
| 139 |
+
R-->>API: Report ready (JSON/PDF)
|
| 140 |
+
API-->>U: Return analysis + download link
|
| 141 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 142 |
|
| 143 |
---
|
| 144 |
|
| 145 |
+
## ๐งฎ Detection Metrics & Mathematical Foundation
|
| 146 |
|
| 147 |
+
This section provides the exact metric definitions implemented in `metrics/` and rationale for their selection. The ensemble combines these orthogonal signals to increase robustness against adversarial or edited AI content.
|
| 148 |
|
| 149 |
+
### Metric summary (weights are configurable per domain)
|
| 150 |
+
- Perplexity โ 25%
|
| 151 |
+
- Entropy โ 20%
|
| 152 |
+
- Structural โ 15%
|
| 153 |
+
- Semantic โ 15%
|
| 154 |
+
- Linguistic โ 15%
|
| 155 |
+
- DetectGPT (perturbation stability) โ 10%
|
|
|
|
|
|
|
| 156 |
|
| 157 |
+
### 1) Perplexity (25% weight)
|
|
|
|
|
|
|
| 158 |
|
| 159 |
+
**Definition**
|
| 160 |
+
\(\displaystyle Perplexity = \exp\left(-\frac{1}{N}\sum_{i=1}^N \log P(w_i\mid context)\right)\)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 161 |
|
| 162 |
+
**Implementation sketch**
|
| 163 |
```python
|
| 164 |
+
def calculate_perplexity(text, model, k=512):
|
| 165 |
tokens = tokenize(text)
|
| 166 |
log_probs = []
|
|
|
|
| 167 |
for i in range(len(tokens)):
|
| 168 |
context = tokens[max(0, i-k):i]
|
| 169 |
prob = model.get_probability(tokens[i], context)
|
| 170 |
log_probs.append(math.log(prob))
|
| 171 |
+
return math.exp(-sum(log_probs)/len(tokens))
|
|
|
|
| 172 |
```
|
| 173 |
|
| 174 |
+
**Domain calibration example**
|
|
|
|
|
|
|
|
|
|
|
|
|
| 175 |
```python
|
| 176 |
+
if domain == Domain.ACADEMIC:
|
| 177 |
+
perplexity_threshold *= 1.2
|
| 178 |
+
elif domain == Domain.SOCIAL_MEDIA:
|
| 179 |
+
perplexity_threshold *= 0.8
|
| 180 |
```
|
| 181 |
|
| 182 |
+
### 2) Entropy (20% weight)
|
| 183 |
+
|
| 184 |
+
**Shannon entropy (token level)**
|
| 185 |
+
\(\;H(X) = -\sum_{i} p(x_i)\log_2 p(x_i)\)
|
| 186 |
+
|
| 187 |
+
**Implementation sketch**
|
| 188 |
```python
|
| 189 |
+
from collections import Counter
|
| 190 |
def calculate_text_entropy(text):
|
| 191 |
tokens = text.split()
|
| 192 |
token_freq = Counter(tokens)
|
| 193 |
+
total = len(tokens)
|
| 194 |
+
entropy = -sum((f/total) * math.log2(f/total) for f in token_freq.values())
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 195 |
return entropy
|
| 196 |
```
|
| 197 |
|
| 198 |
+
### 3) Structural Metric (15% weight)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 199 |
|
| 200 |
+
**Burstiness**
|
| 201 |
+
\(\displaystyle Burstiness=\frac{\sigma-\mu}{\sigma+\mu}\) where \(\mu\)=mean sentence length, \(\sigma\)=std dev
|
|
|
|
|
|
|
| 202 |
|
| 203 |
+
**Uniformity**
|
| 204 |
+
\(\displaystyle Uniformity = 1 - \frac{\sigma}{\mu}\)
|
|
|
|
|
|
|
|
|
|
| 205 |
|
| 206 |
+
**Sketch**
|
| 207 |
```python
|
| 208 |
def calculate_burstiness(text):
|
| 209 |
sentences = split_sentences(text)
|
| 210 |
lengths = [len(s.split()) for s in sentences]
|
|
|
|
| 211 |
mean_len = np.mean(lengths)
|
| 212 |
std_len = np.std(lengths)
|
|
|
|
| 213 |
burstiness = (std_len - mean_len) / (std_len + mean_len)
|
| 214 |
+
uniformity = 1 - (std_len/mean_len if mean_len > 0 else 0)
|
| 215 |
+
return {'burstiness': burstiness, 'uniformity': uniformity}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 216 |
```
|
| 217 |
|
| 218 |
+
### 4) Semantic Analysis (15% weight)
|
|
|
|
|
|
|
|
|
|
| 219 |
|
| 220 |
+
**Coherence (sentence embedding cosine similarity)**
|
| 221 |
+
\(\displaystyle Coherence=\frac{1}{n}\sum_{i=1}^{n-1} \cos(e_i, e_{i+1})\)
|
|
|
|
|
|
|
|
|
|
| 222 |
|
| 223 |
+
**Sketch**
|
| 224 |
```python
|
| 225 |
+
def calculate_semantic_coherence(text, embed_model):
|
| 226 |
sentences = split_sentences(text)
|
| 227 |
+
embeddings = [embed_model.encode(s) for s in sentences]
|
| 228 |
+
sims = [cosine_similarity(embeddings[i], embeddings[i+1]) for i in range(len(embeddings)-1)]
|
| 229 |
+
return {'mean_coherence': np.mean(sims), 'coherence_variance': np.var(sims)}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 230 |
```
|
| 231 |
|
| 232 |
+
### 5) Linguistic Metric (15% weight)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 233 |
|
| 234 |
+
**POS diversity, parse tree depth, syntactic complexity**
|
|
|
|
|
|
|
|
|
|
|
|
|
| 235 |
|
|
|
|
| 236 |
```python
|
| 237 |
def calculate_linguistic_features(text, nlp_model):
|
| 238 |
doc = nlp_model(text)
|
|
|
|
|
|
|
| 239 |
pos_tags = [token.pos_ for token in doc]
|
| 240 |
+
pos_diversity = len(set(pos_tags))/len(pos_tags)
|
| 241 |
+
depths = [max(get_tree_depth(token) for token in sent) for sent in doc.sents]
|
| 242 |
+
return {'pos_diversity': pos_diversity, 'mean_tree_depth': np.mean(depths)}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 243 |
```
|
| 244 |
|
| 245 |
+
### 6) DetectGPT (10% weight)
|
| 246 |
|
| 247 |
+
**Stability under perturbation** (curvature principle)
|
| 248 |
+
\(\displaystyle Stability = \frac{1}{n}\sum_{j} \left|\log P(x) - \log P(x_{perturbed}^j)\right|\)
|
| 249 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 250 |
```python
|
| 251 |
def detect_gpt_score(text, model, num_perturbations=20):
|
| 252 |
+
original = model.get_log_probability(text)
|
| 253 |
+
diffs = []
|
|
|
|
| 254 |
for _ in range(num_perturbations):
|
| 255 |
perturbed = generate_perturbation(text)
|
| 256 |
+
diffs.append(abs(original - model.get_log_probability(perturbed)))
|
| 257 |
+
return np.mean(diffs)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 258 |
```
|
| 259 |
|
| 260 |
---
|
| 261 |
|
| 262 |
## ๐๏ธ Ensemble Methodology
|
| 263 |
|
| 264 |
+
### ConfidenceโCalibrated Aggregation (high level)
|
| 265 |
+
- Start with domain base weights (e.g., `DOMAIN_WEIGHTS` in `config/threshold_config.py`)
|
| 266 |
+
- Adjust weights per metric with a sigmoid confidence scaling function
|
| 267 |
+
- Normalize and compute weighted aggregate
|
| 268 |
+
- Quantify uncertainty using variance, confidence means, and decision distance from 0.5
|
| 269 |
|
| 270 |
```python
|
| 271 |
def ensemble_aggregation(metric_results, domain):
|
| 272 |
+
base = get_domain_weights(domain)
|
| 273 |
+
adj = {m: base[m] * sigmoid_confidence(r.confidence) for m, r in metric_results.items()}
|
| 274 |
+
total = sum(adj.values())
|
| 275 |
+
final_weights = {k: v/total for k, v in adj.items()}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 276 |
return weighted_aggregate(metric_results, final_weights)
|
| 277 |
```
|
| 278 |
|
| 279 |
### Uncertainty Quantification
|
|
|
|
| 280 |
```python
|
| 281 |
def calculate_uncertainty(metric_results, ensemble_result):
|
| 282 |
+
var_uncert = np.var([r.ai_probability for r in metric_results.values()])
|
| 283 |
+
conf_uncert = 1 - np.mean([r.confidence for r in metric_results.values()])
|
| 284 |
+
decision_uncert = 1 - 2*abs(ensemble_result.ai_probability - 0.5)
|
| 285 |
+
return var_uncert*0.4 + conf_uncert*0.3 + decision_uncert*0.3
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 286 |
```
|
| 287 |
|
| 288 |
+
---
|
| 289 |
+
|
| 290 |
+
## ๐งญ DomainโAware Detection
|
| 291 |
+
|
| 292 |
+
Domain weights and thresholds are configurable. Example weights (in `config/threshold_config.py`):
|
| 293 |
|
| 294 |
```python
|
| 295 |
DOMAIN_WEIGHTS = {
|
| 296 |
+
'academic': {'perplexity':0.22,'entropy':0.18,'structural':0.15,'linguistic':0.20,'semantic':0.15,'detect_gpt':0.10},
|
| 297 |
+
'technical': {'perplexity':0.20,'entropy':0.18,'structural':0.12,'linguistic':0.18,'semantic':0.22,'detect_gpt':0.10},
|
| 298 |
+
'creative': {'perplexity':0.25,'entropy':0.25,'structural':0.20,'linguistic':0.12,'semantic':0.10,'detect_gpt':0.08},
|
| 299 |
+
'social_media': {'perplexity':0.30,'entropy':0.22,'structural':0.15,'linguistic':0.10,'semantic':0.13,'detect_gpt':0.10}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 300 |
}
|
| 301 |
```
|
| 302 |
|
| 303 |
+
### Domain Calibration Strategy (brief)
|
| 304 |
+
- **Academic**: increase linguistic weight, raise perplexity multiplier
|
| 305 |
+
- **Technical**: prioritize semantic coherence, maximize AI threshold to reduce false positives
|
| 306 |
+
- **Creative**: boost entropy & structural weights for burstiness detection
|
| 307 |
+
- **Social Media**: prioritize perplexity and relax linguistic demands
|
| 308 |
+
|
| 309 |
+
---
|
| 310 |
+
|
| 311 |
+
## โก Performance & Cost Characteristics
|
| 312 |
+
|
| 313 |
+
### Processing Times & Resource Estimates
|
| 314 |
+
|
| 315 |
+
| Text Length | Typical Time | vCPU | RAM |
|
| 316 |
+
|---:|---:|---:|---:|
|
| 317 |
+
| Short (100โ500 words) | 1.2 s | 0.8 vCPU | 512 MB |
|
| 318 |
+
| Medium (500โ2000 words) | 3.5 s | 1.2 vCPU | 1 GB |
|
| 319 |
+
| Long (2000+ words) | 7.8 s | 2.0 vCPU | 2 GB |
|
| 320 |
+
|
| 321 |
+
**Optimizations implemented**
|
| 322 |
+
- Parallel metric computation (thread/process pools)
|
| 323 |
+
- Conditional execution & early exit on high confidence
|
| 324 |
+
- Model caching & quantization support for memory efficiency
|
| 325 |
+
|
| 326 |
+
### Cost Estimate (example)
|
| 327 |
+
| Scenario | Time | Cost per analysis | Monthly cost (1k analyses) |
|
| 328 |
+
|---|---:|---:|---:|
|
| 329 |
+
| Short | 1.2 s | $0.0008 | $0.80 |
|
| 330 |
+
| Medium | 3.5 s | $0.0025 | $2.50 |
|
| 331 |
+
| Long | 7.8 s | $0.0058 | $5.80 |
|
| 332 |
+
|
| 333 |
---
|
| 334 |
|
| 335 |
+
## ๐ Project Structure (as in repository)
|
| 336 |
|
| 337 |
```text
|
| 338 |
text_auth/
|
| 339 |
โโโ config/
|
| 340 |
+
โ โโโ model_config.py
|
| 341 |
+
โ โโโ settings.py
|
| 342 |
+
โ โโโ threshold_config.py
|
|
|
|
|
|
|
| 343 |
โโโ data/
|
| 344 |
+
โ โโโ reports/
|
| 345 |
+
โ โโโ uploads/
|
|
|
|
| 346 |
โโโ detector/
|
| 347 |
+
โ โโโ attribution.py
|
| 348 |
+
โ โโโ ensemble.py
|
| 349 |
+
โ โโโ highlighter.py
|
| 350 |
+
โ โโโ orchestrator.py
|
|
|
|
|
|
|
|
|
|
|
|
|
| 351 |
โโโ metrics/
|
| 352 |
+
โ โโโ base_metric.py
|
| 353 |
+
โ โโโ detect_gpt.py
|
| 354 |
+
โ โโโ entropy.py
|
| 355 |
+
โ โโโ linguistic.py
|
| 356 |
+
โ โโโ perplexity.py
|
| 357 |
+
โ โโโ semantic_analysis.py
|
| 358 |
+
โ โโโ structural.py
|
|
|
|
|
|
|
| 359 |
โโโ models/
|
| 360 |
+
โ โโโ model_manager.py
|
| 361 |
+
โ โโโ model_registry.py
|
|
|
|
|
|
|
| 362 |
โโโ processors/
|
| 363 |
+
โ โโโ document_extractor.py
|
| 364 |
+
โ โโโ domain_classifier.py
|
| 365 |
+
โ โโโ language_detector.py
|
| 366 |
+
โ โโโ text_processor.py
|
|
|
|
|
|
|
| 367 |
โโโ reporter/
|
| 368 |
+
โ โโโ reasoning_generator.py
|
| 369 |
+
โ โโโ report_generator.py
|
|
|
|
|
|
|
| 370 |
โโโ ui/
|
| 371 |
+
โ โโโ static/index.html
|
|
|
|
|
|
|
|
|
|
| 372 |
โโโ utils/
|
| 373 |
+
โ โโโ logger.py
|
| 374 |
+
โโโ example.py
|
| 375 |
+
โโโ requirements.txt
|
| 376 |
+
โโโ run.sh
|
| 377 |
+
โโโ text_auth_app.py
|
|
|
|
|
|
|
|
|
|
| 378 |
```
|
| 379 |
|
| 380 |
---
|
| 381 |
|
| 382 |
+
## ๐ API Endpoints (Researchโgrade spec)
|
|
|
|
|
|
|
| 383 |
|
| 384 |
+
### `/api/analyze` โ Text Analysis (POST)
|
| 385 |
+
Analyze raw text. Returns ensemble result, perโmetric scores, attribution, highlights, and reasoning.
|
| 386 |
|
| 387 |
+
**Request (JSON)**
|
|
|
|
|
|
|
| 388 |
```json
|
| 389 |
{
|
| 390 |
+
"text":"...",
|
| 391 |
+
"domain":"academic|technical_doc|creative|social_media",
|
| 392 |
"enable_attribution": true,
|
| 393 |
"enable_highlighting": true,
|
| 394 |
"use_sentence_level": true,
|
|
|
|
| 396 |
}
|
| 397 |
```
|
| 398 |
|
| 399 |
+
**Response (JSON)** โ abbreviated
|
| 400 |
```json
|
| 401 |
{
|
| 402 |
+
"status":"success",
|
| 403 |
+
"analysis_id":"analysis_170...",
|
| 404 |
+
"detection_result":{
|
| 405 |
+
"ensemble_result":{ "final_verdict":"AI-Generated", "ai_probability":0.89, "uncertainty_score":0.23 },
|
| 406 |
+
"metric_results":{ "...": { "ai_probability":0.92, "confidence":0.89 } }
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 407 |
},
|
| 408 |
+
"attribution":{ "predicted_model":"gpt-4", "confidence":0.76 },
|
| 409 |
+
"highlighted_html":"<div>...</div>",
|
| 410 |
+
"reasoning":{ "summary":"...", "key_indicators":[ "...", "..."] }
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 411 |
}
|
| 412 |
```
|
| 413 |
|
| 414 |
+
### `/api/analyze/file` โ File Analysis (POST, multipart/form-data)
|
| 415 |
+
Supports PDF, DOCX, TXT, DOC, MD. File size limit default: 10MB. Returns same structure as text analyze endpoint.
|
|
|
|
|
|
|
| 416 |
|
| 417 |
+
### `/api/report/generate` โ Report Generation (POST)
|
| 418 |
+
Generate downloadable JSON or PDF reports for a given analysis id.
|
| 419 |
|
| 420 |
+
### Utility endpoints
|
| 421 |
+
- `GET /health` โ health status, models loaded, uptime
|
| 422 |
+
- `GET /api/domains` โ supported domains and thresholds
|
| 423 |
+
- `GET /api/models` โ detectable model list
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 424 |
|
| 425 |
---
|
| 426 |
|
| 427 |
+
## โ๏ธ Installation & Setup
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 428 |
|
| 429 |
### Prerequisites
|
| 430 |
+
- Python 3.8+
|
| 431 |
+
- 4GB RAM (8GB recommended)
|
| 432 |
+
- Disk: 2GB (models & deps)
|
| 433 |
+
- OS: Linux/macOS/Windows (WSL supported)
|
| 434 |
|
| 435 |
+
### Quickstart
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 436 |
```bash
|
| 437 |
+
git clone https://github.com/satyaki-mitra/text_authentication.git
|
| 438 |
+
cd text_authentication
|
|
|
|
|
|
|
|
|
|
| 439 |
python -m venv venv
|
| 440 |
+
source venv/bin/activate
|
|
|
|
|
|
|
| 441 |
pip install -r requirements.txt
|
| 442 |
+
# Copy .env.example -> .env and set HF_TOKEN if using private models
|
| 443 |
+
python text_auth_app.py
|
| 444 |
+
# or: ./run.sh
|
|
|
|
| 445 |
```
|
| 446 |
|
| 447 |
+
**Dev tips**
|
| 448 |
+
- Use `DEBUG=True` in `config/settings.py` for verbose logs
|
| 449 |
+
- For containerized runs, see `Dockerfile` template (example included in repo suggestions)
|
|
|
|
| 450 |
|
| 451 |
+
---
|
| 452 |
|
| 453 |
+
## ๐ง Model Management & FirstโRun Behavior
|
| 454 |
|
| 455 |
+
- The application **automatically downloads** required model weights from Hugging Face on the first run and caches them to the local HF cache (or a custom path specified in `config/model_config.py`).
|
| 456 |
+
- Model IDs and revisions are maintained in `models/model_registry.py` and referenced by `models/model_manager.py`.
|
| 457 |
+
- **Best practices implemented**:
|
| 458 |
+
- Pin model revisions (e.g., `repo_id@v1.2.0`)
|
| 459 |
+
- Resumeable downloads using `huggingface_hub.snapshot_download`
|
| 460 |
+
- Optional `OFFLINE_MODE` to load local model paths
|
| 461 |
+
- Optional integrity checks (SHA256) after download
|
| 462 |
+
- Support for private HF repos using `HF_TOKEN` env var
|
| 463 |
+
|
| 464 |
+
**Example snippet**
|
| 465 |
```python
|
| 466 |
+
from huggingface_hub import snapshot_download
|
| 467 |
+
snapshot_download(repo_id="satyaki-mitra/text-detector-v1", local_dir="./models/text-detector-v1")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 468 |
```
|
| 469 |
+
|
| 470 |
---
|
| 471 |
|
| 472 |
+
## ๐จ Frontend Features (UI)
|
| 473 |
|
| 474 |
+
- Dualโpanel responsive web UI (left: input / upload; right: live analysis)
|
| 475 |
+
- Sentenceโlevel color highlights with tooltips and perโmetric breakdown
|
| 476 |
+
- Progressive analysis updates (metric-level streaming)
|
| 477 |
+
- Theme: light/dark toggle (UI respects user preference)
|
| 478 |
+
- Export: JSON and PDF report download
|
| 479 |
+
- Interactive elements: click to expand sentence reasoning, copy text snippets, download raw metrics
|
| 480 |
|
| 481 |
+
---
|
| 482 |
|
| 483 |
+
## ๐ Accuracy, Validation & Continuous Improvement
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 484 |
|
| 485 |
+
### Benchmark Summary (reported across internal test sets)
|
| 486 |
+
| Scenario | Samples | Accuracy | Precision | Recall |
|
| 487 |
+
|---|---:|---:|---:|---:|
|
| 488 |
+
| GPTโ4 | 5,000 | 95.8% | 96.2% | 95.3% |
|
| 489 |
+
| Claudeโ3 | 3,000 | 94.2% | 94.8% | 93.5% |
|
| 490 |
+
| Gemini Pro | 2,500 | 93.6% | 94.1% | 93.0% |
|
| 491 |
+
| LLaMA 2 | 2,000 | 92.8% | 93.3% | 92.2% |
|
| 492 |
+
| Human Academic | 10,000 | 96.1% | 95.7% | 96.4% |
|
| 493 |
+
| Mixed Content | 2,000 | 88.7% | 89.2% | 88.1% |
|
| 494 |
+
| **Overall** | 29,500 | **94.3%** | **94.6%** | **94.1%** |
|
| 495 |
|
| 496 |
+
**Confusion matrix (abbreviated)**:
|
| 497 |
```
|
| 498 |
+
Predicted โ AI Human Mixed
|
| 499 |
+
Actual AI 4750 180 70 (5,000)
|
| 500 |
+
Actual Human 240 9680 80 (10,000)
|
| 501 |
+
Actual Mixed 420 580 1000 (2,000)
|
|
|
|
| 502 |
```
|
| 503 |
|
| 504 |
+
### Continuous Improvement Pipeline
|
| 505 |
+
- Regular retraining & calibration on new model releases
|
| 506 |
+
- Feedback loop: user reported FP integrated into training
|
| 507 |
+
- A/B testing for weight adjustments
|
| 508 |
+
- Monthly accuracy audits & quarterly model updates
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 509 |
|
| 510 |
---
|
| 511 |
|
| 512 |
## ๐ผ Business Model & Market Analysis
|
| 513 |
|
| 514 |
+
**TAM**: $20B (education, hiring, publishing) โ see detailed breakdown in original repo.
|
| 515 |
+
**Use cases**: universities (plagiarism & integrity), hiring platforms (resume authenticity), publishers (content verification), social platforms (spam & SEO abuse).
|
| 516 |
|
| 517 |
+
**Competitive landscape** (summary)
|
| 518 |
+
- GPTZero, Originality.ai, Copyleaks โ our advantages: domain adaptation, explainability, attribution, lower false positives and competitive pricing.
|
|
|
|
|
|
|
| 519 |
|
| 520 |
+
**Monetization ideas**
|
| 521 |
+
- SaaS subscription (seat / monthly analyze limits)
|
| 522 |
+
- Enterprise licensing with onโprem deployment & priority support
|
| 523 |
+
- API billing (perโanalysis tiered pricing)
|
| 524 |
+
- Onboarding & consulting for institutions
|
| 525 |
|
| 526 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 527 |
|
| 528 |
+
## ๐ฎ Research Impact & Future Scope
|
|
|
|
|
|
|
|
|
|
|
|
|
| 529 |
|
| 530 |
+
**Research directions**
|
| 531 |
+
- Adversarial robustness (paraphrase & synonym attacks)
|
| 532 |
+
- Crossโmodel generalization & zeroโshot detection
|
| 533 |
+
- Fineโgrained attribution (model versioning, temperature estimation)
|
| 534 |
+
- Explainability: counterfactual examples & feature importance visualization
|
| 535 |
|
| 536 |
+
**Planned features (Q1โQ2 2026)**
|
| 537 |
+
- Multiโlanguage support (Spanish, French, German, Chinese)
|
| 538 |
+
- Realโtime streaming API (WebSocket)
|
| 539 |
+
- Fineโgrained attribution & generation parameter estimation
|
| 540 |
+
- Institutionโspecific calibration & admin dashboards
|
|
|
|
| 541 |
|
| 542 |
---
|
| 543 |
|
| 544 |
+
## ๐๏ธ Infrastructure & Deployment
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 545 |
|
| 546 |
+
### Deployment (Mermaid dark diagram)
|
| 547 |
|
| 548 |
+
```mermaid
|
| 549 |
+
%%{init: {'theme': 'dark'}}%%
|
| 550 |
+
flowchart LR
|
| 551 |
+
CDN[CloudFront / CDN] --> LB[Load Balancer (ALB/NLB)]
|
| 552 |
+
LB --> API1[API Server 1]
|
| 553 |
+
LB --> API2[API Server 2]
|
| 554 |
+
LB --> APIN[API Server N]
|
| 555 |
+
API1 --> Cache[Redis Cache]
|
| 556 |
+
API1 --> DB[PostgreSQL]
|
| 557 |
+
API1 --> S3[S3 / Model Storage]
|
| 558 |
+
DB --> Backup[(RDS Snapshot)]
|
| 559 |
+
S3 --> Archive[(Cold Storage)]
|
| 560 |
+
```
|
| 561 |
|
| 562 |
+
**Deployment notes**
|
| 563 |
+
- Containerize app with Docker, orchestrate with Kubernetes or ECS for scale
|
| 564 |
+
- Autoscaling groups for API servers & worker nodes
|
| 565 |
+
- Use spot GPU instances for retraining & large metric compute jobs
|
| 566 |
+
- Integrate observability: Prometheus + Grafana, Sentry for errors, Datadog if available
|
| 567 |
|
| 568 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 569 |
|
| 570 |
+
## ๐ Security & Risk Mitigation
|
| 571 |
|
| 572 |
+
**Primary risks & mitigations**
|
| 573 |
+
- Model performance drift โ monitoring + retraining + rollback
|
| 574 |
+
- Adversarial attacks โ adversarial training & input sanitization
|
| 575 |
+
- Data privacy โ avoid storing raw uploads unless user consents; redact PII in reports
|
| 576 |
+
- Secrets management โ use env vars, vaults, and avoid committing tokens
|
| 577 |
+
- Rate limits & auth โ JWT/OAuth2, API key rotation, request throttling
|
| 578 |
|
| 579 |
+
**File handling best practices (examples)**
|
| 580 |
+
```python
|
| 581 |
+
ALLOWED_EXT = {'.txt','.pdf','.docx','.doc','.md'}
|
| 582 |
+
def allowed_file(filename):
|
| 583 |
+
return any(filename.lower().endswith(ext) for ext in ALLOWED_EXT)
|
| 584 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 585 |
|
| 586 |
---
|
| 587 |
|
| 588 |
+
## ๐ License & Acknowledgments
|
|
|
|
|
|
|
|
|
|
|
|
|
| 589 |
|
| 590 |
+
This project is licensed under the **MIT License** โ see `LICENSE` in the repo.
|
| 591 |
|
| 592 |
+
Acknowledgments:
|
| 593 |
+
- DetectGPT (Mitchell et al., 2023) โ inspiration for perturbation-based detection
|
| 594 |
+
- Hugging Face Transformers & Hub
|
| 595 |
+
- Open-source NLP community and early beta testers
|
| 596 |
|
| 597 |
---
|
| 598 |
|
| 599 |
<div align="center">
|
| 600 |
|
| 601 |
+
**Built with โค๏ธ โ AI transparency, accountability, and realโworld readiness.**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 602 |
|
| 603 |
+
*Version 1.0.0 โ Last Updated: October 28, 2025*
|
| 604 |
|
| 605 |
+
</div>
|
docs/BLOGPOST.md
ADDED
|
@@ -0,0 +1,182 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ๐ง Building the AI Text Authentication Platform โ Detecting the Fingerprints of Machine-Generated Text
|
| 2 |
+
|
| 3 |
+
**Author:** *Satyaki Mitra โ Data Scientist, AI Researcher*
|
| 4 |
+
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
## ๐ The Context โ When Machines Started Sounding Human
|
| 8 |
+
|
| 9 |
+
In the last few years, AI models like GPT-4, Claude, and Gemini have rewritten the boundaries of natural language generation.
|
| 10 |
+
From essays to resumes, from research papers to blogs, AI can now mimic the nuances of human writing with unsettling precision.
|
| 11 |
+
|
| 12 |
+
This explosion of generative text brings opportunity โ but also uncertainty.
|
| 13 |
+
When *everything* can be generated, how do we know whatโs *authentic*?
|
| 14 |
+
|
| 15 |
+
That question led me to build the **AI Text Authentication Platform** โ a domain-aware, explainable system that detects whether a piece of text was written by a human or an AI model.
|
| 16 |
+
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
## ๐ The Idea โ Beyond Binary Detection
|
| 20 |
+
|
| 21 |
+
Most existing detectors approach the problem as a yes/no question:
|
| 22 |
+
> โWas this written by AI?โ
|
| 23 |
+
|
| 24 |
+
But the real challenge is more nuanced.
|
| 25 |
+
Different domains โ academic papers, social media posts, technical documents, or creative writing โ have very different stylistic baselines.
|
| 26 |
+
A generic model often misfires in one domain while succeeding in another.
|
| 27 |
+
|
| 28 |
+
I wanted to build something smarter โ
|
| 29 |
+
an adaptive detector that understands *context*, *writing style*, and *linguistic diversity*, and still offers transparency in its decision-making.
|
| 30 |
+
|
| 31 |
+
---
|
| 32 |
+
|
| 33 |
+
## ๐งฎ The Statistical Backbone โ Blending Metrics and Machine Learning
|
| 34 |
+
|
| 35 |
+
Coming from a statistics background, I wanted to merge the **interpretability of statistical metrics** with the **depth of modern transformer models**.
|
| 36 |
+
Instead of relying purely on embeddings or a classifier, I designed a **multi-metric ensemble** that captures both linguistic and structural signals.
|
| 37 |
+
|
| 38 |
+
The system uses six core metrics:
|
| 39 |
+
|
| 40 |
+
| Metric | What it Measures | Why it Matters |
|
| 41 |
+
|:--|:--|:--|
|
| 42 |
+
| **Perplexity** | Predictability of word sequences | AI text tends to have smoother probability distributions |
|
| 43 |
+
| **Entropy** | Diversity of token use | Humans are more chaotic; models are more uniform |
|
| 44 |
+
| **Structural (Burstiness)** | Variation in sentence lengths | AI often produces rhythmically even sentences |
|
| 45 |
+
| **Semantic Coherence** | Flow of meaning between sentences | LLMs maintain strong coherence, sometimes too strong |
|
| 46 |
+
| **Linguistic Features** | Grammar complexity, POS diversity | Human syntax is idiosyncratic; AIโs is hyper-consistent |
|
| 47 |
+
| **DetectGPT Stability** | Robustness to perturbations | AI text collapses faster under small changes |
|
| 48 |
+
|
| 49 |
+
Each metric produces an independent *AI-likelihood score*.
|
| 50 |
+
These are then aggregated through a **confidence-calibrated ensemble**, which adjusts weights based on domain context and model confidence.
|
| 51 |
+
|
| 52 |
+
Itโs not just machine learning โ itโs *statistical reasoning, linguistic insight, and AI interpretability* working together.
|
| 53 |
+
|
| 54 |
+
---
|
| 55 |
+
|
| 56 |
+
## ๐๏ธ The Architecture โ A System That Learns, Explains, and Scales
|
| 57 |
+
|
| 58 |
+
I designed the system with modularity in mind.
|
| 59 |
+
Every layer is replaceable and extendable, so researchers can plug in new metrics, models, or rules without breaking the pipeline.
|
| 60 |
+
|
| 61 |
+
```mermaid
|
| 62 |
+
%%{init: {'theme': 'dark'}}%%
|
| 63 |
+
flowchart LR
|
| 64 |
+
UI[Web UI & API]
|
| 65 |
+
ORCH[Orchestrator]
|
| 66 |
+
METRICS[Metric Engines]
|
| 67 |
+
ENSEMBLE[Confidence Ensemble]
|
| 68 |
+
REPORT[Explanation + Report]
|
| 69 |
+
UI --> ORCH --> METRICS --> ENSEMBLE --> REPORT --> UI
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
The backend runs on FastAPI, powered by PyTorch, Transformers, and Scikit-Learn.
|
| 73 |
+
Models are fetched dynamically from Hugging Face on the first run, cached locally, and version-pinned for reproducibility.
|
| 74 |
+
This keeps the repository lightweight but production-ready.
|
| 75 |
+
|
| 76 |
+
The UI (built in HTML + CSS + vanilla JS) provides live metric breakdowns, highlighting sentences most responsible for the final verdict.
|
| 77 |
+
|
| 78 |
+
---
|
| 79 |
+
|
| 80 |
+
## ๐ง Domain Awareness โ One Size Doesnโt Fit All
|
| 81 |
+
|
| 82 |
+
AI writing โfeelsโ different across contexts.
|
| 83 |
+
Academic writing has long, precise sentences with low entropy, while creative writing is expressive and variable.
|
| 84 |
+
|
| 85 |
+
To handle this, I introduced domain calibration.
|
| 86 |
+
Each domain has its own weight configuration, reflecting what matters most in that context:
|
| 87 |
+
|
| 88 |
+
| Domain | Emphasis |
|
| 89 |
+
| :----------- | :------------------------------- |
|
| 90 |
+
| Academic | Linguistic structure, perplexity |
|
| 91 |
+
| Technical | Semantic coherence, consistency |
|
| 92 |
+
| Creative | Entropy, burstiness |
|
| 93 |
+
| Social Media | Short-form unpredictability |
|
| 94 |
+
|
| 95 |
+
This calibration alone improved accuracy by nearly 20% over generic baselines.
|
| 96 |
+
|
| 97 |
+
---
|
| 98 |
+
|
| 99 |
+
## โ๏ธ Engineering Choices That Matter
|
| 100 |
+
|
| 101 |
+
The platform auto-downloads models from Hugging Face on first run โ a deliberate design for scalability.
|
| 102 |
+
It supports offline mode for enterprises and validates checksums for model integrity.
|
| 103 |
+
|
| 104 |
+
Error handling and caching logic were built to ensure robustness โ no dependency on manual model management.
|
| 105 |
+
|
| 106 |
+
This kind of product-level thinking is essential when transitioning from proof-of-concept to MVP.
|
| 107 |
+
|
| 108 |
+
---
|
| 109 |
+
|
| 110 |
+
## ๐ The Results โ What the Data Says
|
| 111 |
+
|
| 112 |
+
Across test sets covering GPT-4, Claude-3, Gemini, and LLaMA content, the system achieved:
|
| 113 |
+
|
| 114 |
+
| Model | Accuracy | Precision | Recall |
|
| 115 |
+
| :---------- | --------: | --------: | --------: |
|
| 116 |
+
| GPT-4 | 95.8% | 96.2% | 95.3% |
|
| 117 |
+
| Claude-3 | 94.2% | 94.8% | 93.5% |
|
| 118 |
+
| Gemini Pro | 93.6% | 94.1% | 93.0% |
|
| 119 |
+
| LLaMA 2 | 92.8% | 93.3% | 92.2% |
|
| 120 |
+
| **Overall** | **94.3%** | **94.6%** | **94.1%** |
|
| 121 |
+
|
| 122 |
+
|
| 123 |
+
False positives dropped below 3% after domain-specific recalibration โ a huge leap compared to most commercial detectors.
|
| 124 |
+
|
| 125 |
+
---
|
| 126 |
+
|
| 127 |
+
## ๐ก Lessons Learned
|
| 128 |
+
|
| 129 |
+
This project wasnโt just about detecting AI text โ it was about understanding why models write the way they do.
|
| 130 |
+
|
| 131 |
+
I learned how deeply metrics like entropy and burstiness connect to human psychology.
|
| 132 |
+
I also learned the importance of explainability โ users trust results only when they can see why a decision was made.
|
| 133 |
+
|
| 134 |
+
Balancing statistical rigor with engineering pragmatism turned this into one of my most complete data science projects.
|
| 135 |
+
|
| 136 |
+
---
|
| 137 |
+
|
| 138 |
+
## ๐ผ Real-World Impact and Vision
|
| 139 |
+
|
| 140 |
+
AI text detection has implications across multiple industries:
|
| 141 |
+
|
| 142 |
+
๐ Education: plagiarism and authorship validation
|
| 143 |
+
|
| 144 |
+
๐ผ Hiring: resume authenticity and candidate writing verification
|
| 145 |
+
|
| 146 |
+
๐ฐ Publishing: editorial transparency
|
| 147 |
+
|
| 148 |
+
๐ Social media: moderation and misinformation detection
|
| 149 |
+
|
| 150 |
+
I envision this project evolving into a scalable SaaS or institutional tool โ blending detection, attribution, and linguistic analytics into one explainable AI platform.
|
| 151 |
+
|
| 152 |
+
---
|
| 153 |
+
|
| 154 |
+
## ๐ฎ Whatโs Next
|
| 155 |
+
|
| 156 |
+
Expanding to multilingual support
|
| 157 |
+
|
| 158 |
+
Incorporating counterfactual explainers (LIME, SHAP)
|
| 159 |
+
|
| 160 |
+
Model-specific attribution (โWhich LLM wrote this?โ)
|
| 161 |
+
|
| 162 |
+
Continuous benchmark pipelines for new generative models
|
| 163 |
+
|
| 164 |
+
The whitepaper version dives deeper into methodology, mathematics, and system design.
|
| 165 |
+
|
| 166 |
+
๐ Read the full Technical Whitepaper (PDF)
|
| 167 |
+
|
| 168 |
+
---
|
| 169 |
+
|
| 170 |
+
## โ๏ธ Closing Thoughts
|
| 171 |
+
|
| 172 |
+
As AI blurs the line between human and machine creativity, itโs essential that we build systems that restore trust, traceability, and transparency.
|
| 173 |
+
Thatโs what the AI Text Authentication Platform stands for โ not just detection, but understanding the fingerprints of intelligence itself.
|
| 174 |
+
|
| 175 |
+
---
|
| 176 |
+
|
| 177 |
+
## Author:
|
| 178 |
+
Satyaki Mitra โ Data Scientist, AI Researcher
|
| 179 |
+
|
| 180 |
+
๐ Building interpretable AI systems that make machine learning transparent and human-centric.
|
| 181 |
+
|
| 182 |
+
---
|