Spaces:
Sleeping
π¨ Launch DimensioDepth - Advanced AI Depth Estimation
Browse filesFEATURES:
- π― Gradio UI with multiple tabs (Depth Estimation, Comparison, 3D Parallax, Batch)
- π Auto-download Depth-Anything V2 models from Hugging Face
- π¨ 8 colormap styles (Inferno, Viridis, Plasma, Turbo, Magma, Hot, Ocean, Rainbow)
- β‘ Demo Mode with synthetic depth (works without models!)
- π¬ Side-by-side comparison view
- π 3D parallax depth displacement effects
- π¦ Batch processing support
BACKEND:
- FastAPI depth estimation API
- ONNX Runtime GPU acceleration
- Model auto-loading from HF Hub
- Smart fallback to Demo Mode
DEMO MODE:
- Ultra-fast (<50ms)
- Edge detection + intensity analysis
- No model downloads needed
- Surprisingly good quality!
MODELS:
- Small Model (94MB) - Fast preview
- Large Model (1.3GB) - High quality (optional)
- Auto-cache on HF Spaces
Ready to transform 2D images into stunning 3D depth visualizations! β¨
Made with β€οΈ for the AI community
- .gitignore +42 -0
- README.md +180 -7
- app.py +480 -0
- backend/.env.example +41 -0
- backend/api/main.py +394 -0
- backend/config.py +62 -0
- backend/download_models.py +177 -0
- backend/requirements.txt +29 -0
- backend/test_api.py +293 -0
- backend/test_model.py +34 -0
- backend/utils/demo_depth.py +92 -0
- backend/utils/image_processing.py +197 -0
- backend/utils/model_loader.py +231 -0
- requirements.txt +14 -0
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Python
|
| 2 |
+
__pycache__/
|
| 3 |
+
*.py[cod]
|
| 4 |
+
*$py.class
|
| 5 |
+
*.so
|
| 6 |
+
.Python
|
| 7 |
+
*.egg-info/
|
| 8 |
+
dist/
|
| 9 |
+
build/
|
| 10 |
+
venv/
|
| 11 |
+
.env
|
| 12 |
+
|
| 13 |
+
# IDE
|
| 14 |
+
.vscode/
|
| 15 |
+
.idea/
|
| 16 |
+
*.swp
|
| 17 |
+
*.swo
|
| 18 |
+
|
| 19 |
+
# OS
|
| 20 |
+
.DS_Store
|
| 21 |
+
Thumbs.db
|
| 22 |
+
|
| 23 |
+
# Models (too large for git)
|
| 24 |
+
backend/models/cache/*.onnx
|
| 25 |
+
*.onnx
|
| 26 |
+
*.pth
|
| 27 |
+
*.pt
|
| 28 |
+
|
| 29 |
+
# Logs
|
| 30 |
+
*.log
|
| 31 |
+
|
| 32 |
+
# Temporary files
|
| 33 |
+
*.tmp
|
| 34 |
+
*.temp
|
| 35 |
+
|
| 36 |
+
# Frontend build
|
| 37 |
+
frontend/node_modules/
|
| 38 |
+
frontend/dist/
|
| 39 |
+
frontend/.vite/
|
| 40 |
+
|
| 41 |
+
# Gradio
|
| 42 |
+
flagged/
|
|
@@ -1,14 +1,187 @@
|
|
| 1 |
---
|
| 2 |
title: DimensioDepth
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
-
sdk_version:
|
| 8 |
app_file: app.py
|
| 9 |
-
pinned:
|
| 10 |
license: mit
|
| 11 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
---
|
| 13 |
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
title: DimensioDepth
|
| 3 |
+
emoji: π¨
|
| 4 |
+
colorFrom: blue
|
| 5 |
+
colorTo: purple
|
| 6 |
sdk: gradio
|
| 7 |
+
sdk_version: 4.44.0
|
| 8 |
app_file: app.py
|
| 9 |
+
pinned: true
|
| 10 |
license: mit
|
| 11 |
+
tags:
|
| 12 |
+
- depth-estimation
|
| 13 |
+
- computer-vision
|
| 14 |
+
- depth-anything-v2
|
| 15 |
+
- 3d-visualization
|
| 16 |
+
- image-processing
|
| 17 |
---
|
| 18 |
|
| 19 |
+
# π¨ DimensioDepth - Add Dimension to Everything
|
| 20 |
+
|
| 21 |
+
Transform 2D images into stunning 3D depth visualizations with state-of-the-art AI depth estimation.
|
| 22 |
+
|
| 23 |
+
## β¨ Features
|
| 24 |
+
|
| 25 |
+
### π― Advanced Depth Estimation
|
| 26 |
+
- **Fast Preview Mode** - Real-time depth estimation (~50-100ms)
|
| 27 |
+
- **High Quality Mode** - Production-grade accuracy (~500-1500ms)
|
| 28 |
+
- **Multiple Colormaps** - Inferno, Viridis, Plasma, Turbo, Magma, Hot, Ocean, Rainbow
|
| 29 |
+
- **Demo Mode** - Works instantly without downloading models!
|
| 30 |
+
|
| 31 |
+
### π¬ Visualization Options
|
| 32 |
+
- **Colored Depth Maps** - Beautiful visualization with customizable color schemes
|
| 33 |
+
- **Grayscale Depth** - Classic depth representation
|
| 34 |
+
- **Side-by-Side Comparison** - Original vs. Depth view
|
| 35 |
+
- **3D Parallax Effect** - Create depth displacement visualizations
|
| 36 |
+
|
| 37 |
+
### π¦ Batch Processing
|
| 38 |
+
- Process multiple images at once
|
| 39 |
+
- Consistent depth estimation across your dataset
|
| 40 |
+
- Perfect for batch workflows
|
| 41 |
+
|
| 42 |
+
## π How to Use
|
| 43 |
+
|
| 44 |
+
### Basic Usage
|
| 45 |
+
1. **Upload an Image** - Drag & drop or click to upload
|
| 46 |
+
2. **Choose Quality Mode** - Fast for preview, High Quality for final output
|
| 47 |
+
3. **Select Colormap** - Pick your favorite depth visualization style
|
| 48 |
+
4. **Generate** - Click the button and watch the magic happen! β¨
|
| 49 |
+
|
| 50 |
+
### Advanced Features
|
| 51 |
+
- **Side-by-Side**: Compare original and depth maps
|
| 52 |
+
- **3D Parallax**: Create depth displacement effects
|
| 53 |
+
- **Batch Processing**: Process multiple images efficiently
|
| 54 |
+
|
| 55 |
+
## π οΈ Technical Details
|
| 56 |
+
|
| 57 |
+
### Architecture
|
| 58 |
+
- **Model**: Depth-Anything V2 (ViT-S and ViT-L variants)
|
| 59 |
+
- **Inference**: ONNX Runtime with GPU acceleration
|
| 60 |
+
- **Backend**: FastAPI + Python
|
| 61 |
+
- **Frontend**: Gradio
|
| 62 |
+
- **3D Rendering**: Custom GLSL shaders (original web app)
|
| 63 |
+
|
| 64 |
+
### Performance
|
| 65 |
+
| Mode | Model | Speed | Quality |
|
| 66 |
+
|------|-------|-------|---------|
|
| 67 |
+
| Fast Preview | Small (94MB) | 50-100ms | Good |
|
| 68 |
+
| High Quality | Large (1.3GB) | 500-1500ms | Excellent |
|
| 69 |
+
| Demo Mode | Synthetic | <50ms | Decent |
|
| 70 |
+
|
| 71 |
+
### Demo Mode
|
| 72 |
+
Don't have models downloaded? No problem! DimensioDepth includes a **Demo Mode** that uses:
|
| 73 |
+
- Edge detection
|
| 74 |
+
- Intensity analysis
|
| 75 |
+
- Gaussian smoothing
|
| 76 |
+
- Depth synthesis algorithms
|
| 77 |
+
|
| 78 |
+
This creates surprisingly good depth maps without any AI models!
|
| 79 |
+
|
| 80 |
+
## π Use Cases
|
| 81 |
+
|
| 82 |
+
### π¨ Creative & Artistic
|
| 83 |
+
- Create depth-enhanced photos
|
| 84 |
+
- Generate 3D parallax effects
|
| 85 |
+
- Artistic depth visualization
|
| 86 |
+
|
| 87 |
+
### π¬ VFX & Film Production
|
| 88 |
+
- Depth map generation for compositing
|
| 89 |
+
- 3D reconstruction preparation
|
| 90 |
+
- Scene depth analysis
|
| 91 |
+
|
| 92 |
+
### π¬ Research & Development
|
| 93 |
+
- Computer vision research
|
| 94 |
+
- Depth perception studies
|
| 95 |
+
- Dataset augmentation
|
| 96 |
+
|
| 97 |
+
### π± Social Media & Content Creation
|
| 98 |
+
- Create engaging 3D effects
|
| 99 |
+
- Enhance photos with depth
|
| 100 |
+
- Generate unique visual content
|
| 101 |
+
|
| 102 |
+
## π About Depth-Anything V2
|
| 103 |
+
|
| 104 |
+
Depth-Anything V2 is a state-of-the-art monocular depth estimation model that:
|
| 105 |
+
- Works on any image (indoor/outdoor, any domain)
|
| 106 |
+
- Produces high-quality depth maps
|
| 107 |
+
- Runs efficiently on consumer hardware
|
| 108 |
+
- Supports both fast and accurate modes
|
| 109 |
+
|
| 110 |
+
[Read the Paper](https://arxiv.org/abs/2406.09414)
|
| 111 |
+
|
| 112 |
+
## π Examples
|
| 113 |
+
|
| 114 |
+
Try these types of images:
|
| 115 |
+
- **Portraits** - See facial depth structure
|
| 116 |
+
- **Landscapes** - Visualize scene depth layers
|
| 117 |
+
- **Architecture** - Analyze building geometry
|
| 118 |
+
- **Street Scenes** - Understand urban depth
|
| 119 |
+
- **Nature** - Explore organic depth patterns
|
| 120 |
+
|
| 121 |
+
## π‘ Tips for Best Results
|
| 122 |
+
|
| 123 |
+
1. **Image Quality**: Higher resolution = better depth detail
|
| 124 |
+
2. **Lighting**: Well-lit images produce clearer depth maps
|
| 125 |
+
3. **Contrast**: Images with good contrast show better depth separation
|
| 126 |
+
4. **Colormap**: Inferno is great for general use, Viridis for scientific visualization
|
| 127 |
+
5. **Mode Selection**: Use Fast for experimentation, High Quality for final output
|
| 128 |
+
|
| 129 |
+
## π§ Running Locally
|
| 130 |
+
|
| 131 |
+
Want to run DimensioDepth on your own machine?
|
| 132 |
+
|
| 133 |
+
```bash
|
| 134 |
+
# Clone the repository
|
| 135 |
+
git clone https://github.com/chromahubz/dimensiodepth.git
|
| 136 |
+
cd dimensiodepth
|
| 137 |
+
|
| 138 |
+
# Install dependencies
|
| 139 |
+
pip install -r requirements.txt
|
| 140 |
+
|
| 141 |
+
# Run the Gradio app
|
| 142 |
+
python app.py
|
| 143 |
+
```
|
| 144 |
+
|
| 145 |
+
For the full web experience with Three.js 3D viewer:
|
| 146 |
+
```bash
|
| 147 |
+
# Backend
|
| 148 |
+
cd backend
|
| 149 |
+
pip install -r requirements.txt
|
| 150 |
+
python -m uvicorn api.main:app --reload
|
| 151 |
+
|
| 152 |
+
# Frontend (separate terminal)
|
| 153 |
+
cd frontend
|
| 154 |
+
npm install
|
| 155 |
+
npm run dev
|
| 156 |
+
```
|
| 157 |
+
|
| 158 |
+
## π― Roadmap
|
| 159 |
+
|
| 160 |
+
- [ ] Video depth estimation
|
| 161 |
+
- [ ] Point cloud export
|
| 162 |
+
- [ ] 3D mesh reconstruction
|
| 163 |
+
- [ ] Real-time webcam depth
|
| 164 |
+
- [ ] Depth-guided editing tools
|
| 165 |
+
- [ ] Multi-frame temporal consistency
|
| 166 |
+
|
| 167 |
+
## π License
|
| 168 |
+
|
| 169 |
+
MIT License - Feel free to use in your projects!
|
| 170 |
+
|
| 171 |
+
## π Acknowledgments
|
| 172 |
+
|
| 173 |
+
- **Depth-Anything V2** - For the amazing depth estimation model
|
| 174 |
+
- **Hugging Face** - For the incredible Spaces platform
|
| 175 |
+
- **Gradio** - For making ML demos beautiful and easy
|
| 176 |
+
|
| 177 |
+
## π Contact & Links
|
| 178 |
+
|
| 179 |
+
- **GitHub**: [DimensioDepth Repository](https://github.com/chromahubz/dimensiodepth)
|
| 180 |
+
- **Original Web App**: Full-featured web application with 3D viewer and video export
|
| 181 |
+
- **Issues**: Report bugs on GitHub Issues
|
| 182 |
+
|
| 183 |
+
---
|
| 184 |
+
|
| 185 |
+
**Made with β€οΈ for the AI community**
|
| 186 |
+
|
| 187 |
+
*Transform your 2D world into 3D magic! π¨β¨*
|
|
@@ -0,0 +1,480 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
DimensioDepth - Add Dimension to Everything
|
| 3 |
+
Advanced AI Depth Estimation with 3D Visualization & Video Export
|
| 4 |
+
|
| 5 |
+
Powered by Depth-Anything V2 | Runs on Hugging Face Spaces
|
| 6 |
+
"""
|
| 7 |
+
|
| 8 |
+
import gradio as gr
|
| 9 |
+
import numpy as np
|
| 10 |
+
import cv2
|
| 11 |
+
from PIL import Image
|
| 12 |
+
import io
|
| 13 |
+
import base64
|
| 14 |
+
from pathlib import Path
|
| 15 |
+
import sys
|
| 16 |
+
|
| 17 |
+
# Add backend to path
|
| 18 |
+
sys.path.append(str(Path(__file__).parent / "backend"))
|
| 19 |
+
|
| 20 |
+
# Import backend utilities
|
| 21 |
+
from backend.utils.demo_depth import generate_smart_depth
|
| 22 |
+
from backend.utils.image_processing import (
|
| 23 |
+
load_image_from_bytes,
|
| 24 |
+
depth_to_colormap,
|
| 25 |
+
array_to_base64,
|
| 26 |
+
create_side_by_side
|
| 27 |
+
)
|
| 28 |
+
|
| 29 |
+
# Try to import model loader (may not be available in demo mode)
|
| 30 |
+
try:
|
| 31 |
+
from backend.utils.model_loader import ModelManager
|
| 32 |
+
from huggingface_hub import hf_hub_download
|
| 33 |
+
MODEL_AVAILABLE = True
|
| 34 |
+
except Exception as e:
|
| 35 |
+
MODEL_AVAILABLE = False
|
| 36 |
+
print(f"[!] Model loader not available - running in DEMO MODE: {e}")
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def download_models_from_hf():
|
| 40 |
+
"""Auto-download Depth-Anything V2 models from Hugging Face on startup"""
|
| 41 |
+
print("[*] Checking for Depth-Anything V2 models...")
|
| 42 |
+
|
| 43 |
+
model_cache_dir = Path(__file__).parent / "backend" / "models" / "cache"
|
| 44 |
+
model_cache_dir.mkdir(parents=True, exist_ok=True)
|
| 45 |
+
|
| 46 |
+
# Model configurations
|
| 47 |
+
models_to_download = {
|
| 48 |
+
"small": {
|
| 49 |
+
"repo_id": "depth-anything/Depth-Anything-V2-Small",
|
| 50 |
+
"filename": "depth_anything_v2_vits.onnx",
|
| 51 |
+
"size": "~94MB"
|
| 52 |
+
},
|
| 53 |
+
# Optionally include large model (comment out if too big)
|
| 54 |
+
# "large": {
|
| 55 |
+
# "repo_id": "depth-anything/Depth-Anything-V2-Large",
|
| 56 |
+
# "filename": "depth_anything_v2_vitl.onnx",
|
| 57 |
+
# "size": "~1.3GB"
|
| 58 |
+
# }
|
| 59 |
+
}
|
| 60 |
+
|
| 61 |
+
downloaded_models = {}
|
| 62 |
+
|
| 63 |
+
for model_name, config in models_to_download.items():
|
| 64 |
+
local_path = model_cache_dir / config["filename"]
|
| 65 |
+
|
| 66 |
+
if local_path.exists():
|
| 67 |
+
print(f"[+] {model_name.upper()} model already exists: {local_path}")
|
| 68 |
+
downloaded_models[model_name] = str(local_path)
|
| 69 |
+
else:
|
| 70 |
+
try:
|
| 71 |
+
print(f"[*] Downloading {model_name.upper()} model ({config['size']})...")
|
| 72 |
+
print(f" From: {config['repo_id']}")
|
| 73 |
+
|
| 74 |
+
# Download from Hugging Face Hub
|
| 75 |
+
model_path = hf_hub_download(
|
| 76 |
+
repo_id=config["repo_id"],
|
| 77 |
+
filename=config["filename"],
|
| 78 |
+
cache_dir=str(model_cache_dir)
|
| 79 |
+
)
|
| 80 |
+
|
| 81 |
+
print(f"[+] {model_name.upper()} model downloaded successfully!")
|
| 82 |
+
downloaded_models[model_name] = model_path
|
| 83 |
+
|
| 84 |
+
except Exception as e:
|
| 85 |
+
print(f"[!] Failed to download {model_name} model: {e}")
|
| 86 |
+
print(f" Will use DEMO MODE for {model_name} requests")
|
| 87 |
+
|
| 88 |
+
return downloaded_models
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
# Initialize model manager if available
|
| 92 |
+
model_manager = None
|
| 93 |
+
if MODEL_AVAILABLE:
|
| 94 |
+
model_manager = ModelManager()
|
| 95 |
+
try:
|
| 96 |
+
# Auto-download models from Hugging Face
|
| 97 |
+
downloaded_models = download_models_from_hf()
|
| 98 |
+
|
| 99 |
+
# Load each downloaded model
|
| 100 |
+
for model_name, model_path in downloaded_models.items():
|
| 101 |
+
try:
|
| 102 |
+
model_manager.load_model(
|
| 103 |
+
model_name,
|
| 104 |
+
model_path,
|
| 105 |
+
use_gpu=True,
|
| 106 |
+
use_tensorrt=False # Disable TensorRT for HF Spaces compatibility
|
| 107 |
+
)
|
| 108 |
+
print(f"[+] {model_name.upper()} model loaded into inference engine")
|
| 109 |
+
except Exception as e:
|
| 110 |
+
print(f"[!] Could not load {model_name} model: {e}")
|
| 111 |
+
|
| 112 |
+
if not model_manager.models:
|
| 113 |
+
print("[!] No models loaded - falling back to DEMO MODE")
|
| 114 |
+
MODEL_AVAILABLE = False
|
| 115 |
+
|
| 116 |
+
except Exception as e:
|
| 117 |
+
print(f"[!] Error during model initialization: {e}")
|
| 118 |
+
MODEL_AVAILABLE = False
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
def estimate_depth(image, quality_mode="Fast (Preview)", colormap_style="Inferno"):
|
| 122 |
+
"""
|
| 123 |
+
Estimate depth from an input image
|
| 124 |
+
|
| 125 |
+
Args:
|
| 126 |
+
image: PIL Image or numpy array
|
| 127 |
+
quality_mode: "Fast (Preview)" or "High Quality"
|
| 128 |
+
colormap_style: Color scheme for depth visualization
|
| 129 |
+
|
| 130 |
+
Returns:
|
| 131 |
+
tuple: (depth_colored, depth_grayscale, processing_info)
|
| 132 |
+
"""
|
| 133 |
+
try:
|
| 134 |
+
# Convert PIL to numpy if needed
|
| 135 |
+
if isinstance(image, Image.Image):
|
| 136 |
+
image = np.array(image)
|
| 137 |
+
|
| 138 |
+
# Check if we should use model or demo mode
|
| 139 |
+
use_demo = not MODEL_AVAILABLE
|
| 140 |
+
if MODEL_AVAILABLE and model_manager:
|
| 141 |
+
model_name = "small" if quality_mode == "Fast (Preview)" else "large"
|
| 142 |
+
model = model_manager.get_model(model_name)
|
| 143 |
+
if model is None:
|
| 144 |
+
use_demo = True
|
| 145 |
+
else:
|
| 146 |
+
use_demo = True
|
| 147 |
+
|
| 148 |
+
# Generate depth map
|
| 149 |
+
if use_demo:
|
| 150 |
+
depth = generate_smart_depth(image)
|
| 151 |
+
model_info = "DEMO MODE (Synthetic Depth)"
|
| 152 |
+
else:
|
| 153 |
+
depth = model.predict(image)
|
| 154 |
+
model_info = f"AI Model: {model_name.upper()}"
|
| 155 |
+
|
| 156 |
+
# Convert colormap style to cv2 constant
|
| 157 |
+
colormap_dict = {
|
| 158 |
+
"Inferno": cv2.COLORMAP_INFERNO,
|
| 159 |
+
"Viridis": cv2.COLORMAP_VIRIDIS,
|
| 160 |
+
"Plasma": cv2.COLORMAP_PLASMA,
|
| 161 |
+
"Turbo": cv2.COLORMAP_TURBO,
|
| 162 |
+
"Magma": cv2.COLORMAP_MAGMA,
|
| 163 |
+
"Hot": cv2.COLORMAP_HOT,
|
| 164 |
+
"Ocean": cv2.COLORMAP_OCEAN,
|
| 165 |
+
"Rainbow": cv2.COLORMAP_RAINBOW
|
| 166 |
+
}
|
| 167 |
+
|
| 168 |
+
# Create colored depth map
|
| 169 |
+
depth_colored = depth_to_colormap(depth, colormap_dict[colormap_style])
|
| 170 |
+
|
| 171 |
+
# Create grayscale depth map
|
| 172 |
+
depth_gray = (depth * 255).astype(np.uint8)
|
| 173 |
+
depth_gray = cv2.cvtColor(depth_gray, cv2.COLOR_GRAY2RGB)
|
| 174 |
+
|
| 175 |
+
# Processing info
|
| 176 |
+
info = f"""
|
| 177 |
+
### Depth Estimation Results
|
| 178 |
+
|
| 179 |
+
**Model Used:** {model_info}
|
| 180 |
+
**Input Size:** {image.shape[1]}x{image.shape[0]}
|
| 181 |
+
**Output Size:** {depth.shape[1]}x{depth.shape[0]}
|
| 182 |
+
**Colormap:** {colormap_style}
|
| 183 |
+
**Quality Mode:** {quality_mode}
|
| 184 |
+
|
| 185 |
+
β
Depth estimation complete!
|
| 186 |
+
"""
|
| 187 |
+
|
| 188 |
+
return depth_colored, depth_gray, info
|
| 189 |
+
|
| 190 |
+
except Exception as e:
|
| 191 |
+
error_msg = f"Error during depth estimation: {str(e)}"
|
| 192 |
+
print(error_msg)
|
| 193 |
+
return None, None, error_msg
|
| 194 |
+
|
| 195 |
+
|
| 196 |
+
def create_side_by_side_comparison(image, quality_mode="Fast (Preview)", colormap_style="Inferno"):
|
| 197 |
+
"""Create side-by-side comparison of original and depth map"""
|
| 198 |
+
try:
|
| 199 |
+
if isinstance(image, Image.Image):
|
| 200 |
+
image = np.array(image)
|
| 201 |
+
|
| 202 |
+
# Get depth estimation
|
| 203 |
+
use_demo = not MODEL_AVAILABLE or model_manager is None
|
| 204 |
+
if not use_demo:
|
| 205 |
+
model_name = "small" if quality_mode == "Fast (Preview)" else "large"
|
| 206 |
+
model = model_manager.get_model(model_name)
|
| 207 |
+
if model is None:
|
| 208 |
+
use_demo = True
|
| 209 |
+
|
| 210 |
+
if use_demo:
|
| 211 |
+
depth = generate_smart_depth(image)
|
| 212 |
+
else:
|
| 213 |
+
depth = model.predict(image)
|
| 214 |
+
|
| 215 |
+
# Convert colormap
|
| 216 |
+
colormap_dict = {
|
| 217 |
+
"Inferno": cv2.COLORMAP_INFERNO,
|
| 218 |
+
"Viridis": cv2.COLORMAP_VIRIDIS,
|
| 219 |
+
"Plasma": cv2.COLORMAP_PLASMA,
|
| 220 |
+
"Turbo": cv2.COLORMAP_TURBO,
|
| 221 |
+
"Magma": cv2.COLORMAP_MAGMA,
|
| 222 |
+
"Hot": cv2.COLORMAP_HOT,
|
| 223 |
+
"Ocean": cv2.COLORMAP_OCEAN,
|
| 224 |
+
"Rainbow": cv2.COLORMAP_RAINBOW
|
| 225 |
+
}
|
| 226 |
+
|
| 227 |
+
# Create side-by-side
|
| 228 |
+
comparison = create_side_by_side(image, depth, colormap=colormap_dict[colormap_style])
|
| 229 |
+
|
| 230 |
+
return comparison
|
| 231 |
+
|
| 232 |
+
except Exception as e:
|
| 233 |
+
print(f"Error creating comparison: {e}")
|
| 234 |
+
return None
|
| 235 |
+
|
| 236 |
+
|
| 237 |
+
def create_3d_visualization(image, depth_map, parallax_strength=0.5):
|
| 238 |
+
"""
|
| 239 |
+
Create a simple 3D displacement visualization
|
| 240 |
+
"""
|
| 241 |
+
try:
|
| 242 |
+
if isinstance(image, Image.Image):
|
| 243 |
+
image = np.array(image)
|
| 244 |
+
if isinstance(depth_map, Image.Image):
|
| 245 |
+
depth_map = np.array(depth_map)
|
| 246 |
+
|
| 247 |
+
# Convert depth to grayscale if colored
|
| 248 |
+
if len(depth_map.shape) == 3:
|
| 249 |
+
depth_map = cv2.cvtColor(depth_map, cv2.COLOR_RGB2GRAY)
|
| 250 |
+
|
| 251 |
+
# Normalize depth
|
| 252 |
+
depth_norm = depth_map.astype(float) / 255.0
|
| 253 |
+
|
| 254 |
+
# Create parallax effect (simple x-shift based on depth)
|
| 255 |
+
h, w = image.shape[:2]
|
| 256 |
+
result = image.copy()
|
| 257 |
+
|
| 258 |
+
# Apply horizontal shift based on depth
|
| 259 |
+
shift_amount = int(w * parallax_strength * 0.05)
|
| 260 |
+
|
| 261 |
+
for y in range(h):
|
| 262 |
+
for x in range(w):
|
| 263 |
+
depth_val = depth_norm[y, x]
|
| 264 |
+
shift = int(shift_amount * depth_val)
|
| 265 |
+
new_x = min(max(x + shift, 0), w - 1)
|
| 266 |
+
result[y, new_x] = image[y, x]
|
| 267 |
+
|
| 268 |
+
return result
|
| 269 |
+
|
| 270 |
+
except Exception as e:
|
| 271 |
+
print(f"Error creating 3D viz: {e}")
|
| 272 |
+
return image
|
| 273 |
+
|
| 274 |
+
|
| 275 |
+
# Create Gradio interface
|
| 276 |
+
with gr.Blocks(
|
| 277 |
+
theme=gr.themes.Soft(primary_hue="blue", secondary_hue="purple"),
|
| 278 |
+
title="DimensioDepth - Add Dimension to Everything"
|
| 279 |
+
) as demo:
|
| 280 |
+
|
| 281 |
+
gr.Markdown("""
|
| 282 |
+
# π¨ DimensioDepth - Add Dimension to Everything
|
| 283 |
+
|
| 284 |
+
### Transform 2D images into stunning 3D depth visualizations with AI
|
| 285 |
+
|
| 286 |
+
Powered by **Depth-Anything V2** | Advanced depth estimation with cinematic effects
|
| 287 |
+
|
| 288 |
+
---
|
| 289 |
+
""")
|
| 290 |
+
|
| 291 |
+
with gr.Tabs():
|
| 292 |
+
# Tab 1: Main Depth Estimation
|
| 293 |
+
with gr.Tab("π― Depth Estimation"):
|
| 294 |
+
with gr.Row():
|
| 295 |
+
with gr.Column(scale=1):
|
| 296 |
+
input_image = gr.Image(
|
| 297 |
+
label="Upload Your Image",
|
| 298 |
+
type="pil",
|
| 299 |
+
height=400
|
| 300 |
+
)
|
| 301 |
+
|
| 302 |
+
with gr.Row():
|
| 303 |
+
quality_mode = gr.Radio(
|
| 304 |
+
choices=["Fast (Preview)", "High Quality"],
|
| 305 |
+
value="Fast (Preview)",
|
| 306 |
+
label="Quality Mode",
|
| 307 |
+
info="Fast for real-time, High Quality for best results"
|
| 308 |
+
)
|
| 309 |
+
|
| 310 |
+
colormap_style = gr.Dropdown(
|
| 311 |
+
choices=["Inferno", "Viridis", "Plasma", "Turbo", "Magma", "Hot", "Ocean", "Rainbow"],
|
| 312 |
+
value="Inferno",
|
| 313 |
+
label="Colormap Style",
|
| 314 |
+
info="Choose your depth visualization color scheme"
|
| 315 |
+
)
|
| 316 |
+
|
| 317 |
+
estimate_btn = gr.Button("π Generate Depth Map", variant="primary", size="lg")
|
| 318 |
+
|
| 319 |
+
with gr.Column(scale=1):
|
| 320 |
+
depth_colored = gr.Image(label="Depth Map (Colored)", height=400)
|
| 321 |
+
depth_gray = gr.Image(label="Depth Map (Grayscale)", height=400)
|
| 322 |
+
|
| 323 |
+
processing_info = gr.Markdown()
|
| 324 |
+
|
| 325 |
+
estimate_btn.click(
|
| 326 |
+
fn=estimate_depth,
|
| 327 |
+
inputs=[input_image, quality_mode, colormap_style],
|
| 328 |
+
outputs=[depth_colored, depth_gray, processing_info]
|
| 329 |
+
)
|
| 330 |
+
|
| 331 |
+
# Tab 2: Side-by-Side Comparison
|
| 332 |
+
with gr.Tab("π Side-by-Side Comparison"):
|
| 333 |
+
gr.Markdown("""
|
| 334 |
+
### Compare Original Image with Depth Map
|
| 335 |
+
Perfect for analyzing depth estimation quality and understanding 3D structure.
|
| 336 |
+
""")
|
| 337 |
+
|
| 338 |
+
with gr.Row():
|
| 339 |
+
with gr.Column(scale=1):
|
| 340 |
+
compare_input = gr.Image(label="Upload Image", type="pil", height=400)
|
| 341 |
+
|
| 342 |
+
compare_quality = gr.Radio(
|
| 343 |
+
choices=["Fast (Preview)", "High Quality"],
|
| 344 |
+
value="Fast (Preview)",
|
| 345 |
+
label="Quality Mode"
|
| 346 |
+
)
|
| 347 |
+
|
| 348 |
+
compare_colormap = gr.Dropdown(
|
| 349 |
+
choices=["Inferno", "Viridis", "Plasma", "Turbo", "Magma", "Hot", "Ocean", "Rainbow"],
|
| 350 |
+
value="Turbo",
|
| 351 |
+
label="Colormap"
|
| 352 |
+
)
|
| 353 |
+
|
| 354 |
+
compare_btn = gr.Button("π¬ Create Comparison", variant="primary")
|
| 355 |
+
|
| 356 |
+
with gr.Column(scale=1):
|
| 357 |
+
comparison_output = gr.Image(label="Side-by-Side Comparison", height=500)
|
| 358 |
+
|
| 359 |
+
compare_btn.click(
|
| 360 |
+
fn=create_side_by_side_comparison,
|
| 361 |
+
inputs=[compare_input, compare_quality, compare_colormap],
|
| 362 |
+
outputs=comparison_output
|
| 363 |
+
)
|
| 364 |
+
|
| 365 |
+
# Tab 3: 3D Parallax Effect
|
| 366 |
+
with gr.Tab("π 3D Parallax Effect"):
|
| 367 |
+
gr.Markdown("""
|
| 368 |
+
### Create 3D Depth Displacement Effect
|
| 369 |
+
Generate a parallax effect to visualize the 3D structure of your image.
|
| 370 |
+
""")
|
| 371 |
+
|
| 372 |
+
with gr.Row():
|
| 373 |
+
with gr.Column(scale=1):
|
| 374 |
+
parallax_input = gr.Image(label="Original Image", type="pil")
|
| 375 |
+
parallax_depth = gr.Image(label="Depth Map (from previous tab)", type="pil")
|
| 376 |
+
parallax_strength = gr.Slider(
|
| 377 |
+
minimum=0, maximum=2, value=0.5, step=0.1,
|
| 378 |
+
label="Parallax Strength",
|
| 379 |
+
info="Control the 3D displacement effect intensity"
|
| 380 |
+
)
|
| 381 |
+
parallax_btn = gr.Button("β¨ Generate 3D Effect", variant="primary")
|
| 382 |
+
|
| 383 |
+
with gr.Column(scale=1):
|
| 384 |
+
parallax_output = gr.Image(label="3D Parallax Result", height=500)
|
| 385 |
+
|
| 386 |
+
parallax_btn.click(
|
| 387 |
+
fn=create_3d_visualization,
|
| 388 |
+
inputs=[parallax_input, parallax_depth, parallax_strength],
|
| 389 |
+
outputs=parallax_output
|
| 390 |
+
)
|
| 391 |
+
|
| 392 |
+
# Tab 4: Batch Processing
|
| 393 |
+
with gr.Tab("π¦ Batch Processing"):
|
| 394 |
+
gr.Markdown("""
|
| 395 |
+
### Process Multiple Images
|
| 396 |
+
Upload multiple images and generate depth maps for all of them at once.
|
| 397 |
+
""")
|
| 398 |
+
|
| 399 |
+
batch_input = gr.Files(label="Upload Multiple Images", file_types=["image"])
|
| 400 |
+
batch_quality = gr.Radio(
|
| 401 |
+
choices=["Fast (Preview)", "High Quality"],
|
| 402 |
+
value="Fast (Preview)",
|
| 403 |
+
label="Quality Mode"
|
| 404 |
+
)
|
| 405 |
+
batch_colormap = gr.Dropdown(
|
| 406 |
+
choices=["Inferno", "Viridis", "Plasma", "Turbo"],
|
| 407 |
+
value="Inferno",
|
| 408 |
+
label="Colormap"
|
| 409 |
+
)
|
| 410 |
+
batch_btn = gr.Button("π Process Batch", variant="primary")
|
| 411 |
+
batch_gallery = gr.Gallery(label="Batch Results", columns=3, height=600)
|
| 412 |
+
|
| 413 |
+
# Examples section
|
| 414 |
+
gr.Markdown("---")
|
| 415 |
+
gr.Markdown("""
|
| 416 |
+
## π‘ Tips for Best Results
|
| 417 |
+
|
| 418 |
+
- **Fast Mode**: Great for real-time preview and testing (~50-100ms)
|
| 419 |
+
- **High Quality Mode**: Best depth accuracy, slower processing (~500-1500ms)
|
| 420 |
+
- **Colormap**: Choose based on your preference - Inferno (default), Viridis, Plasma, etc.
|
| 421 |
+
- **3D Effect**: Increase parallax strength for more dramatic depth displacement
|
| 422 |
+
|
| 423 |
+
### Current Status
|
| 424 |
+
""")
|
| 425 |
+
|
| 426 |
+
if MODEL_AVAILABLE and model_manager and model_manager.models:
|
| 427 |
+
model_list = ', '.join(model_manager.models.keys()).upper()
|
| 428 |
+
status_text = f"""
|
| 429 |
+
### β
AI Models Status
|
| 430 |
+
|
| 431 |
+
**Loaded Models**: {model_list}
|
| 432 |
+
**GPU Acceleration**: Enabled
|
| 433 |
+
**Mode**: Full AI Depth Estimation
|
| 434 |
+
|
| 435 |
+
You're running with real Depth-Anything V2 models! π
|
| 436 |
+
"""
|
| 437 |
+
else:
|
| 438 |
+
status_text = """
|
| 439 |
+
### π¨ Demo Mode Active
|
| 440 |
+
|
| 441 |
+
**Status**: Running with Synthetic Depth Generation
|
| 442 |
+
**Speed**: Ultra-fast (<50ms per image)
|
| 443 |
+
**Quality**: Surprisingly good! Uses advanced edge detection + intensity analysis
|
| 444 |
+
|
| 445 |
+
**Demo Mode Features**:
|
| 446 |
+
- β
Works instantly (no model downloads)
|
| 447 |
+
- β
Fast processing
|
| 448 |
+
- β
Good quality for most use cases
|
| 449 |
+
- β
Perfect for testing and demos
|
| 450 |
+
|
| 451 |
+
*Try it out - you might be surprised by the quality!* π
|
| 452 |
+
"""
|
| 453 |
+
|
| 454 |
+
gr.Markdown(status_text)
|
| 455 |
+
|
| 456 |
+
gr.Markdown("""
|
| 457 |
+
---
|
| 458 |
+
|
| 459 |
+
### About DimensioDepth
|
| 460 |
+
|
| 461 |
+
DimensioDepth transforms 2D images into stunning 3D depth visualizations using state-of-the-art AI depth estimation.
|
| 462 |
+
Perfect for:
|
| 463 |
+
- 3D artists and VFX professionals
|
| 464 |
+
- Computer vision researchers
|
| 465 |
+
- Content creators and photographers
|
| 466 |
+
- Anyone interested in depth perception!
|
| 467 |
+
|
| 468 |
+
**Tech Stack**: Depth-Anything V2, ONNX Runtime, FastAPI, Gradio
|
| 469 |
+
|
| 470 |
+
Made with β€οΈ for the AI community
|
| 471 |
+
""")
|
| 472 |
+
|
| 473 |
+
|
| 474 |
+
# Launch the app
|
| 475 |
+
if __name__ == "__main__":
|
| 476 |
+
demo.launch(
|
| 477 |
+
server_name="0.0.0.0",
|
| 478 |
+
server_port=7860,
|
| 479 |
+
share=False
|
| 480 |
+
)
|
|
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Server Configuration
|
| 2 |
+
HOST=0.0.0.0
|
| 3 |
+
PORT=8000
|
| 4 |
+
DEBUG=True
|
| 5 |
+
WORKERS=4
|
| 6 |
+
|
| 7 |
+
# Model Configuration
|
| 8 |
+
DEPTH_MODEL_SMALL=depth_anything_v2_vits.onnx
|
| 9 |
+
DEPTH_MODEL_LARGE=depth_anything_v2_vitl.onnx
|
| 10 |
+
MODEL_CACHE_DIR=./models/cache
|
| 11 |
+
|
| 12 |
+
# Redis Configuration
|
| 13 |
+
REDIS_HOST=localhost
|
| 14 |
+
REDIS_PORT=6379
|
| 15 |
+
REDIS_DB=0
|
| 16 |
+
REDIS_PASSWORD=
|
| 17 |
+
|
| 18 |
+
# Cache Settings
|
| 19 |
+
ENABLE_CACHE=True
|
| 20 |
+
CACHE_TTL=3600
|
| 21 |
+
|
| 22 |
+
# Processing Configuration
|
| 23 |
+
MAX_IMAGE_SIZE=4096
|
| 24 |
+
DEFAULT_IMAGE_SIZE=1024
|
| 25 |
+
PREVIEW_SIZE=384
|
| 26 |
+
|
| 27 |
+
# GPU Configuration
|
| 28 |
+
CUDA_VISIBLE_DEVICES=0
|
| 29 |
+
USE_GPU=True
|
| 30 |
+
TRT_OPTIMIZATION=True
|
| 31 |
+
|
| 32 |
+
# Storage (optional)
|
| 33 |
+
S3_BUCKET=
|
| 34 |
+
S3_REGION=us-east-1
|
| 35 |
+
AWS_ACCESS_KEY_ID=
|
| 36 |
+
AWS_SECRET_ACCESS_KEY=
|
| 37 |
+
|
| 38 |
+
# API Settings
|
| 39 |
+
CORS_ORIGINS=["http://localhost:3000","http://localhost:5173"]
|
| 40 |
+
MAX_UPLOAD_SIZE=10485760
|
| 41 |
+
RATE_LIMIT_PER_MINUTE=60
|
|
@@ -0,0 +1,394 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from fastapi import FastAPI, UploadFile, File, HTTPException, WebSocket, WebSocketDisconnect
|
| 2 |
+
from fastapi.middleware.cors import CORSMiddleware
|
| 3 |
+
from fastapi.responses import JSONResponse, StreamingResponse
|
| 4 |
+
from pydantic import BaseModel
|
| 5 |
+
from typing import Optional, Literal
|
| 6 |
+
import asyncio
|
| 7 |
+
import time
|
| 8 |
+
import hashlib
|
| 9 |
+
import io
|
| 10 |
+
|
| 11 |
+
# Import our utilities
|
| 12 |
+
import sys
|
| 13 |
+
from pathlib import Path
|
| 14 |
+
sys.path.append(str(Path(__file__).parent.parent))
|
| 15 |
+
|
| 16 |
+
from config import get_settings
|
| 17 |
+
from utils.model_loader import ModelManager
|
| 18 |
+
from utils.image_processing import (
|
| 19 |
+
load_image_from_bytes,
|
| 20 |
+
load_image_from_base64,
|
| 21 |
+
array_to_base64,
|
| 22 |
+
depth_to_colormap,
|
| 23 |
+
create_side_by_side
|
| 24 |
+
)
|
| 25 |
+
from utils.demo_depth import generate_smart_depth
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
# Initialize FastAPI app
|
| 29 |
+
app = FastAPI(
|
| 30 |
+
title="Dimensio API",
|
| 31 |
+
description="Add Dimension to Everything - High-performance depth estimation and 3D visualization API",
|
| 32 |
+
version="1.0.0"
|
| 33 |
+
)
|
| 34 |
+
|
| 35 |
+
settings = get_settings()
|
| 36 |
+
|
| 37 |
+
# CORS middleware
|
| 38 |
+
app.add_middleware(
|
| 39 |
+
CORSMiddleware,
|
| 40 |
+
allow_origins=settings.CORS_ORIGINS,
|
| 41 |
+
allow_credentials=True,
|
| 42 |
+
allow_methods=["*"],
|
| 43 |
+
allow_headers=["*"],
|
| 44 |
+
)
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
# Global model manager
|
| 48 |
+
model_manager = ModelManager()
|
| 49 |
+
DEMO_MODE = False # Will be set to True if no models available
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
# Request/Response models
|
| 53 |
+
class DepthRequest(BaseModel):
|
| 54 |
+
"""Request model for depth estimation"""
|
| 55 |
+
image: str # Base64 encoded image
|
| 56 |
+
model: Literal["small", "large"] = "small"
|
| 57 |
+
output_format: Literal["grayscale", "colormap", "both"] = "colormap"
|
| 58 |
+
colormap: Literal["inferno", "viridis", "plasma", "turbo"] = "inferno"
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
class DepthResponse(BaseModel):
|
| 62 |
+
"""Response model for depth estimation"""
|
| 63 |
+
depth_map: str # Base64 encoded depth map
|
| 64 |
+
metadata: dict
|
| 65 |
+
processing_time_ms: float
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
# Startup/shutdown events
|
| 69 |
+
@app.on_event("startup")
|
| 70 |
+
async def startup_event():
|
| 71 |
+
"""Initialize models on startup"""
|
| 72 |
+
print(">> Starting Dimensio API...")
|
| 73 |
+
|
| 74 |
+
try:
|
| 75 |
+
# Load small model (fast preview)
|
| 76 |
+
small_model_path = Path(settings.MODEL_CACHE_DIR) / settings.DEPTH_MODEL_SMALL
|
| 77 |
+
if small_model_path.exists():
|
| 78 |
+
model_manager.load_model(
|
| 79 |
+
"small",
|
| 80 |
+
str(small_model_path),
|
| 81 |
+
use_gpu=settings.USE_GPU,
|
| 82 |
+
use_tensorrt=settings.TRT_OPTIMIZATION
|
| 83 |
+
)
|
| 84 |
+
print("[+] Small model loaded")
|
| 85 |
+
else:
|
| 86 |
+
print(f"[!] Small model not found: {small_model_path}")
|
| 87 |
+
|
| 88 |
+
# Load large model (high quality)
|
| 89 |
+
large_model_path = Path(settings.MODEL_CACHE_DIR) / settings.DEPTH_MODEL_LARGE
|
| 90 |
+
if large_model_path.exists():
|
| 91 |
+
model_manager.load_model(
|
| 92 |
+
"large",
|
| 93 |
+
str(large_model_path),
|
| 94 |
+
use_gpu=settings.USE_GPU,
|
| 95 |
+
use_tensorrt=settings.TRT_OPTIMIZATION
|
| 96 |
+
)
|
| 97 |
+
print("[+] Large model loaded")
|
| 98 |
+
else:
|
| 99 |
+
print(f"[!] Large model not found: {large_model_path}")
|
| 100 |
+
|
| 101 |
+
if not model_manager.models:
|
| 102 |
+
global DEMO_MODE
|
| 103 |
+
DEMO_MODE = True
|
| 104 |
+
print("\n[!] No models loaded - Running in DEMO MODE")
|
| 105 |
+
print("Demo mode uses synthetic depth maps for testing the UI.")
|
| 106 |
+
print("\nTo use real AI models:")
|
| 107 |
+
print("1. Run: python download_models.py")
|
| 108 |
+
print("2. Place ONNX models in models/cache/")
|
| 109 |
+
print("3. Restart the server")
|
| 110 |
+
|
| 111 |
+
except Exception as e:
|
| 112 |
+
print(f"[X] Error loading models: {e}")
|
| 113 |
+
print("Server will start but depth estimation will not work.")
|
| 114 |
+
|
| 115 |
+
|
| 116 |
+
@app.on_event("shutdown")
|
| 117 |
+
async def shutdown_event():
|
| 118 |
+
"""Cleanup on shutdown"""
|
| 119 |
+
print(">> Shutting down Depth Flow Pro API...")
|
| 120 |
+
|
| 121 |
+
|
| 122 |
+
# Health check
|
| 123 |
+
@app.get("/")
|
| 124 |
+
async def root():
|
| 125 |
+
"""API health check"""
|
| 126 |
+
return {
|
| 127 |
+
"name": "Depth Flow Pro API",
|
| 128 |
+
"version": "1.0.0",
|
| 129 |
+
"status": "online",
|
| 130 |
+
"models_loaded": list(model_manager.models.keys())
|
| 131 |
+
}
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
@app.get("/health")
|
| 135 |
+
async def health_check():
|
| 136 |
+
"""Detailed health check"""
|
| 137 |
+
return {
|
| 138 |
+
"status": "healthy",
|
| 139 |
+
"models": {
|
| 140 |
+
name: "loaded" for name in model_manager.models.keys()
|
| 141 |
+
},
|
| 142 |
+
"gpu_enabled": settings.USE_GPU,
|
| 143 |
+
"tensorrt_enabled": settings.TRT_OPTIMIZATION
|
| 144 |
+
}
|
| 145 |
+
|
| 146 |
+
|
| 147 |
+
# Depth estimation endpoints
|
| 148 |
+
@app.post("/api/v1/depth/preview", response_model=DepthResponse)
|
| 149 |
+
async def estimate_depth_preview(file: UploadFile = File(...)):
|
| 150 |
+
"""
|
| 151 |
+
Fast depth estimation using small model (preview quality)
|
| 152 |
+
Optimized for speed, ~50-100ms on GPU
|
| 153 |
+
"""
|
| 154 |
+
try:
|
| 155 |
+
start_time = time.time()
|
| 156 |
+
|
| 157 |
+
# Load image
|
| 158 |
+
image_bytes = await file.read()
|
| 159 |
+
image = load_image_from_bytes(image_bytes)
|
| 160 |
+
|
| 161 |
+
# Check if demo mode or use real model
|
| 162 |
+
if DEMO_MODE:
|
| 163 |
+
# Use synthetic depth for demo
|
| 164 |
+
depth = generate_smart_depth(image)
|
| 165 |
+
model_name = "demo"
|
| 166 |
+
else:
|
| 167 |
+
# Get small model
|
| 168 |
+
model = model_manager.get_model("small")
|
| 169 |
+
if model is None:
|
| 170 |
+
raise HTTPException(
|
| 171 |
+
status_code=503,
|
| 172 |
+
detail="Small model not loaded. Please check server logs."
|
| 173 |
+
)
|
| 174 |
+
# Run depth estimation
|
| 175 |
+
depth = model.predict(image)
|
| 176 |
+
model_name = "small"
|
| 177 |
+
|
| 178 |
+
# Convert to colormap
|
| 179 |
+
depth_colored = depth_to_colormap(depth)
|
| 180 |
+
|
| 181 |
+
# Encode to base64
|
| 182 |
+
depth_base64 = array_to_base64(depth_colored, format='PNG')
|
| 183 |
+
|
| 184 |
+
processing_time = (time.time() - start_time) * 1000
|
| 185 |
+
|
| 186 |
+
return DepthResponse(
|
| 187 |
+
depth_map=depth_base64,
|
| 188 |
+
metadata={
|
| 189 |
+
"model": model_name,
|
| 190 |
+
"input_size": image.shape[:2],
|
| 191 |
+
"output_size": depth.shape[:2],
|
| 192 |
+
"demo_mode": DEMO_MODE
|
| 193 |
+
},
|
| 194 |
+
processing_time_ms=round(processing_time, 2)
|
| 195 |
+
)
|
| 196 |
+
|
| 197 |
+
except Exception as e:
|
| 198 |
+
print(f"β Error: {type(e).__name__}: {str(e)}")
|
| 199 |
+
import traceback
|
| 200 |
+
traceback.print_exc()
|
| 201 |
+
raise HTTPException(status_code=500, detail=str(e))
|
| 202 |
+
|
| 203 |
+
|
| 204 |
+
@app.post("/api/v1/depth/hq", response_model=DepthResponse)
|
| 205 |
+
async def estimate_depth_hq(file: UploadFile = File(...)):
|
| 206 |
+
"""
|
| 207 |
+
High-quality depth estimation using large model
|
| 208 |
+
Slower but more accurate, ~500-1500ms on GPU
|
| 209 |
+
"""
|
| 210 |
+
try:
|
| 211 |
+
start_time = time.time()
|
| 212 |
+
|
| 213 |
+
# Load image
|
| 214 |
+
image_bytes = await file.read()
|
| 215 |
+
image = load_image_from_bytes(image_bytes)
|
| 216 |
+
|
| 217 |
+
# Check if demo mode or use real model
|
| 218 |
+
if DEMO_MODE:
|
| 219 |
+
# Use synthetic depth for demo
|
| 220 |
+
depth = generate_smart_depth(image)
|
| 221 |
+
model_name = "demo (HQ)"
|
| 222 |
+
else:
|
| 223 |
+
# Get large model
|
| 224 |
+
model = model_manager.get_model("large")
|
| 225 |
+
if model is None:
|
| 226 |
+
# Fallback to small model if large not available
|
| 227 |
+
model = model_manager.get_model("small")
|
| 228 |
+
if model is None:
|
| 229 |
+
raise HTTPException(
|
| 230 |
+
status_code=503,
|
| 231 |
+
detail="No models loaded. Please check server logs."
|
| 232 |
+
)
|
| 233 |
+
model_name = "small (fallback)"
|
| 234 |
+
else:
|
| 235 |
+
model_name = "large"
|
| 236 |
+
|
| 237 |
+
# Run depth estimation
|
| 238 |
+
depth = model.predict(image)
|
| 239 |
+
|
| 240 |
+
# Convert to colormap
|
| 241 |
+
depth_colored = depth_to_colormap(depth)
|
| 242 |
+
|
| 243 |
+
# Encode to base64
|
| 244 |
+
depth_base64 = array_to_base64(depth_colored, format='PNG')
|
| 245 |
+
|
| 246 |
+
processing_time = (time.time() - start_time) * 1000
|
| 247 |
+
|
| 248 |
+
return DepthResponse(
|
| 249 |
+
depth_map=depth_base64,
|
| 250 |
+
metadata={
|
| 251 |
+
"model": model_name,
|
| 252 |
+
"input_size": image.shape[:2],
|
| 253 |
+
"output_size": depth.shape[:2],
|
| 254 |
+
"demo_mode": DEMO_MODE
|
| 255 |
+
},
|
| 256 |
+
processing_time_ms=round(processing_time, 2)
|
| 257 |
+
)
|
| 258 |
+
|
| 259 |
+
except Exception as e:
|
| 260 |
+
print(f"β Error: {type(e).__name__}: {str(e)}")
|
| 261 |
+
import traceback
|
| 262 |
+
traceback.print_exc()
|
| 263 |
+
raise HTTPException(status_code=500, detail=str(e))
|
| 264 |
+
|
| 265 |
+
|
| 266 |
+
@app.post("/api/v1/depth/estimate")
|
| 267 |
+
async def estimate_depth(request: DepthRequest):
|
| 268 |
+
"""
|
| 269 |
+
Depth estimation with custom options
|
| 270 |
+
Accepts base64 encoded image
|
| 271 |
+
"""
|
| 272 |
+
try:
|
| 273 |
+
start_time = time.time()
|
| 274 |
+
|
| 275 |
+
# Load image from base64
|
| 276 |
+
image = load_image_from_base64(request.image)
|
| 277 |
+
|
| 278 |
+
# Get model
|
| 279 |
+
model = model_manager.get_model(request.model)
|
| 280 |
+
if model is None:
|
| 281 |
+
raise HTTPException(
|
| 282 |
+
status_code=503,
|
| 283 |
+
detail=f"Model '{request.model}' not loaded"
|
| 284 |
+
)
|
| 285 |
+
|
| 286 |
+
# Run depth estimation
|
| 287 |
+
depth = model.predict(image)
|
| 288 |
+
|
| 289 |
+
# Process output based on format
|
| 290 |
+
if request.output_format == "grayscale":
|
| 291 |
+
output = (depth * 255).astype('uint8')
|
| 292 |
+
depth_base64 = array_to_base64(output, format='PNG')
|
| 293 |
+
elif request.output_format == "colormap":
|
| 294 |
+
import cv2
|
| 295 |
+
colormap_dict = {
|
| 296 |
+
"inferno": cv2.COLORMAP_INFERNO,
|
| 297 |
+
"viridis": cv2.COLORMAP_VIRIDIS,
|
| 298 |
+
"plasma": cv2.COLORMAP_PLASMA,
|
| 299 |
+
"turbo": cv2.COLORMAP_TURBO
|
| 300 |
+
}
|
| 301 |
+
depth_colored = depth_to_colormap(depth, colormap_dict[request.colormap])
|
| 302 |
+
depth_base64 = array_to_base64(depth_colored, format='PNG')
|
| 303 |
+
else: # both
|
| 304 |
+
side_by_side = create_side_by_side(image, depth, colormap=True)
|
| 305 |
+
depth_base64 = array_to_base64(side_by_side, format='PNG')
|
| 306 |
+
|
| 307 |
+
processing_time = (time.time() - start_time) * 1000
|
| 308 |
+
|
| 309 |
+
return DepthResponse(
|
| 310 |
+
depth_map=depth_base64,
|
| 311 |
+
metadata={
|
| 312 |
+
"model": request.model,
|
| 313 |
+
"output_format": request.output_format,
|
| 314 |
+
"colormap": request.colormap,
|
| 315 |
+
"input_size": image.shape[:2],
|
| 316 |
+
"output_size": depth.shape[:2]
|
| 317 |
+
},
|
| 318 |
+
processing_time_ms=round(processing_time, 2)
|
| 319 |
+
)
|
| 320 |
+
|
| 321 |
+
except Exception as e:
|
| 322 |
+
print(f"β Error: {type(e).__name__}: {str(e)}")
|
| 323 |
+
import traceback
|
| 324 |
+
traceback.print_exc()
|
| 325 |
+
raise HTTPException(status_code=500, detail=str(e))
|
| 326 |
+
|
| 327 |
+
|
| 328 |
+
# WebSocket for streaming
|
| 329 |
+
@app.websocket("/api/v1/stream")
|
| 330 |
+
async def websocket_endpoint(websocket: WebSocket):
|
| 331 |
+
"""
|
| 332 |
+
WebSocket endpoint for real-time depth estimation
|
| 333 |
+
Supports streaming multiple images
|
| 334 |
+
"""
|
| 335 |
+
await websocket.accept()
|
| 336 |
+
|
| 337 |
+
try:
|
| 338 |
+
while True:
|
| 339 |
+
# Receive image data
|
| 340 |
+
data = await websocket.receive_json()
|
| 341 |
+
|
| 342 |
+
if data.get("action") == "estimate":
|
| 343 |
+
start_time = time.time()
|
| 344 |
+
|
| 345 |
+
# Load image
|
| 346 |
+
image = load_image_from_base64(data["image"])
|
| 347 |
+
|
| 348 |
+
# Get model
|
| 349 |
+
model_name = data.get("model", "small")
|
| 350 |
+
model = model_manager.get_model(model_name)
|
| 351 |
+
|
| 352 |
+
if model is None:
|
| 353 |
+
await websocket.send_json({
|
| 354 |
+
"error": f"Model '{model_name}' not loaded"
|
| 355 |
+
})
|
| 356 |
+
continue
|
| 357 |
+
|
| 358 |
+
# Send progress update
|
| 359 |
+
await websocket.send_json({
|
| 360 |
+
"status": "processing",
|
| 361 |
+
"progress": 50
|
| 362 |
+
})
|
| 363 |
+
|
| 364 |
+
# Run depth estimation
|
| 365 |
+
depth = model.predict(image)
|
| 366 |
+
|
| 367 |
+
# Convert to colormap
|
| 368 |
+
depth_colored = depth_to_colormap(depth)
|
| 369 |
+
depth_base64 = array_to_base64(depth_colored, format='PNG')
|
| 370 |
+
|
| 371 |
+
processing_time = (time.time() - start_time) * 1000
|
| 372 |
+
|
| 373 |
+
# Send result
|
| 374 |
+
await websocket.send_json({
|
| 375 |
+
"status": "complete",
|
| 376 |
+
"depth_map": depth_base64,
|
| 377 |
+
"processing_time_ms": round(processing_time, 2)
|
| 378 |
+
})
|
| 379 |
+
|
| 380 |
+
except WebSocketDisconnect:
|
| 381 |
+
print("WebSocket disconnected")
|
| 382 |
+
except Exception as e:
|
| 383 |
+
print(f"WebSocket error: {e}")
|
| 384 |
+
await websocket.send_json({"error": str(e)})
|
| 385 |
+
|
| 386 |
+
|
| 387 |
+
if __name__ == "__main__":
|
| 388 |
+
import uvicorn
|
| 389 |
+
uvicorn.run(
|
| 390 |
+
"main:app",
|
| 391 |
+
host=settings.HOST,
|
| 392 |
+
port=settings.PORT,
|
| 393 |
+
reload=settings.DEBUG
|
| 394 |
+
)
|
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from pydantic_settings import BaseSettings
|
| 2 |
+
from typing import List
|
| 3 |
+
import os
|
| 4 |
+
|
| 5 |
+
|
| 6 |
+
class Settings(BaseSettings):
|
| 7 |
+
"""Application settings loaded from environment variables"""
|
| 8 |
+
|
| 9 |
+
# Server Configuration
|
| 10 |
+
HOST: str = "0.0.0.0"
|
| 11 |
+
PORT: int = 8000
|
| 12 |
+
DEBUG: bool = True
|
| 13 |
+
WORKERS: int = 4
|
| 14 |
+
|
| 15 |
+
# Model Configuration
|
| 16 |
+
DEPTH_MODEL_SMALL: str = "depth_anything_v2_vits.onnx"
|
| 17 |
+
DEPTH_MODEL_LARGE: str = "depth_anything_v2_vitl.onnx"
|
| 18 |
+
MODEL_CACHE_DIR: str = "./models/cache"
|
| 19 |
+
|
| 20 |
+
# Redis Configuration
|
| 21 |
+
REDIS_HOST: str = "localhost"
|
| 22 |
+
REDIS_PORT: int = 6379
|
| 23 |
+
REDIS_DB: int = 0
|
| 24 |
+
REDIS_PASSWORD: str = ""
|
| 25 |
+
|
| 26 |
+
# Cache Settings
|
| 27 |
+
ENABLE_CACHE: bool = True
|
| 28 |
+
CACHE_TTL: int = 3600
|
| 29 |
+
|
| 30 |
+
# Processing Configuration
|
| 31 |
+
MAX_IMAGE_SIZE: int = 4096
|
| 32 |
+
DEFAULT_IMAGE_SIZE: int = 1024
|
| 33 |
+
PREVIEW_SIZE: int = 384
|
| 34 |
+
|
| 35 |
+
# GPU Configuration
|
| 36 |
+
CUDA_VISIBLE_DEVICES: str = "0"
|
| 37 |
+
USE_GPU: bool = True
|
| 38 |
+
TRT_OPTIMIZATION: bool = True
|
| 39 |
+
|
| 40 |
+
# Storage (optional)
|
| 41 |
+
S3_BUCKET: str = ""
|
| 42 |
+
S3_REGION: str = "us-east-1"
|
| 43 |
+
AWS_ACCESS_KEY_ID: str = ""
|
| 44 |
+
AWS_SECRET_ACCESS_KEY: str = ""
|
| 45 |
+
|
| 46 |
+
# API Settings
|
| 47 |
+
CORS_ORIGINS: List[str] = ["http://localhost:3000", "http://localhost:5173"]
|
| 48 |
+
MAX_UPLOAD_SIZE: int = 10485760 # 10MB
|
| 49 |
+
RATE_LIMIT_PER_MINUTE: int = 60
|
| 50 |
+
|
| 51 |
+
class Config:
|
| 52 |
+
env_file = ".env"
|
| 53 |
+
case_sensitive = True
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
# Global settings instance
|
| 57 |
+
settings = Settings()
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
def get_settings() -> Settings:
|
| 61 |
+
"""Get application settings"""
|
| 62 |
+
return settings
|
|
@@ -0,0 +1,177 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
Download Depth-Anything V2 ONNX models from HuggingFace
|
| 4 |
+
|
| 5 |
+
This script downloads optimized ONNX versions of Depth-Anything V2 models
|
| 6 |
+
for fast inference without PyTorch dependency.
|
| 7 |
+
"""
|
| 8 |
+
|
| 9 |
+
import os
|
| 10 |
+
from pathlib import Path
|
| 11 |
+
from huggingface_hub import hf_hub_download
|
| 12 |
+
import sys
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
# Model configurations
|
| 16 |
+
MODELS = {
|
| 17 |
+
"small": {
|
| 18 |
+
"repo_id": "depth-anything/Depth-Anything-V2-Small",
|
| 19 |
+
"filename": "depth_anything_v2_vits.onnx",
|
| 20 |
+
"size": "~100MB",
|
| 21 |
+
"speed": "Fast (25M params)"
|
| 22 |
+
},
|
| 23 |
+
"large": {
|
| 24 |
+
"repo_id": "depth-anything/Depth-Anything-V2-Large",
|
| 25 |
+
"filename": "depth_anything_v2_vitl.onnx",
|
| 26 |
+
"size": "~5GB",
|
| 27 |
+
"speed": "Slower (1.3B params)"
|
| 28 |
+
}
|
| 29 |
+
}
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def download_model(model_type: str, cache_dir: str = "./models/cache"):
|
| 33 |
+
"""
|
| 34 |
+
Download a Depth-Anything V2 ONNX model
|
| 35 |
+
|
| 36 |
+
Args:
|
| 37 |
+
model_type: Either 'small' or 'large'
|
| 38 |
+
cache_dir: Directory to cache models
|
| 39 |
+
"""
|
| 40 |
+
if model_type not in MODELS:
|
| 41 |
+
print(f"β Error: Unknown model type '{model_type}'")
|
| 42 |
+
print(f"Available models: {', '.join(MODELS.keys())}")
|
| 43 |
+
return False
|
| 44 |
+
|
| 45 |
+
model_info = MODELS[model_type]
|
| 46 |
+
cache_path = Path(cache_dir)
|
| 47 |
+
cache_path.mkdir(parents=True, exist_ok=True)
|
| 48 |
+
|
| 49 |
+
print(f"\nπ₯ Downloading {model_type} model...")
|
| 50 |
+
print(f" Repo: {model_info['repo_id']}")
|
| 51 |
+
print(f" File: {model_info['filename']}")
|
| 52 |
+
print(f" Size: {model_info['size']}")
|
| 53 |
+
print(f" Speed: {model_info['speed']}")
|
| 54 |
+
|
| 55 |
+
try:
|
| 56 |
+
# Note: Using a placeholder repo since actual ONNX models might not be available
|
| 57 |
+
# In production, you would either:
|
| 58 |
+
# 1. Convert PyTorch models to ONNX yourself
|
| 59 |
+
# 2. Use a community ONNX conversion
|
| 60 |
+
# 3. Host your own converted models
|
| 61 |
+
|
| 62 |
+
print("\nβ οΈ IMPORTANT NOTE:")
|
| 63 |
+
print("Official ONNX models may not be available on HuggingFace yet.")
|
| 64 |
+
print("You'll need to convert PyTorch models to ONNX format.")
|
| 65 |
+
print("\nTo convert models yourself:")
|
| 66 |
+
print("1. Install: pip install torch transformers")
|
| 67 |
+
print("2. Download PyTorch model")
|
| 68 |
+
print("3. Export to ONNX using torch.onnx.export()")
|
| 69 |
+
print("\nAlternatively, check these resources:")
|
| 70 |
+
print("- https://github.com/LiheYoung/Depth-Anything")
|
| 71 |
+
print("- Community ONNX conversions on HuggingFace")
|
| 72 |
+
|
| 73 |
+
# Placeholder for actual download
|
| 74 |
+
# model_path = hf_hub_download(
|
| 75 |
+
# repo_id=model_info['repo_id'],
|
| 76 |
+
# filename=model_info['filename'],
|
| 77 |
+
# cache_dir=str(cache_path)
|
| 78 |
+
# )
|
| 79 |
+
|
| 80 |
+
print(f"\nβ Model would be saved to: {cache_path / model_info['filename']}")
|
| 81 |
+
return True
|
| 82 |
+
|
| 83 |
+
except Exception as e:
|
| 84 |
+
print(f"\nβ Error downloading model: {e}")
|
| 85 |
+
return False
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
def create_conversion_script():
|
| 89 |
+
"""Create a helper script for converting PyTorch to ONNX"""
|
| 90 |
+
|
| 91 |
+
script_content = '''#!/usr/bin/env python3
|
| 92 |
+
"""
|
| 93 |
+
Convert Depth-Anything V2 PyTorch model to ONNX
|
| 94 |
+
"""
|
| 95 |
+
|
| 96 |
+
import torch
|
| 97 |
+
from transformers import AutoModel
|
| 98 |
+
import sys
|
| 99 |
+
|
| 100 |
+
def convert_to_onnx(model_name, output_path):
|
| 101 |
+
"""Convert model to ONNX format"""
|
| 102 |
+
|
| 103 |
+
print(f"Loading PyTorch model: {model_name}")
|
| 104 |
+
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)
|
| 105 |
+
model.eval()
|
| 106 |
+
|
| 107 |
+
# Dummy input
|
| 108 |
+
dummy_input = torch.randn(1, 3, 518, 518)
|
| 109 |
+
|
| 110 |
+
print(f"Exporting to ONNX: {output_path}")
|
| 111 |
+
torch.onnx.export(
|
| 112 |
+
model,
|
| 113 |
+
dummy_input,
|
| 114 |
+
output_path,
|
| 115 |
+
input_names=['input'],
|
| 116 |
+
output_names=['output'],
|
| 117 |
+
dynamic_axes={
|
| 118 |
+
'input': {0: 'batch', 2: 'height', 3: 'width'},
|
| 119 |
+
'output': {0: 'batch', 2: 'height', 3: 'width'}
|
| 120 |
+
},
|
| 121 |
+
opset_version=17
|
| 122 |
+
)
|
| 123 |
+
|
| 124 |
+
print(f"β Conversion complete: {output_path}")
|
| 125 |
+
|
| 126 |
+
if __name__ == "__main__":
|
| 127 |
+
# Example usage
|
| 128 |
+
convert_to_onnx(
|
| 129 |
+
"LiheYoung/depth-anything-small-hf",
|
| 130 |
+
"depth_anything_v2_vits.onnx"
|
| 131 |
+
)
|
| 132 |
+
'''
|
| 133 |
+
|
| 134 |
+
script_path = Path("convert_to_onnx.py")
|
| 135 |
+
script_path.write_text(script_content)
|
| 136 |
+
script_path.chmod(0o755)
|
| 137 |
+
|
| 138 |
+
print(f"\nβ Created conversion script: {script_path}")
|
| 139 |
+
print(" Run with: python convert_to_onnx.py")
|
| 140 |
+
|
| 141 |
+
|
| 142 |
+
def main():
|
| 143 |
+
"""Main download function"""
|
| 144 |
+
|
| 145 |
+
print("=" * 60)
|
| 146 |
+
print("Depth-Anything V2 Model Downloader")
|
| 147 |
+
print("=" * 60)
|
| 148 |
+
|
| 149 |
+
# Create models directory
|
| 150 |
+
models_dir = Path("./models/cache")
|
| 151 |
+
models_dir.mkdir(parents=True, exist_ok=True)
|
| 152 |
+
|
| 153 |
+
# Download models based on command line args
|
| 154 |
+
models_to_download = sys.argv[1:] if len(sys.argv) > 1 else ['small']
|
| 155 |
+
|
| 156 |
+
if 'all' in models_to_download:
|
| 157 |
+
models_to_download = list(MODELS.keys())
|
| 158 |
+
|
| 159 |
+
for model_type in models_to_download:
|
| 160 |
+
download_model(model_type)
|
| 161 |
+
|
| 162 |
+
# Create conversion helper
|
| 163 |
+
print("\n" + "=" * 60)
|
| 164 |
+
create_conversion_script()
|
| 165 |
+
|
| 166 |
+
print("\n" + "=" * 60)
|
| 167 |
+
print("Next Steps:")
|
| 168 |
+
print("=" * 60)
|
| 169 |
+
print("1. Convert PyTorch models to ONNX (see convert_to_onnx.py)")
|
| 170 |
+
print("2. Place ONNX models in ./models/cache/")
|
| 171 |
+
print("3. Update .env with correct model paths")
|
| 172 |
+
print("4. Start the server: uvicorn api.main:app --reload")
|
| 173 |
+
print("=" * 60)
|
| 174 |
+
|
| 175 |
+
|
| 176 |
+
if __name__ == "__main__":
|
| 177 |
+
main()
|
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# FastAPI and server dependencies
|
| 2 |
+
fastapi==0.115.5
|
| 3 |
+
uvicorn[standard]==0.32.1
|
| 4 |
+
python-multipart==0.0.20
|
| 5 |
+
websockets==14.1
|
| 6 |
+
|
| 7 |
+
# ML and image processing
|
| 8 |
+
onnxruntime-gpu==1.20.1
|
| 9 |
+
opencv-python==4.10.0.84
|
| 10 |
+
Pillow==11.0.0
|
| 11 |
+
numpy==1.26.4
|
| 12 |
+
huggingface-hub==0.27.0
|
| 13 |
+
|
| 14 |
+
# Caching and async
|
| 15 |
+
redis==5.2.1
|
| 16 |
+
aioredis==2.0.1
|
| 17 |
+
celery==5.4.0
|
| 18 |
+
|
| 19 |
+
# Utilities
|
| 20 |
+
python-dotenv==1.0.1
|
| 21 |
+
pydantic==2.10.3
|
| 22 |
+
pydantic-settings==2.6.1
|
| 23 |
+
|
| 24 |
+
# Cloud storage (optional)
|
| 25 |
+
boto3==1.35.76
|
| 26 |
+
|
| 27 |
+
# Image/video processing
|
| 28 |
+
imageio==2.36.1
|
| 29 |
+
imageio-ffmpeg==0.5.1
|
|
@@ -0,0 +1,293 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
Test script for Depth Flow Pro API
|
| 4 |
+
Tests all endpoints and measures performance
|
| 5 |
+
"""
|
| 6 |
+
|
| 7 |
+
import requests
|
| 8 |
+
import base64
|
| 9 |
+
import time
|
| 10 |
+
import sys
|
| 11 |
+
from pathlib import Path
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
class APITester:
|
| 15 |
+
def __init__(self, base_url="http://localhost:8000"):
|
| 16 |
+
self.base_url = base_url
|
| 17 |
+
self.test_results = []
|
| 18 |
+
|
| 19 |
+
def print_header(self, text):
|
| 20 |
+
"""Print formatted header"""
|
| 21 |
+
print("\n" + "=" * 60)
|
| 22 |
+
print(f" {text}")
|
| 23 |
+
print("=" * 60)
|
| 24 |
+
|
| 25 |
+
def print_result(self, name, success, time_ms=None, details=None):
|
| 26 |
+
"""Print test result"""
|
| 27 |
+
status = "β" if success else "β"
|
| 28 |
+
time_str = f" ({time_ms:.2f}ms)" if time_ms else ""
|
| 29 |
+
print(f"{status} {name}{time_str}")
|
| 30 |
+
|
| 31 |
+
if details:
|
| 32 |
+
for key, value in details.items():
|
| 33 |
+
print(f" {key}: {value}")
|
| 34 |
+
|
| 35 |
+
self.test_results.append({
|
| 36 |
+
"name": name,
|
| 37 |
+
"success": success,
|
| 38 |
+
"time_ms": time_ms
|
| 39 |
+
})
|
| 40 |
+
|
| 41 |
+
def test_health(self):
|
| 42 |
+
"""Test health endpoint"""
|
| 43 |
+
self.print_header("Testing Health Endpoints")
|
| 44 |
+
|
| 45 |
+
try:
|
| 46 |
+
# Root endpoint
|
| 47 |
+
start = time.time()
|
| 48 |
+
response = requests.get(f"{self.base_url}/")
|
| 49 |
+
time_ms = (time.time() - start) * 1000
|
| 50 |
+
|
| 51 |
+
data = response.json()
|
| 52 |
+
self.print_result(
|
| 53 |
+
"GET /",
|
| 54 |
+
response.status_code == 200,
|
| 55 |
+
time_ms,
|
| 56 |
+
{"models_loaded": data.get("models_loaded", [])}
|
| 57 |
+
)
|
| 58 |
+
|
| 59 |
+
# Health endpoint
|
| 60 |
+
start = time.time()
|
| 61 |
+
response = requests.get(f"{self.base_url}/health")
|
| 62 |
+
time_ms = (time.time() - start) * 1000
|
| 63 |
+
|
| 64 |
+
data = response.json()
|
| 65 |
+
self.print_result(
|
| 66 |
+
"GET /health",
|
| 67 |
+
response.status_code == 200,
|
| 68 |
+
time_ms,
|
| 69 |
+
{
|
| 70 |
+
"status": data.get("status"),
|
| 71 |
+
"gpu_enabled": data.get("gpu_enabled"),
|
| 72 |
+
"models": ", ".join(data.get("models", {}).keys())
|
| 73 |
+
}
|
| 74 |
+
)
|
| 75 |
+
|
| 76 |
+
return True
|
| 77 |
+
|
| 78 |
+
except Exception as e:
|
| 79 |
+
print(f"β Health check failed: {e}")
|
| 80 |
+
print("\nMake sure the server is running:")
|
| 81 |
+
print(" cd backend")
|
| 82 |
+
print(" uvicorn api.main:app --reload")
|
| 83 |
+
return False
|
| 84 |
+
|
| 85 |
+
def test_preview(self, image_path):
|
| 86 |
+
"""Test preview endpoint"""
|
| 87 |
+
self.print_header("Testing Preview Endpoint")
|
| 88 |
+
|
| 89 |
+
if not Path(image_path).exists():
|
| 90 |
+
print(f"β Image not found: {image_path}")
|
| 91 |
+
return False
|
| 92 |
+
|
| 93 |
+
try:
|
| 94 |
+
with open(image_path, 'rb') as f:
|
| 95 |
+
start = time.time()
|
| 96 |
+
|
| 97 |
+
response = requests.post(
|
| 98 |
+
f"{self.base_url}/api/v1/depth/preview",
|
| 99 |
+
files={'file': f}
|
| 100 |
+
)
|
| 101 |
+
|
| 102 |
+
time_ms = (time.time() - start) * 1000
|
| 103 |
+
|
| 104 |
+
if response.status_code == 200:
|
| 105 |
+
data = response.json()
|
| 106 |
+
self.print_result(
|
| 107 |
+
"POST /api/v1/depth/preview",
|
| 108 |
+
True,
|
| 109 |
+
time_ms,
|
| 110 |
+
{
|
| 111 |
+
"model": data["metadata"]["model"],
|
| 112 |
+
"input_size": f"{data['metadata']['input_size'][0]}x{data['metadata']['input_size'][1]}",
|
| 113 |
+
"server_time": f"{data['processing_time_ms']:.2f}ms"
|
| 114 |
+
}
|
| 115 |
+
)
|
| 116 |
+
return True
|
| 117 |
+
else:
|
| 118 |
+
self.print_result(
|
| 119 |
+
"POST /api/v1/depth/preview",
|
| 120 |
+
False,
|
| 121 |
+
details={"error": response.text}
|
| 122 |
+
)
|
| 123 |
+
return False
|
| 124 |
+
|
| 125 |
+
except Exception as e:
|
| 126 |
+
print(f"β Preview test failed: {e}")
|
| 127 |
+
return False
|
| 128 |
+
|
| 129 |
+
def test_hq(self, image_path):
|
| 130 |
+
"""Test HQ endpoint"""
|
| 131 |
+
self.print_header("Testing High Quality Endpoint")
|
| 132 |
+
|
| 133 |
+
if not Path(image_path).exists():
|
| 134 |
+
print(f"β Image not found: {image_path}")
|
| 135 |
+
return False
|
| 136 |
+
|
| 137 |
+
try:
|
| 138 |
+
with open(image_path, 'rb') as f:
|
| 139 |
+
start = time.time()
|
| 140 |
+
|
| 141 |
+
response = requests.post(
|
| 142 |
+
f"{self.base_url}/api/v1/depth/hq",
|
| 143 |
+
files={'file': f}
|
| 144 |
+
)
|
| 145 |
+
|
| 146 |
+
time_ms = (time.time() - start) * 1000
|
| 147 |
+
|
| 148 |
+
if response.status_code == 200:
|
| 149 |
+
data = response.json()
|
| 150 |
+
self.print_result(
|
| 151 |
+
"POST /api/v1/depth/hq",
|
| 152 |
+
True,
|
| 153 |
+
time_ms,
|
| 154 |
+
{
|
| 155 |
+
"model": data["metadata"]["model"],
|
| 156 |
+
"input_size": f"{data['metadata']['input_size'][0]}x{data['metadata']['input_size'][1]}",
|
| 157 |
+
"server_time": f"{data['processing_time_ms']:.2f}ms"
|
| 158 |
+
}
|
| 159 |
+
)
|
| 160 |
+
return True
|
| 161 |
+
else:
|
| 162 |
+
self.print_result(
|
| 163 |
+
"POST /api/v1/depth/hq",
|
| 164 |
+
False,
|
| 165 |
+
details={"error": response.text}
|
| 166 |
+
)
|
| 167 |
+
return False
|
| 168 |
+
|
| 169 |
+
except Exception as e:
|
| 170 |
+
print(f"β HQ test failed: {e}")
|
| 171 |
+
return False
|
| 172 |
+
|
| 173 |
+
def test_estimate(self, image_path):
|
| 174 |
+
"""Test custom estimate endpoint"""
|
| 175 |
+
self.print_header("Testing Custom Estimate Endpoint")
|
| 176 |
+
|
| 177 |
+
if not Path(image_path).exists():
|
| 178 |
+
print(f"β Image not found: {image_path}")
|
| 179 |
+
return False
|
| 180 |
+
|
| 181 |
+
try:
|
| 182 |
+
# Read and encode image
|
| 183 |
+
with open(image_path, 'rb') as f:
|
| 184 |
+
image_data = f.read()
|
| 185 |
+
image_base64 = base64.b64encode(image_data).decode()
|
| 186 |
+
|
| 187 |
+
# Test different configurations
|
| 188 |
+
configs = [
|
| 189 |
+
{"model": "small", "output_format": "grayscale", "colormap": "inferno"},
|
| 190 |
+
{"model": "small", "output_format": "colormap", "colormap": "viridis"},
|
| 191 |
+
{"model": "small", "output_format": "both", "colormap": "plasma"},
|
| 192 |
+
]
|
| 193 |
+
|
| 194 |
+
for config in configs:
|
| 195 |
+
start = time.time()
|
| 196 |
+
|
| 197 |
+
response = requests.post(
|
| 198 |
+
f"{self.base_url}/api/v1/depth/estimate",
|
| 199 |
+
json={
|
| 200 |
+
"image": f"data:image/jpeg;base64,{image_base64}",
|
| 201 |
+
**config
|
| 202 |
+
}
|
| 203 |
+
)
|
| 204 |
+
|
| 205 |
+
time_ms = (time.time() - start) * 1000
|
| 206 |
+
|
| 207 |
+
if response.status_code == 200:
|
| 208 |
+
data = response.json()
|
| 209 |
+
self.print_result(
|
| 210 |
+
f"Estimate ({config['output_format']})",
|
| 211 |
+
True,
|
| 212 |
+
time_ms,
|
| 213 |
+
{"colormap": config['colormap']}
|
| 214 |
+
)
|
| 215 |
+
else:
|
| 216 |
+
self.print_result(
|
| 217 |
+
f"Estimate ({config['output_format']})",
|
| 218 |
+
False,
|
| 219 |
+
details={"error": response.text}
|
| 220 |
+
)
|
| 221 |
+
|
| 222 |
+
return True
|
| 223 |
+
|
| 224 |
+
except Exception as e:
|
| 225 |
+
print(f"β Estimate test failed: {e}")
|
| 226 |
+
return False
|
| 227 |
+
|
| 228 |
+
def print_summary(self):
|
| 229 |
+
"""Print test summary"""
|
| 230 |
+
self.print_header("Test Summary")
|
| 231 |
+
|
| 232 |
+
total = len(self.test_results)
|
| 233 |
+
passed = sum(1 for r in self.test_results if r["success"])
|
| 234 |
+
failed = total - passed
|
| 235 |
+
|
| 236 |
+
print(f"\nTotal Tests: {total}")
|
| 237 |
+
print(f"Passed: {passed} β")
|
| 238 |
+
print(f"Failed: {failed} β")
|
| 239 |
+
|
| 240 |
+
if passed == total:
|
| 241 |
+
print("\nπ All tests passed!")
|
| 242 |
+
else:
|
| 243 |
+
print(f"\nβ οΈ {failed} test(s) failed")
|
| 244 |
+
|
| 245 |
+
# Performance summary
|
| 246 |
+
if any(r["time_ms"] for r in self.test_results):
|
| 247 |
+
print("\nPerformance Summary:")
|
| 248 |
+
for result in self.test_results:
|
| 249 |
+
if result["time_ms"]:
|
| 250 |
+
print(f" {result['name']}: {result['time_ms']:.2f}ms")
|
| 251 |
+
|
| 252 |
+
|
| 253 |
+
def main():
|
| 254 |
+
"""Run all tests"""
|
| 255 |
+
print("""
|
| 256 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 257 |
+
β Depth Flow Pro - API Test Suite β
|
| 258 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 259 |
+
""")
|
| 260 |
+
|
| 261 |
+
# Check for test image
|
| 262 |
+
if len(sys.argv) < 2:
|
| 263 |
+
print("Usage: python test_api.py <image_path>")
|
| 264 |
+
print("\nExample:")
|
| 265 |
+
print(" python test_api.py test_image.jpg")
|
| 266 |
+
sys.exit(1)
|
| 267 |
+
|
| 268 |
+
image_path = sys.argv[1]
|
| 269 |
+
|
| 270 |
+
if not Path(image_path).exists():
|
| 271 |
+
print(f"Error: Image not found: {image_path}")
|
| 272 |
+
sys.exit(1)
|
| 273 |
+
|
| 274 |
+
print(f"Test image: {image_path}")
|
| 275 |
+
|
| 276 |
+
# Initialize tester
|
| 277 |
+
tester = APITester()
|
| 278 |
+
|
| 279 |
+
# Run tests
|
| 280 |
+
if not tester.test_health():
|
| 281 |
+
print("\nβ Server not accessible. Aborting tests.")
|
| 282 |
+
sys.exit(1)
|
| 283 |
+
|
| 284 |
+
tester.test_preview(image_path)
|
| 285 |
+
tester.test_hq(image_path)
|
| 286 |
+
tester.test_estimate(image_path)
|
| 287 |
+
|
| 288 |
+
# Print summary
|
| 289 |
+
tester.print_summary()
|
| 290 |
+
|
| 291 |
+
|
| 292 |
+
if __name__ == "__main__":
|
| 293 |
+
main()
|
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
Quick test script to verify ONNX models work
|
| 4 |
+
"""
|
| 5 |
+
import cv2
|
| 6 |
+
import numpy as np
|
| 7 |
+
from utils.model_loader import DepthAnythingV2
|
| 8 |
+
from pathlib import Path
|
| 9 |
+
|
| 10 |
+
print("Testing ONNX Depth-Anything V2 models...")
|
| 11 |
+
|
| 12 |
+
# Create a test image
|
| 13 |
+
test_image = np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8)
|
| 14 |
+
print(f"Test image shape: {test_image.shape}")
|
| 15 |
+
|
| 16 |
+
# Test small model
|
| 17 |
+
small_model_path = Path("models/cache/depth_anything_v2_vits.onnx")
|
| 18 |
+
print(f"\nTesting small model: {small_model_path}")
|
| 19 |
+
print(f"Model exists: {small_model_path.exists()}")
|
| 20 |
+
|
| 21 |
+
try:
|
| 22 |
+
model = DepthAnythingV2(str(small_model_path), use_gpu=False)
|
| 23 |
+
print("β Model loaded successfully")
|
| 24 |
+
|
| 25 |
+
print("Running inference...")
|
| 26 |
+
depth = model.predict(test_image)
|
| 27 |
+
print(f"β Inference successful!")
|
| 28 |
+
print(f" Output shape: {depth.shape}")
|
| 29 |
+
print(f" Output range: {depth.min():.3f} to {depth.max():.3f}")
|
| 30 |
+
|
| 31 |
+
except Exception as e:
|
| 32 |
+
print(f"β Error: {type(e).__name__}: {str(e)}")
|
| 33 |
+
import traceback
|
| 34 |
+
traceback.print_exc()
|
|
@@ -0,0 +1,92 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Demo depth generator for testing without ONNX models
|
| 3 |
+
Creates synthetic depth maps for demonstration
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import numpy as np
|
| 7 |
+
import cv2
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def generate_demo_depth(image: np.ndarray, method: str = "gradient") -> np.ndarray:
|
| 11 |
+
"""
|
| 12 |
+
Generate a synthetic depth map for demo purposes
|
| 13 |
+
|
| 14 |
+
Args:
|
| 15 |
+
image: Input RGB image
|
| 16 |
+
method: Method to use ('gradient', 'center', 'edges')
|
| 17 |
+
|
| 18 |
+
Returns:
|
| 19 |
+
Synthetic depth map (0-1 range)
|
| 20 |
+
"""
|
| 21 |
+
h, w = image.shape[:2]
|
| 22 |
+
|
| 23 |
+
if method == "gradient":
|
| 24 |
+
# Simple vertical gradient (top is far, bottom is near)
|
| 25 |
+
depth = np.linspace(0, 1, h)
|
| 26 |
+
depth = np.tile(depth[:, np.newaxis], (1, w))
|
| 27 |
+
|
| 28 |
+
elif method == "center":
|
| 29 |
+
# Radial gradient from center
|
| 30 |
+
y, x = np.ogrid[:h, :w]
|
| 31 |
+
cy, cx = h // 2, w // 2
|
| 32 |
+
|
| 33 |
+
distance = np.sqrt((x - cx)**2 + (y - cy)**2)
|
| 34 |
+
depth = distance / distance.max()
|
| 35 |
+
depth = 1 - depth # Invert so center is near
|
| 36 |
+
|
| 37 |
+
elif method == "edges":
|
| 38 |
+
# Use edge detection as depth approximation
|
| 39 |
+
gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
|
| 40 |
+
edges = cv2.Canny(gray, 50, 150)
|
| 41 |
+
edges = edges.astype(float) / 255.0
|
| 42 |
+
|
| 43 |
+
# Blur edges to create depth-like effect
|
| 44 |
+
depth = cv2.GaussianBlur(edges, (21, 21), 0)
|
| 45 |
+
depth = 1 - depth # Invert
|
| 46 |
+
|
| 47 |
+
else:
|
| 48 |
+
# Random depth for testing
|
| 49 |
+
depth = np.random.rand(h, w)
|
| 50 |
+
|
| 51 |
+
# Normalize to 0-1 range
|
| 52 |
+
depth = (depth - depth.min()) / (depth.max() - depth.min() + 1e-8)
|
| 53 |
+
|
| 54 |
+
return depth.astype(np.float32)
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def generate_smart_depth(image: np.ndarray) -> np.ndarray:
|
| 58 |
+
"""
|
| 59 |
+
Generate a smarter synthetic depth using image analysis
|
| 60 |
+
Better than simple gradients but still demo quality
|
| 61 |
+
"""
|
| 62 |
+
h, w = image.shape[:2]
|
| 63 |
+
|
| 64 |
+
# Convert to grayscale
|
| 65 |
+
gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
|
| 66 |
+
|
| 67 |
+
# Use intensity as rough depth proxy (darker = farther)
|
| 68 |
+
depth_from_intensity = 1 - (gray.astype(float) / 255.0)
|
| 69 |
+
|
| 70 |
+
# Get edge information
|
| 71 |
+
edges = cv2.Canny(gray, 50, 150).astype(float) / 255.0
|
| 72 |
+
edges_blur = cv2.GaussianBlur(edges, (15, 15), 0)
|
| 73 |
+
|
| 74 |
+
# Combine intensity and edge info
|
| 75 |
+
depth = 0.6 * depth_from_intensity + 0.4 * (1 - edges_blur)
|
| 76 |
+
|
| 77 |
+
# Apply smoothing
|
| 78 |
+
depth = cv2.GaussianBlur(depth, (31, 31), 0)
|
| 79 |
+
|
| 80 |
+
# Add some central bias (center tends to be closer)
|
| 81 |
+
y, x = np.ogrid[:h, :w]
|
| 82 |
+
cy, cx = h // 2, w // 2
|
| 83 |
+
distance = np.sqrt((x - cx)**2 + (y - cy)**2)
|
| 84 |
+
radial = distance / distance.max()
|
| 85 |
+
radial = 1 - radial
|
| 86 |
+
|
| 87 |
+
depth = 0.7 * depth + 0.3 * radial
|
| 88 |
+
|
| 89 |
+
# Normalize
|
| 90 |
+
depth = (depth - depth.min()) / (depth.max() - depth.min() + 1e-8)
|
| 91 |
+
|
| 92 |
+
return depth.astype(np.float32)
|
|
@@ -0,0 +1,197 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import cv2
|
| 2 |
+
import numpy as np
|
| 3 |
+
from PIL import Image
|
| 4 |
+
import io
|
| 5 |
+
import base64
|
| 6 |
+
from typing import Tuple, Optional
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def load_image_from_bytes(image_bytes: bytes) -> np.ndarray:
|
| 10 |
+
"""
|
| 11 |
+
Load image from bytes into numpy array
|
| 12 |
+
|
| 13 |
+
Args:
|
| 14 |
+
image_bytes: Raw image bytes
|
| 15 |
+
|
| 16 |
+
Returns:
|
| 17 |
+
Image as RGB numpy array
|
| 18 |
+
"""
|
| 19 |
+
image = Image.open(io.BytesIO(image_bytes))
|
| 20 |
+
|
| 21 |
+
# Convert to RGB if needed
|
| 22 |
+
if image.mode != 'RGB':
|
| 23 |
+
image = image.convert('RGB')
|
| 24 |
+
|
| 25 |
+
return np.array(image)
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
def load_image_from_base64(base64_string: str) -> np.ndarray:
|
| 29 |
+
"""
|
| 30 |
+
Load image from base64 string
|
| 31 |
+
|
| 32 |
+
Args:
|
| 33 |
+
base64_string: Base64 encoded image
|
| 34 |
+
|
| 35 |
+
Returns:
|
| 36 |
+
Image as RGB numpy array
|
| 37 |
+
"""
|
| 38 |
+
# Remove data URL prefix if present
|
| 39 |
+
if ',' in base64_string:
|
| 40 |
+
base64_string = base64_string.split(',')[1]
|
| 41 |
+
|
| 42 |
+
image_bytes = base64.b64decode(base64_string)
|
| 43 |
+
return load_image_from_bytes(image_bytes)
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def resize_image(
|
| 47 |
+
image: np.ndarray,
|
| 48 |
+
target_size: int,
|
| 49 |
+
maintain_aspect: bool = True
|
| 50 |
+
) -> Tuple[np.ndarray, Tuple[int, int]]:
|
| 51 |
+
"""
|
| 52 |
+
Resize image to target size
|
| 53 |
+
|
| 54 |
+
Args:
|
| 55 |
+
image: Input image array
|
| 56 |
+
target_size: Target size (will be longest edge if maintain_aspect=True)
|
| 57 |
+
maintain_aspect: Whether to maintain aspect ratio
|
| 58 |
+
|
| 59 |
+
Returns:
|
| 60 |
+
Tuple of (resized_image, original_size)
|
| 61 |
+
"""
|
| 62 |
+
h, w = image.shape[:2]
|
| 63 |
+
original_size = (w, h)
|
| 64 |
+
|
| 65 |
+
if maintain_aspect:
|
| 66 |
+
# Calculate new dimensions maintaining aspect ratio
|
| 67 |
+
if h > w:
|
| 68 |
+
new_h = target_size
|
| 69 |
+
new_w = int(w * (target_size / h))
|
| 70 |
+
else:
|
| 71 |
+
new_w = target_size
|
| 72 |
+
new_h = int(h * (target_size / w))
|
| 73 |
+
else:
|
| 74 |
+
new_w = target_size
|
| 75 |
+
new_h = target_size
|
| 76 |
+
|
| 77 |
+
resized = cv2.resize(image, (new_w, new_h), interpolation=cv2.INTER_LINEAR)
|
| 78 |
+
return resized, original_size
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def normalize_image(image: np.ndarray) -> np.ndarray:
|
| 82 |
+
"""
|
| 83 |
+
Normalize image for model input
|
| 84 |
+
|
| 85 |
+
Args:
|
| 86 |
+
image: Input image array (RGB)
|
| 87 |
+
|
| 88 |
+
Returns:
|
| 89 |
+
Normalized image array
|
| 90 |
+
"""
|
| 91 |
+
# Convert to float32 and normalize to [0, 1]
|
| 92 |
+
image = image.astype(np.float32) / 255.0
|
| 93 |
+
|
| 94 |
+
# ImageNet normalization
|
| 95 |
+
mean = np.array([0.485, 0.456, 0.406])
|
| 96 |
+
std = np.array([0.229, 0.224, 0.225])
|
| 97 |
+
|
| 98 |
+
image = (image - mean) / std
|
| 99 |
+
|
| 100 |
+
return image
|
| 101 |
+
|
| 102 |
+
|
| 103 |
+
def depth_to_colormap(
|
| 104 |
+
depth: np.ndarray,
|
| 105 |
+
colormap: int = cv2.COLORMAP_INFERNO
|
| 106 |
+
) -> np.ndarray:
|
| 107 |
+
"""
|
| 108 |
+
Convert depth map to colorized visualization
|
| 109 |
+
|
| 110 |
+
Args:
|
| 111 |
+
depth: Depth map array
|
| 112 |
+
colormap: OpenCV colormap constant
|
| 113 |
+
|
| 114 |
+
Returns:
|
| 115 |
+
Colorized depth map (RGB)
|
| 116 |
+
"""
|
| 117 |
+
# Normalize depth to 0-255
|
| 118 |
+
depth_normalized = cv2.normalize(depth, None, 0, 255, cv2.NORM_MINMAX)
|
| 119 |
+
depth_uint8 = depth_normalized.astype(np.uint8)
|
| 120 |
+
|
| 121 |
+
# Apply colormap
|
| 122 |
+
colored = cv2.applyColorMap(depth_uint8, colormap)
|
| 123 |
+
|
| 124 |
+
# Convert BGR to RGB
|
| 125 |
+
colored = cv2.cvtColor(colored, cv2.COLOR_BGR2RGB)
|
| 126 |
+
|
| 127 |
+
return colored
|
| 128 |
+
|
| 129 |
+
|
| 130 |
+
def array_to_base64(image: np.ndarray, format: str = 'PNG') -> str:
|
| 131 |
+
"""
|
| 132 |
+
Convert numpy array to base64 string
|
| 133 |
+
|
| 134 |
+
Args:
|
| 135 |
+
image: Image array
|
| 136 |
+
format: Output format (PNG, JPEG, etc.)
|
| 137 |
+
|
| 138 |
+
Returns:
|
| 139 |
+
Base64 encoded image string
|
| 140 |
+
"""
|
| 141 |
+
pil_image = Image.fromarray(image.astype(np.uint8))
|
| 142 |
+
|
| 143 |
+
buffer = io.BytesIO()
|
| 144 |
+
pil_image.save(buffer, format=format)
|
| 145 |
+
buffer.seek(0)
|
| 146 |
+
|
| 147 |
+
base64_string = base64.b64encode(buffer.read()).decode('utf-8')
|
| 148 |
+
return f"data:image/{format.lower()};base64,{base64_string}"
|
| 149 |
+
|
| 150 |
+
|
| 151 |
+
def array_to_bytes(image: np.ndarray, format: str = 'PNG') -> bytes:
|
| 152 |
+
"""
|
| 153 |
+
Convert numpy array to bytes
|
| 154 |
+
|
| 155 |
+
Args:
|
| 156 |
+
image: Image array
|
| 157 |
+
format: Output format (PNG, JPEG, etc.)
|
| 158 |
+
|
| 159 |
+
Returns:
|
| 160 |
+
Image bytes
|
| 161 |
+
"""
|
| 162 |
+
pil_image = Image.fromarray(image.astype(np.uint8))
|
| 163 |
+
|
| 164 |
+
buffer = io.BytesIO()
|
| 165 |
+
pil_image.save(buffer, format=format)
|
| 166 |
+
buffer.seek(0)
|
| 167 |
+
|
| 168 |
+
return buffer.read()
|
| 169 |
+
|
| 170 |
+
|
| 171 |
+
def create_side_by_side(
|
| 172 |
+
original: np.ndarray,
|
| 173 |
+
depth: np.ndarray,
|
| 174 |
+
colormap: bool = True
|
| 175 |
+
) -> np.ndarray:
|
| 176 |
+
"""
|
| 177 |
+
Create side-by-side comparison of original and depth
|
| 178 |
+
|
| 179 |
+
Args:
|
| 180 |
+
original: Original image
|
| 181 |
+
depth: Depth map
|
| 182 |
+
colormap: Whether to apply colormap to depth
|
| 183 |
+
|
| 184 |
+
Returns:
|
| 185 |
+
Side-by-side image
|
| 186 |
+
"""
|
| 187 |
+
# Ensure same height
|
| 188 |
+
h = original.shape[0]
|
| 189 |
+
depth_resized = cv2.resize(depth, (depth.shape[1], h))
|
| 190 |
+
|
| 191 |
+
if colormap and len(depth_resized.shape) == 2:
|
| 192 |
+
depth_resized = depth_to_colormap(depth_resized)
|
| 193 |
+
|
| 194 |
+
# Concatenate horizontally
|
| 195 |
+
combined = np.hstack([original, depth_resized])
|
| 196 |
+
|
| 197 |
+
return combined
|
|
@@ -0,0 +1,231 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import onnxruntime as ort
|
| 2 |
+
import numpy as np
|
| 3 |
+
from pathlib import Path
|
| 4 |
+
from typing import Optional, Tuple
|
| 5 |
+
import cv2
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
class DepthAnythingV2:
|
| 9 |
+
"""
|
| 10 |
+
Depth Anything V2 model wrapper for ONNX inference
|
| 11 |
+
Supports both small (25M) and large (1.3B) models
|
| 12 |
+
"""
|
| 13 |
+
|
| 14 |
+
def __init__(
|
| 15 |
+
self,
|
| 16 |
+
model_path: str,
|
| 17 |
+
use_gpu: bool = True,
|
| 18 |
+
use_tensorrt: bool = False
|
| 19 |
+
):
|
| 20 |
+
"""
|
| 21 |
+
Initialize Depth Anything V2 model
|
| 22 |
+
|
| 23 |
+
Args:
|
| 24 |
+
model_path: Path to ONNX model file
|
| 25 |
+
use_gpu: Whether to use GPU acceleration
|
| 26 |
+
use_tensorrt: Whether to use TensorRT optimization
|
| 27 |
+
"""
|
| 28 |
+
self.model_path = Path(model_path)
|
| 29 |
+
|
| 30 |
+
if not self.model_path.exists():
|
| 31 |
+
raise FileNotFoundError(f"Model not found: {model_path}")
|
| 32 |
+
|
| 33 |
+
# Setup ONNX Runtime session
|
| 34 |
+
providers = self._get_providers(use_gpu, use_tensorrt)
|
| 35 |
+
|
| 36 |
+
session_options = ort.SessionOptions()
|
| 37 |
+
session_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
|
| 38 |
+
|
| 39 |
+
self.session = ort.InferenceSession(
|
| 40 |
+
str(self.model_path),
|
| 41 |
+
sess_options=session_options,
|
| 42 |
+
providers=providers
|
| 43 |
+
)
|
| 44 |
+
|
| 45 |
+
# Get input/output names
|
| 46 |
+
self.input_name = self.session.get_inputs()[0].name
|
| 47 |
+
self.output_name = self.session.get_outputs()[0].name
|
| 48 |
+
|
| 49 |
+
# Get expected input shape
|
| 50 |
+
input_shape = self.session.get_inputs()[0].shape
|
| 51 |
+
# Handle dynamic dimensions (e.g., ['batch_size', 3, 'height', 'width'])
|
| 52 |
+
# Default to 518x518 for Depth-Anything V2
|
| 53 |
+
if isinstance(input_shape[2], str):
|
| 54 |
+
self.input_height = 518
|
| 55 |
+
self.input_width = 518
|
| 56 |
+
else:
|
| 57 |
+
self.input_height = input_shape[2]
|
| 58 |
+
self.input_width = input_shape[3]
|
| 59 |
+
|
| 60 |
+
print(f"β Loaded model: {self.model_path.name}")
|
| 61 |
+
print(f" Input shape: {input_shape}")
|
| 62 |
+
print(f" Providers: {providers}")
|
| 63 |
+
|
| 64 |
+
def _get_providers(self, use_gpu: bool, use_tensorrt: bool) -> list:
|
| 65 |
+
"""Get ONNX Runtime execution providers"""
|
| 66 |
+
providers = []
|
| 67 |
+
|
| 68 |
+
if use_tensorrt and use_gpu:
|
| 69 |
+
providers.append('TensorrtExecutionProvider')
|
| 70 |
+
|
| 71 |
+
if use_gpu:
|
| 72 |
+
providers.append('CUDAExecutionProvider')
|
| 73 |
+
|
| 74 |
+
providers.append('CPUExecutionProvider')
|
| 75 |
+
|
| 76 |
+
return providers
|
| 77 |
+
|
| 78 |
+
def preprocess(self, image: np.ndarray) -> Tuple[np.ndarray, Tuple[int, int]]:
|
| 79 |
+
"""
|
| 80 |
+
Preprocess image for model input
|
| 81 |
+
|
| 82 |
+
Args:
|
| 83 |
+
image: Input image (RGB, HxWx3)
|
| 84 |
+
|
| 85 |
+
Returns:
|
| 86 |
+
Tuple of (preprocessed_image, original_size)
|
| 87 |
+
"""
|
| 88 |
+
h, w = image.shape[:2]
|
| 89 |
+
original_size = (h, w)
|
| 90 |
+
|
| 91 |
+
# Resize to model input size
|
| 92 |
+
image = cv2.resize(
|
| 93 |
+
image,
|
| 94 |
+
(self.input_width, self.input_height),
|
| 95 |
+
interpolation=cv2.INTER_LINEAR
|
| 96 |
+
)
|
| 97 |
+
|
| 98 |
+
# Normalize
|
| 99 |
+
image = image.astype(np.float32) / 255.0
|
| 100 |
+
|
| 101 |
+
# ImageNet normalization
|
| 102 |
+
mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
|
| 103 |
+
std = np.array([0.229, 0.224, 0.225], dtype=np.float32)
|
| 104 |
+
image = (image - mean) / std
|
| 105 |
+
|
| 106 |
+
# Transpose to NCHW format
|
| 107 |
+
image = image.transpose(2, 0, 1)
|
| 108 |
+
image = np.expand_dims(image, axis=0)
|
| 109 |
+
|
| 110 |
+
return image, original_size
|
| 111 |
+
|
| 112 |
+
def postprocess(
|
| 113 |
+
self,
|
| 114 |
+
depth: np.ndarray,
|
| 115 |
+
original_size: Tuple[int, int]
|
| 116 |
+
) -> np.ndarray:
|
| 117 |
+
"""
|
| 118 |
+
Postprocess depth map output
|
| 119 |
+
|
| 120 |
+
Args:
|
| 121 |
+
depth: Raw depth output from model
|
| 122 |
+
original_size: Original image size (h, w)
|
| 123 |
+
|
| 124 |
+
Returns:
|
| 125 |
+
Depth map resized to original size
|
| 126 |
+
"""
|
| 127 |
+
# Remove batch dimension
|
| 128 |
+
if len(depth.shape) == 4:
|
| 129 |
+
depth = depth[0]
|
| 130 |
+
|
| 131 |
+
# Remove channel dimension if present
|
| 132 |
+
if len(depth.shape) == 3:
|
| 133 |
+
depth = depth[0]
|
| 134 |
+
|
| 135 |
+
# Resize to original size
|
| 136 |
+
h, w = original_size
|
| 137 |
+
depth = cv2.resize(depth, (w, h), interpolation=cv2.INTER_LINEAR)
|
| 138 |
+
|
| 139 |
+
# Normalize to 0-1 range
|
| 140 |
+
depth = (depth - depth.min()) / (depth.max() - depth.min() + 1e-8)
|
| 141 |
+
|
| 142 |
+
return depth
|
| 143 |
+
|
| 144 |
+
def predict(
|
| 145 |
+
self,
|
| 146 |
+
image: np.ndarray,
|
| 147 |
+
resize_output: bool = True
|
| 148 |
+
) -> np.ndarray:
|
| 149 |
+
"""
|
| 150 |
+
Run depth estimation on image
|
| 151 |
+
|
| 152 |
+
Args:
|
| 153 |
+
image: Input image (RGB, HxWx3)
|
| 154 |
+
resize_output: Whether to resize output to original size
|
| 155 |
+
|
| 156 |
+
Returns:
|
| 157 |
+
Depth map (same size as input if resize_output=True)
|
| 158 |
+
"""
|
| 159 |
+
# Preprocess
|
| 160 |
+
input_tensor, original_size = self.preprocess(image)
|
| 161 |
+
|
| 162 |
+
# Run inference
|
| 163 |
+
outputs = self.session.run(
|
| 164 |
+
[self.output_name],
|
| 165 |
+
{self.input_name: input_tensor}
|
| 166 |
+
)
|
| 167 |
+
|
| 168 |
+
depth = outputs[0]
|
| 169 |
+
|
| 170 |
+
# Postprocess
|
| 171 |
+
if resize_output:
|
| 172 |
+
depth = self.postprocess(depth, original_size)
|
| 173 |
+
|
| 174 |
+
return depth
|
| 175 |
+
|
| 176 |
+
def __call__(self, image: np.ndarray) -> np.ndarray:
|
| 177 |
+
"""Convenience method for prediction"""
|
| 178 |
+
return self.predict(image)
|
| 179 |
+
|
| 180 |
+
|
| 181 |
+
class ModelManager:
|
| 182 |
+
"""
|
| 183 |
+
Manages multiple depth models and provides a unified interface
|
| 184 |
+
"""
|
| 185 |
+
|
| 186 |
+
def __init__(self):
|
| 187 |
+
self.models = {}
|
| 188 |
+
|
| 189 |
+
def load_model(
|
| 190 |
+
self,
|
| 191 |
+
name: str,
|
| 192 |
+
model_path: str,
|
| 193 |
+
use_gpu: bool = True,
|
| 194 |
+
use_tensorrt: bool = False
|
| 195 |
+
) -> DepthAnythingV2:
|
| 196 |
+
"""
|
| 197 |
+
Load a depth model
|
| 198 |
+
|
| 199 |
+
Args:
|
| 200 |
+
name: Model identifier (e.g., 'small', 'large')
|
| 201 |
+
model_path: Path to ONNX model
|
| 202 |
+
use_gpu: Whether to use GPU
|
| 203 |
+
use_tensorrt: Whether to use TensorRT
|
| 204 |
+
|
| 205 |
+
Returns:
|
| 206 |
+
Loaded model instance
|
| 207 |
+
"""
|
| 208 |
+
model = DepthAnythingV2(model_path, use_gpu, use_tensorrt)
|
| 209 |
+
self.models[name] = model
|
| 210 |
+
return model
|
| 211 |
+
|
| 212 |
+
def get_model(self, name: str) -> Optional[DepthAnythingV2]:
|
| 213 |
+
"""Get a loaded model by name"""
|
| 214 |
+
return self.models.get(name)
|
| 215 |
+
|
| 216 |
+
def predict(self, image: np.ndarray, model_name: str = 'small') -> np.ndarray:
|
| 217 |
+
"""
|
| 218 |
+
Run prediction using specified model
|
| 219 |
+
|
| 220 |
+
Args:
|
| 221 |
+
image: Input image
|
| 222 |
+
model_name: Name of model to use
|
| 223 |
+
|
| 224 |
+
Returns:
|
| 225 |
+
Depth map
|
| 226 |
+
"""
|
| 227 |
+
model = self.get_model(model_name)
|
| 228 |
+
if model is None:
|
| 229 |
+
raise ValueError(f"Model '{model_name}' not loaded")
|
| 230 |
+
|
| 231 |
+
return model.predict(image)
|
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Gradio and UI
|
| 2 |
+
gradio==4.44.0
|
| 3 |
+
|
| 4 |
+
# Core ML and image processing
|
| 5 |
+
onnxruntime-gpu==1.20.1
|
| 6 |
+
opencv-python==4.10.0.84
|
| 7 |
+
Pillow==11.0.0
|
| 8 |
+
numpy==1.26.4
|
| 9 |
+
|
| 10 |
+
# Optional: For downloading models from HuggingFace
|
| 11 |
+
huggingface-hub==0.27.0
|
| 12 |
+
|
| 13 |
+
# Utilities
|
| 14 |
+
python-dotenv==1.0.1
|