Claude
1456995462
Implemented a full-stack web application for fine-tuning LLMs on email data, optimized for Apple Silicon (M4 Pro with 24GB RAM). Features: - Mail import with drag & drop support (.mbox, .eml, .txt) - Automated mail cleaning and preprocessing - Interactive labeling interface with keyboard shortcuts - Training data export to JSONL format - MLX-based LoRA fine-tuning with live updates - Model evaluation and comparison interface - Server-Sent Events for real-time training progress - Dark theme UI optimized for extended use Technical Stack: - Backend: FastAPI with SQLite database - Frontend: Vanilla HTML/CSS/JavaScript (no external dependencies) - ML Framework: MLX for Apple Silicon optimization - Models: Support for Mistral 7B and Llama 3 8B via MLX Components: - data_manager.py: SQLite operations for mail storage and labeling - mail_parser.py: Parser for multiple mail formats with cleaning - training.py: MLX training wrapper with LoRA support - inference.py: Model loading and inference for evaluation - main.py: FastAPI backend with REST API and SSE - Frontend: Complete UI with all features Documentation: - Comprehensive README with installation and usage guide - Quick-start guide for rapid setup - Example mails for testing - Troubleshooting and best practices Ready for local deployment and fine-tuning workflows.
25 lines
357 B
Plaintext
25 lines
357 B
Plaintext
# Mail Fine-Tuning App Dependencies
|
|
|
|
# Web Framework
|
|
fastapi==0.109.0
|
|
uvicorn[standard]==0.27.0
|
|
python-multipart==0.0.6
|
|
|
|
# ML Framework (Apple Silicon)
|
|
mlx==0.6.0
|
|
mlx-lm==0.8.0
|
|
|
|
# Mail Parsing
|
|
beautifulsoup4==4.12.3
|
|
chardet==5.2.0
|
|
|
|
# Database
|
|
aiosqlite==0.19.0
|
|
|
|
# Utilities
|
|
aiofiles==23.2.1
|
|
psutil==5.9.8
|
|
|
|
# Optional but recommended
|
|
huggingface-hub==0.20.3
|