Home Media AI

Open-source ML system for organizing and curating 200K+ personal photo collections

Quick Takeaways

Problem: Finding “great photos” in 200K+ personal collections—combining objective technical metrics (sharpness, exposure) with subjective aesthetic judgment

Solution: Open-source ML system using computer vision, YOLO object detection, and hierarchical classification to surface best images

Philosophy: The intersection of art and engineering—translating professional photography judgment into algorithmic rules that scale

Key Insight: ML should multiply human capability, not replace it—review top 2,000 candidates instead of 20,000 images

Technical Approach: Python/SQLAlchemy, MariaDB, OpenCV/Pillow, YOLO for people/pet detection, scikit-learn for quality models

Status: Active development (43+ commits), verified on 700K+ file collections, open source on GitHub

GitHub → | Related: Photography Philosophy →

The Challenge: Finding Signal in 200,000 Images

Most families accumulate thousands of photos each year. My household averages 20,000+ annually—vacations, birthdays, pets, everyday moments. That’s over 200,000 images spanning a decade.

Every year, my wife goes through this entire collection to curate our family calendar. She’s looking for:

  • Technical quality - Sharp focus, good exposure, balanced composition
  • Aesthetic appeal - Compelling moments, good light, visual interest
  • Semantic relevance - Birthday photos for birthday months, holiday images for December, pictures featuring specific people or pets

It’s meaningful work—these images tell our family’s story. But manually reviewing 20,000 images is exhausting.

The core problem: How do you automatically identify “great photos” when “great” combines objective technical metrics (sharpness, exposure) with subjective aesthetic judgment (compelling composition, emotional impact)?

The Solution: ML-Powered Photo Curation

Home Media AI is an open-source system that combines computer vision, machine learning, and hierarchical classification to surface the best images from massive personal collections.

Core Capabilities

1. Automated Quality Assessment

  • Technical metrics: sharpness (edge detection), exposure balance, noise levels
  • Compositional analysis: rule-of-thirds compliance, negative space, subject positioning
  • Duplicate and near-duplicate detection (checksummed file tracking)

2. Rich Semantic Search

  • Hierarchical classification support (people, pets, places, events)
  • Works with any taxonomy: biological classification (World Flora Online integration), geographic hierarchies, custom family categories
  • Relationship tracking: “Find photos of [person] at [location] during [time period]”

3. Scale-Tested Architecture

  • Verified to handle 700K+ file collections
  • Files stay in original locations—metadata tracked separately
  • Relationship mapping between originals and edited versions
  • Cross-platform: Windows, Mac, Linux path support

4. Long-Term Reliability

  • Everything checksummed and versioned
  • Auditable: never moves or deletes files
  • Handles duplicate detection across drives and backups

The Goal: Multiply Human Curation

The system doesn’t replace human judgment—it multiplies it. Instead of reviewing 20,000 images, curators review the top 2,000 candidates the ML system surfaces. The human still makes the final decision, but with 90% less exhaustion.

Technical Architecture

Backend Stack

  • Python with SQLAlchemy ORM for flexible data modeling
  • MariaDB 10.11+ (UTF-8mb4 support for international text/emoji)
  • NumPy/Pandas for data processing pipelines
  • Scikit-learn for ML classification models

Media Processing & ML

  • OpenCV for image analysis (edge detection, histogram analysis, feature extraction)
  • Pillow for format handling and EXIF data extraction
  • exifread for metadata parsing (camera settings, timestamps, GPS)
  • YOLO (You Only Look Once) for real-time object detection and classification
    • Identifying people, pets, and objects in images
    • Pre-trained models fine-tuned on personal photo collection
    • Enables semantic search: “Find photos with our dog at the beach”

Development Environment

  • Jupyter Lab for exploratory data analysis and model tuning
  • Mamba for reproducible environment management
  • Phase-based development: Core discovery (Phase I) → ML models (Phase II) → Web interface (Phase III)

Database Design Philosophy

The system uses a relational database to track media and their relationships to classification hierarchies:

MediaFiles (checksummed tracking)
  ↓
MediaRelationships (originals → edits)
  ↓
ClassificationLinks (files → hierarchical taxonomies)
  ↓
TaxonomyNodes (hierarchical classification systems)

This design allows:

  • Multiple simultaneous classification systems per media item (e.g., a photo can be tagged with biological taxonomy AND family event categories)
  • Version tracking (linking RAW → edited JPG)
  • Flexible hierarchies (person/pet names, locations, dates, custom categories)

The Intersection: Art Meets Engineering

This project embodies the “intersection philosophy” that drives my technical work:

Artistic Training → Technical Implementation

My background as a professional photographer taught me what makes a photo compelling:

  • Compositional balance (rule-of-thirds, leading lines, negative space)
  • Light quality (golden hour, directional lighting, shadow/highlight balance)
  • The decisive moment (timing, expression, action peaks)

Translating these intuitions into algorithmic rules is the challenge:

  • “Good composition” → edge detection patterns, symmetry analysis, subject positioning heuristics
  • “Compelling light” → histogram analysis, dynamic range metrics, color temperature evaluation
  • “The decisive moment” → temporal burst analysis, facial expression detection, motion blur assessment

Human-Centered ML

The system is designed around a core insight: ML should amplify human capability, not replace human judgment.

My wife has curated our family photos for years—she has taste, context, and emotional connection that no algorithm can replicate. The goal isn’t to automate her out of the process. It’s to give her back hours of her life by surfacing candidates worth considering.

This philosophy applies to all my ML work: physics-informed neural networks that encode domain knowledge, GPU-accelerated pipelines that augment scientist workflows, explainable models that support human decision-making.

Implementation Status & Roadmap

Phase I: Core Infrastructure (In Progress)

  • Database schema design and migration system
  • Media file discovery and checksumming
  • Cross-platform path handling
  • Duplicate detection logic
  • Configuration management
  • Complete EXIF/metadata extraction pipeline

Phase II: ML Classification (In Progress)

  • Quality assessment models (technical metrics)
  • Aesthetic scoring (compositional analysis)
  • YOLO-based object detection - Real-time identification of people, pets, and objects
    • Fine-tuning pre-trained models on personal photo collection
    • Enabling semantic queries: “Photos with [person] and [pet]”
    • Bounding box extraction for face/subject cropping
  • Hierarchical classification integration
  • Model training pipeline for quality/aesthetic scoring

Phase III: User Interface (Future)

  • Web-based curation interface
  • Search and filter capabilities
  • Calendar generation automation
  • Batch export and sharing tools

Phase IV: Community Features (Future)

  • Shared classification taxonomies
  • Pre-trained aesthetic models
  • Plugin system for custom classifiers
  • Documentation and tutorials

Real-World Impact

Current Use Case: Family photo archival and calendar curation

  • 200K+ images spanning 10+ years
  • 20K+ annual additions
  • Handles multiple cameras, phones, and formats
  • Preserves original files while tracking edits and versions

Potential Applications:

  • Nature photography organization - Integrate with biological taxonomies (World Flora Online, eBird)
  • Travel documentation - Hierarchical geographic classification
  • Technical equipment cataloging - Product photo organization
  • Professional portfolio curation - Quality-based filtering for client galleries
  • Digital archival projects - Long-term family/organizational media preservation

Open Source & Community

The project is open source on GitHub with active development (43+ commits).

Why open source?

  • Personal media archival is a universal challenge
  • Shared taxonomies and models benefit everyone
  • Community contributions accelerate development
  • Transparency builds trust for family archival systems

Contributing: The project welcomes contributions—whether suggesting features, reporting issues, contributing classification taxonomies, or improving ML models. Check the repository for current priorities and development guidelines.

Technical Highlights

Modern ML: YOLO for Object Detection

  • Real-time object detection using YOLO (You Only Look Once)
  • Pre-trained on COCO dataset, fine-tuned on personal collection
  • Enables semantic search beyond simple keyword tags
  • Example queries: “Photos with both kids and the dog”, “Images featuring specific people at specific locations”
  • Bounding box data stored for future cropping/thumbnail generation

Scale-Tested Design

  • Verified on 700K+ file collections
  • Efficient database indexing for fast queries
  • Incremental processing (only analyze new/changed files)

File Integrity

  • Checksummed tracking prevents duplicates
  • Never moves or deletes original files
  • Separate metadata database keeps files untouched

Flexible Classification

  • Works with any hierarchical taxonomy
  • Example: World Flora Online (WFO) plant classification integrated
  • Custom taxonomies for family/personal categories

Cross-Platform

  • Native Windows, Mac, Linux path support
  • Handles network shares and external drives
  • Docker deployment for portability

Lessons Learned: Building ML for Real Users

1. Perfect is the Enemy of Shipped

Early versions focused on sophisticated aesthetic models. Reality: even basic quality metrics (sharpness, exposure) provide 80% of the value. Ship incrementally.

2. Respect User Workflows

Users already have folder structures, backup routines, and editing workflows. Don’t force them to change. The “files stay put” design principle came from listening to real needs.

3. Explain Decisions

ML models that rank photos should explain why. “High quality” is less useful than “Sharp focus, good exposure, rule-of-thirds composition”. Explainability builds trust.

4. Build for Maintenance

Personal photo collections span decades. The system must be maintainable long-term. Clear database schema, versioned migrations, and minimal dependencies matter.

These lessons apply beyond family photos—they’re principles for any production ML system.

Future Directions

Short-Term

  • Complete Phase I media discovery pipeline
  • Implement basic quality assessment models
  • Build proof-of-concept web interface

Medium-Term

  • Train aesthetic models on curated personal dataset
  • Integrate face detection for person-based search
  • Automated calendar generation from top-ranked images

Long-Term

  • Federated learning across multiple family collections (privacy-preserving)
  • Mobile app for field tagging during photography
  • Integration with cloud photo services (Google Photos, iCloud)
  • Community-curated aesthetic models

Links & Resources

GitHub Repository: https://github.com/ericwait/home-media-ai

Related Projects:

Technologies:

  • Python, SQLAlchemy, MariaDB
  • OpenCV, Pillow, NumPy, Pandas
  • YOLO (object detection), Scikit-learn, Jupyter Lab

How This Connects

Intersection Philosophy: See how art meets engineering across my work → Photography

Human-Centered ML: Explore systems that multiply human capability → Home page

Professional Photography Background: Learn how artistic training shapes technical work → Photography and Full Journey

Building What Lasts: Compare with long-term development practices → MATLAB Utilities

Technical Approach: See related ML systems and GPU acceleration → Hydra Image Processor

Design Philosophy: Read about making optimization accessible → About

Building tools that respect both the art and the scale of personal photography—because every family’s story deserves to be found.