ImageAnalysis

I have a lot of images that are not for publication.

Photos, screenshots, receipts, scans, reference images, workshop evidence, product labels, places, objects, portraits, little reminders to future me. Different file types, same problem: I saved the image for a reason.

That reason is obvious when I save it. Later it often is not.

ImageAnalysis is my attempt to make those images searchable without sending them to a public cloud service, and without pretending that a folder name is enough.

Why this exists

Photo libraries handle dates, locations, faces, and albums. Filesystems handle ownership and long life. My later question is usually messier:

Where was that thing I wanted to build?

Or:

Show me the photos with clamps, lino cuts, coffee places, receipts, odd mechanisms, certain persons, certain motives, or that useful shelf detail.

The image might be in a camera folder, a chat export, a screenshot folder, or an old sync directory. Moving it into the perfect folder does not solve much. It creates another classification decision that I will probably disagree with in six months.

So the first rule is plain: describe the image before sorting it.

Each image should get a useful title, a grounded description, visible facts, OCR when there is text, search phrases, tags, and a cautious guess at why it was captured.

Normal private photos do not need an ignore bucket. They are private, yes. That does not make them meaningless.

Capture reason

The capture reason is not the visible subject.

A photo of a chair might be:

• something to buy
• something to build
• a construction detail to copy
• a place I visited
• a motive I wanted to remember
• a person I photographed
• a visual reference for drawing
• evidence of something finished

The pixels can suggest some of this, but they cannot know all of it. The system has to stay modest. It can propose an intent. It must not treat that proposal as archive truth.

Useful intent labels include:

• to-buy
• to-build
• to-make
• to-draw
• visual-reference
• place-visited
• person-photographed
• motive-study
• receipt-proof
• workshop-documentation
• unknown-intentional

That last one matters. If the system cannot infer the reason, the image should still be searchable. The fallback is not “ignore”. The fallback is “I probably had a reason, but the machine cannot see it yet.”

How it is split

There are two machines because the jobs are different.

Alice handles the archive side:

• scans folders
• computes stable hashes
• creates bounded analysis derivatives
• keeps the SQLite state
• leases and retries jobs
• stores results
• builds the search index
• writes approved metadata projections later

Bob handles the GPU side:

• receives derivatives over Tailscale
• runs OCR and vision-language analysis
• returns structured JSON
• stays disposable enough that the archive does not depend on it

Originals stay local. Bob sees derivatives, not a command to reorganize the archive. Public internet services are not part of the normal path.

The current test model is Qwen through Ollama on Bob. That can change. The contract is the useful part: title, description, OCR, visible facts, tags, people metadata when people are visible, a safety signal, capture intent, and search phrases.

Design choices

The system is built around a few boring constraints.

It must resume cleanly. Large photo collections are not a weekend script. Jobs need leases, retries, versioned prompts, versioned model outputs, and idempotence based on content hash.

The output must be reviewable. A model can help describe and index images, but it should not silently mutate the archive. Embedded XMP/IPTC metadata is only a small projection after review. The heavier private analysis belongs in the local catalog or in a FileFirst enrichment payload.

Folder categorization is only a projection. A folder can help with browsing, but it is a poor source of truth. One image can be place, person, motive, object reference, and future project input at the same time. Search handles that better than a single folder.

Privacy is metadata too. If people are visible, the system records coarse visible attributes such as perceived age range, perceived gender, and hair color. It does not identify people by name. A later trusted private catalog could add identity, but the vision model should not invent it.

Safety labels are operational, not moral. sfw, nsfw, or unclear is enough for filtering and review. It is not a judgment on why the image exists.

FileFirst boundary

ImageAnalysis is not the archive manager.

FileFirst is still the archive idea: files remain files, metadata travels beside them, and the filesystem stays understandable without a special app. ImageAnalysis sits at the edge. It enriches images, builds search, and prepares metadata drops that FileFirst can validate later.

The detailed FileFirst pieces live separately because they cover more than images:

• FileFirst Specification: archive identity, manifests, validation, and the longer design
• FileFirst Adoption Guide: when this filesystem-first approach is worth the trouble
• FileFirst Extensibility: how outside tools can add metadata without writing straight into archive truth

ImageAnalysis should use that boundary instead of inventing its own archive rules.

That keeps the archive boring. Good.

Current state

The first end-to-end tests work. Alice can queue images, Bob can analyze them through a Dockerized service, and the results can be indexed. Recent batches included normal mobile-photo folders as well as screenshots.

That changed the design in a useful way. Screenshots often look like tasks. Normal photos are broader, but still intentional. The system has to care less about folder labels and more about retrieval:

What is in this image?
Why might I have kept it?
How would I look for it later?

If those three questions are answered well, the folder can be almost boring.

See also: FileFirst, Processes, Workbench