# Media Tagging Problems

I recently took an interest in tag-based systems of file organization after realizing that my disjoint collections of documents, videos, and images have become rather too large and disorganized to navigate efficiently. Plenty of people have had the same realization, resulting in projects like [TMSU](https://tmsu.org) (now unmaintained), but all existing software falls short in one way or another. I want my tagging systems to do the following.

1. Integrate a thesaurus or thesauri along the lines of ISO Standard 25964-1, "Thesauri for information retrieval".
2. Provide for attributes, e.g. "author = Friedrich Engels" and "modified = 1970-01-01". This is technically distinct from tagging, but a good system needs both.
3. Provide for logical "entities" which can themselves bear tags/attributes and which each represent some aspect or component of a given piece of media. For example, an image of pair of cows might have two associated entities, one for each cow. If one cow is an unidentifiable brown, it gets the "brown" tag, while the other cow might get "holstein". A story about a man who gets lost in the woods could have an entity representing the man with the tag "fear".
2. Facilitate advanced queries by exposing a decent subset of SQL or, preferably, Datalog.

I'm no good at consistently coming up with the same term for the same concept, which makes searching a database of tags difficult. The point of a thesaurus is to ensure that the same term is always used for the same concept. A well-structured thesaurus also provides tag implication, where tagging an entity with "cow" gives it "female" and "mammal" automatically. This is critical for searchability down-the-line. Compound concepts also allow implication from conjunctions, e.g. "happy" + "crying" might yield "tears of joy" (in addition to "secretion", and more).

Even multiplicity inference/propagation takes place. If a frame has two entities, one with a "man" tag and another with a "woman" tag, containing entities will have multiple instances of the "person" tag. The ability for any entity (the frame, in this case) to have multiple instances of any given tag means we can do things like search for images that have exactly 4 people and trust that the propagation/inference will work correctly.



