How ModelTree decides what to record.

What earns a record a place

Inclusion in the catalog and selection for the homepage are two different decisions, and this section states the first one. The procedure is recorded besidedatasetSchema in web/src/data/schema.ts, published here word for word so the page and the schema cannot say different things:

What earns a record a place in this dataset, and what keeps it out. Apply in order: record exactly one entity kind per record, so a fact about a creator, a family, a release, a product, a serving platform, a source, or a publisher lives on that entity and never on a neighbour; cite at least one primary source and carry the day it was read, which every record-bearing schema above requires of itself rather than leaving to judgement; leave a field unset when no cited source states it, because a blank is a fact this dataset publishes happily -- nobody has sourced this yet -- and a plausible value no source states is not a fact at all; withhold the whole record when its required fields cannot be sourced that way, and record the gap rather than the guess, so that what is missing stays visible instead of being smoothed over; and admit the record only as a reviewed change to this repository, never as runtime input and never as an open crawl, per ADR 0002. Inclusion decides presence and nothing else. It states no order, no score, and no rank; it is not the featured procedure recorded beside that field, which is applied afterwards and only to releases already admitted here; and a record admitted by this procedure gains its catalog entry, its canonical route, and its correction path whether or not any editorial list names it.

The two decisions run in that order, and only that order: a record is admitted on the evidence behind it, and only then does the editorial list below decide where a reader meets it. Nothing about being admitted argues for being featured, and nothing about being left off the featured list removes a record or any of its pages.

Featured versus complete coverage

Coverage is deliberately incomplete. Complete historical coverage is a stated non-goal at launch, so a model’s absence is not a judgement about it. The homepage and Model Tree present a curated slice under the labels“Featured ecosystems” and “Widely used model families”, never an unsupported usage ranking.

“Featured” is a per-release editorial flag, not a score. The schema requires any release marked featured to carry a written featuredRationale, and the dataset validator requires a featured release to cite at least one primary source. Featured placement is editorial and non-ranked; it derives from reviewed catalog flags and does not imply popularity or rank.

The criterion for the Model Tree’s Featured branch is the decision procedure recorded beside the featured field inweb/src/data/schema.ts, published here word for word so the page and the schema cannot say different things:

Editorial lead selection, not a ranking and not a sourced claim. Apply in order: flagfeatured only on a release whose creator is one of the five this site leads with -- anthropic, google-deepmind, meta,microsoft, openai; flag at least one release for each of those five, so that each one reaches the Featured branch, because a creator is featured exactly when it holds a featured release and the schema carries no organization-level flag; flag no release of any other creator, which is what places every creator the list omits on the Others branch; write afeaturedRationale on exactly the releases flagged, so that no rationale outlives the placement it explains; and let no lifecycle status decide the flag in either direction: status is a sourced measurement, and deriving this list from it would make the list track recency, which is an order this procedure refuses -- so a legacy release may stay flagged and a current one is not owed the flag, and what a flagged release owes instead is afeaturedRationale saying why it carries its creator's placement, in terms that stay true once it is superseded and that could not be written of another release of the same creator. The list records what this site leads with, which is a choice about its own entry point rather than a measurement of the creators: it states no order, no score, and no claim that a listed creator is larger, better, or more important than one it omits. A creator the list omits keeps every catalog entry, every release, its place on the Others branch, and its own provider page. Changing the five is an editorial change to this list, reviewed like any other change here.

Featuring is therefore a statement about where this site starts a reader, not a claim about a creator’s size or standing or about the quality of its models. It is also not permanent, and it is not a filter on the data: a creator the list omits is recorded here exactly as fully as one it names, with its own provider page, its releases in the full catalog, and its place on the Model Tree’s Others branch. What changes is which creators the homepage opens on.

Editorial inclusion may consider consumer reach, API availability, cloud distribution, open-weight adoption, developer-tool integration, and sustained release activity. None of these is combined into a number.

A reviewed set, not a ranking

Beyond the featured ecosystems, long-tail creators are a reviewed set, not an open crawl: ADR 0002 records that a long-tail profile is a reviewed artefact of this repository rather than operator input. The Model Tree’s “Others” branch carries that reviewed long-tail coverage, as the same expandable creator, family, and release disclosures the Featured branch uses, and every release under it is listed in the full catalog exactly as any other release is. Inclusion is a reviewable repository change like any other fact here.

Known coverage gaps

Coverage being incomplete is only honest if the incompleteness is visible, so the gaps are published rather than left to be inferred from an absence. Each automated refresh run records what it looked at and did not add, and the reason it gave on the day; those records are the list below, derived from the runs themselves so there is no second list to keep true. Across every recorded run, 439 such decisions stand:

• Rejected by the review panel — 204 (rejected-by-panel)
• Accepted by the panel, then dropped — 127 (dropped-after-acceptance)
• Verification date deliberately held back — 20 (verification-held)
• Sources conflict, so no value changed — 9 (conflict-recorded)
• Source refused by the approval gate — 24 (source-refused)
• Blocked by policy before it could run — 48 (blocked-by-policy)
• Out of the run’s reach — 7 (not-covered)

Read these as history, not as a live inventory: each says that a particular run did not add a particular thing, which stays true forever. It does not say the gap is still open, and several have since been closed by ordinary reviewed changes — a creator an automated run had to withhold for want of an approved origin is exactly the kind of record a reviewed change may later add on its own evidence. The reason a run gives is kept whichever way that goes, because a withheld record and the reason for withholding it are the part a bare count of what is present cannot show.

Separate entities

A single model name blurs together several distinct things. ModelTree models them as separate entity types and never collapses one into another:

• Creator (organization) — who built the model.
• Model family — the line a release belongs to.
• Release — one dated, versioned model variant.
• Product — a user-facing offering, which may route between models rather than name one.
• Serving platform — where a model is served, usually operated by someone other than the creator.
• Publisher — who stands behind a source, an entity in its own right rather than a free-text label.

These boundaries are enforced, not just described. A deterministic gate rejects a release attributed to one organization while its family belongs to another, because that silently reassigns authorship. Publisher identity is resolved by id and ownership rather than by display name, so two corporate siblings collapse to one voice and two unrelated publishers that share a name stay distinct.

Keeping the types separate is a schema guarantee, not a claim that each is populated today. Organizations, families, releases, publishers, sources, productsand serving platforms carry records now; pricing is defined in the schema — defaulting to an empty list — and still holds no data. The absence is recorded honestly rather than filled with guesses.

Editorial functional classification, not a sourced claim. Choose the first match:community when independent contributors outside any one entity’s employment or appointment chain can initiate and decide its model releases, not merely submit work;company when the entity offers model products or access for payment under its name (a parent’s sales do not count); research-lab when one standalone institution or named unit controls releases and exists primarily for research; nonprofit when a centrally governed nonprofit matches none above; otherwise company for the centrally operated creator that runs the model work.An organization’s required type therefore classifies function rather than legal form and is not a ranking. Reviewers apply that observable decision procedure rather than requiring a primary-source quote to use the category’s exact words.

Lifecycle statuses

Every release carries exactly one lifecycle status, drawn from the schema’slifecycleStatus enum and shown with the same label the rest of the site renders. Note that current displays as “Available”.

Access and licensing

Access and licence are separate claims. Downloadable weights and an OSI-approved licence are recorded as two independent booleans on the licence record, soopen-weight does not imply open-source. The schema enforces the coupling that does hold: a release whose access type claims weights must carry a licence marked downloadable.

osiApproved must rest on a source that states OSI approval — OSI’s own published licence list at opensource.org. An spdxId or a licenceurl alone is not evidence of OSI status: where the schema requires one, that requirement is a structural floor — it ensures a licence is identified — not the evidence rule for the field’s truth, which is the reviewer’s to apply.A validator can check that a licence is identified; whether OSI approved it is a fact about the world that only a source states.

Categories describe a release’s documented focus. They are labels, never summed or weighted into a ranking.

Source types and priority

Every important fact carries a source and a verification date; the evidence gate rejects a sourced record that carries no source id or no usable verifiedAt. A source’s URL must be HTTPS, free of embedded credentials, and not a local or internal host. Sources fall into two tiers: primary sources — the creator’s own announcements, docs, model cards, and repositories — and everything else. The split below is read from the validator’s own PRIMARY_SOURCE_TYPES set.

Priority is enforced where it matters: a featured release must cite at least one primary source, and a usage figure a creator published about its own model is refused unless it carries a primary source. Among several primary sources describing one release, official documentation is preferred — this last preference is editorial policy, not a rule enforced in code: no source-ordering logic exists in the schema or validators.

Conflicting and incomparable evidence

Disagreement stays explicit. Conflicting readings are linked to each other and shown side by side; nothing picks a winner. A cross-source usage synthesis may report that readings agree or conflict, but it may not resolve the conflict, and it requires at least two independent non-creator publishers — counted by controlling company, so an arm of the creator is not independent and two unrelated publishers that share a name are not merged. Two figures that measure different metrics, units, or populations are treated as incomparable rather than contradictory, because recording them as a conflict would imply a shared scale they do not have.

Dates and partial precision

A date is recorded only at the precision a source states. The schema acceptsYYYY, YYYY-MM, or YYYY-MM-DD and records which, so a year is never silently read as 1 January. Deterministic date gates reject impossible dates, dates in the future, a last-checked date earlier than a published date, a release earlier than its family’s first release, and a release earlier than a model it descends from.

Some families have no first release date to record at any precision: no primary source states when they began. That is a different claim from a vague one, so it is written down as one — those families carry a precision of unstated and readNot stated by any source rather than a blank or an inferred year. Omitting the date without saying so stays a hard failure, and a record cannot both claim the date is unstated and carry one.

Verification and freshness

Every sourced record carries a verifiedAt date — the day the fact was last checked against its source — and the site footer shows the most recent one. A Model Passport marks a figure stale against the build date rather than the day it happens to be read, so freshness does not drift silently.

The most recent date alone would be a poor freshness claim, because a single record checked today sets it however long everything else has gone unchecked. So the span is stated instead: verification across this dataset runs from Aug 26, 2026 to Sep 12, 2026. The older end is the honest half of that pair, and the per-record date remains the only thing to trust for any particular fact.

What this page will not promise is a fixed verification cadence the maintainers cannot guarantee. ADR 0003 authorises an automated daily refresh, and each run is recorded under Data refresh, but that is a capability, not a service-level guarantee. Trust the per-record verification date, not a schedule. The evidence gate is also honest about its own limit (ADR 0005): it checks that a citation is well-formed — a quote of sufficient length and a correctly-shaped content hash — but it fetches nothing, so it cannot confirm the quote appears on the page or that the hash is the page’s. What compensates is the pull-request trail, the revert path, the dataset-only scope of automated changes, and human review after the fact.

Pricing

A price is recorded with everything needed to read it correctly and nothing that would distort it. Each pricing record preserves its currency (a three-letter ISO 4217 code), its unit (such as per-1M-tokens or per-image), itsprocessing tier, its region, and itseffective dates. Rates are never converted between currencies or units; a figure is shown as its source stated it.

Benchmark comparability

Benchmarks are contextual evidence, not universal truth. There is no composite or universal score; a deterministic gate refuses any field that reads as a ranking anywhere in the dataset. Each benchmark result records the configuration below where its source discloses it — the schema’s own comment marks these as “configuration that decides whether two results may be compared at all” — and the validator refuses two results for the same model under an identical disclosed setup and requires a result’s unit to match its benchmark’s unit.

Several of these fields are optional, and disclosing that matters most on the page that defines the evidence policy. reasoningMode, toolsEnabled and harness are defined in the schema but currently unrecorded on every result, so two results are told apart only by the fields that are present — benchmark, version, and variant. An unrecorded field is not evidence that two runs matched on it, the same posture the dataset takes toward every unknown fact.

The worked examples below show when two results may be read on the same axis and when they may not. They are derived from real benchmark results in the dataset, not authored here: each verdict is computed from the disclosed configuration the schema records — two results are comparable only when every configuration field except the model matches — so the page cannot state a comparison the data does not support. No score is shown; the point is which recorded differences make two results non-comparable, not who scored higher.

Even that comparable pair matches only as far as the disclosure goes. Neither result records reasoningMode, toolsEnabled, harness, each optional in the schema, so the comparison rests on those conditions being equal rather than on evidence that they are. Absence of a recorded difference is not evidence of sameness — the same posture the dataset takes toward every unknown fact.

Not yet enforced. How benchmark results run under *different* disclosed setups are normalised or transformed so they can still be compared — beyond recording each result’s configuration and refusing duplicate results under an identical setup — is not implemented. That policy is issue #22, which itself depends on #21 for benchmark seed data. Until it lands, this page illustrates which recorded configuration differences make two results non-comparable, but states no rule for reconciling those differences, because none yet exists to describe. See Benchmark comparability and evidence transformations (#22).

Usage evidence

Usage figures are labelled by how they were produced, and the label is a provenance statement, never a quality ranking of the source. Creator self-reports are kept in their own labelled list and never counted toward a cross-source synthesis.

Model-fit guidance and no universal ranking

ModelTree offers conditional model-fit guidance, never a verdict. Every statement is filed as exactly one of three kinds, each tied to a condition and backed by facts the dataset already records. The schema rejects universal-winner language outright, and this page’s own prose is held to the same filter.

Where a rubric dimension has no qualifying evidence, the gap is recorded with a reason rather than left to read as a silent negative.

Reporting a correction

A data correction is an ordinary pull request — that is the point of treating data as reviewable repository content. If you cannot open a pull request, file an issue: be specific about what is wrong, cite a primary source, and say what is explicitly out of scope.

Report a correction or contributionon the project repository. Every accepted change lands as a reviewable diff, and every automated run is transcribed under Data refresh.

How automated changes are reviewed

Most changes reach main only after a human merges them. One bounded class is different: under ADR 0003, a source-backed dataset refresh produced by the review skill set may open a pull request and auto-merge without a human approving it — but only the dataset JSON documents may change that way, and every other change keeps the human merge. That authorisation is not a loosening of review:

• Three reviewer sub-agents with distinct rubrics judge each claim against its fetched evidence — acceptance is two-of-three for a reviewed creator and unanimous for a long-tail one.
• Deterministic gates run after review and cannot be outvoted: unsafe URLs, malformed records, impossible or future dates, broken references, entity-boundary and lineage violations, an unapproved source, or any composite-score vocabulary each fail the run.
• GitHub performs the merge once the required checks are green; the agent never pushes to main, and a failed deploy is reverted.

So automation removes the human reviewer for one narrow class, not review itself. Everything outside that class takes the ordinary path.