Methodology
How Rankings are scored — formula version v2. The weights and constants below are versioned in our git repository; any change bumps the version and is noted here.
The integrity policy
- No pay-for-placement, ever. There is no “featured”, “sponsored”, or “promoted” field anywhere in the Index data model — a paid position is not just against policy, it is structurally impossible.
- Sponsors, when they exist, buy labeled media — never rank. Money is not an input to any formula on this page.
- Scores are recomputed live from raw data on every view. Nothing is hand-adjusted; changing the formula changes it for every agent at once, in public.
- CodexMaster is independent and not affiliated with any tool it ranks.
The formula (v2)
Each agent's score is a weighted blend of three components, each normalized to 0–1, totaled to 100:
score = 100 × ( 0.4 × objective + 0.4 × sentiment + 0.2 × opinions )
| Component | Weight | Normalization |
|---|
| Objective (hard data) | 40% | Split evenly between two hard-data signals, 0.2 of the total each. Pricing value = 1 − (typical monthly spend / $200/mo), floored at 0; free and open-weight tools score 1. Release cadence = releases logged in the last 90 days ÷ 12, capped at 1 (≈ weekly shipping earns full marks). Missing data scores a neutral 0.5, marked "provisional" — never zero. |
| Sentiment (your votes) | 40% | Better/same/worse votes (+1/0/−1) read over three trailing windows — 24 hours at 0.45, 7 days at 0.35, 30 days at 0.2 — so today's mood moves the number fastest without throwing history away. Each window is shrunk toward neutral 0.5 by a 10-vote prior, so a handful of votes can't swing the ranking. |
| Opinions (community discussion) | 20% | Derived sentiment from public community discussion, over 7 days at 0.6 and 30 days at 0.4. A window backed by fewer than 5 scored posts is hidden entirely rather than averaged — see "Discussion sentiment" below. |
v2 rebalances toward what the market is doing right now: hard data and your votes carry equal weight (40% each), with community discussion contributing the remaining 20%. Where a sample is too thin to be honest, the component falls back to a neutral 0.5 and is labeled “provisional” rather than presented as a real reading. Every entity page shows its exact component arithmetic, so you can always answer “why is this #3?”.
How votes are handled
- One click: is this agent better, the same, or worse than yesterday? Votes count +1 / 0 / −1.
- Anonymous is fine. Visitors are identified only by a salted hash of IP + browser signature; the raw values are never stored.
- One vote per agent per visitor per day, enforced by a database constraint — not just the UI.
- Votes are an immutable, append-only event log. Rows are never edited or deleted (the database physically refuses), so any score can be recomputed from raw history at any time.
- Anti-abuse is automated, not moderated: 8+ votes from one visitor within 60s, or more than 20 votes in a day, are flagged at insert time and excluded from every aggregate (they remain in the log). Beyond 40/day, votes are rejected outright.
Discussion sentiment
Separate from the votes above, entity pages can show a community discussion sentiment reading (version sent-v1). It is derived, not voted: our enricher reads public discussion and estimates how practitioners feel about each agent.
- Source — public community posts only: Reddit, Hacker News, X, and Discord threads that mention a given agent. Never private data, never the vote log.
- Scoring — a language model (Gemini 2.5 Flash) rates each tagged post from −1 (strongly negative) to +1 (strongly positive) toward the agent it discusses, judging only that post's own title and text. Posts with no clear sentiment are recorded as “unclear” and left out of the average.
- Aggregation — the per-post scores are day-bucketed and blended into an item-weighted mean over the last 30 days. Every score carries its version, so any reading is reproducible.
- Honesty threshold — we hide the reading entirely until at least 5 posts back it. A handful of posts is noise, not a signal, and we would rather show nothing than a shaky number.
- It moves the rank — at 20%, and no more. As of v2, discussion sentiment feeds the Opinions component of the formula above. It is still not a vote, and it is deliberately the smallest of the three terms: what people say is evidence, but weaker evidence than what they pay, what ships, and how they vote. Earlier versions excluded it entirely.
Where the data comes from
- Pricing — maintained by hand from public pricing pages, with an append-only change log per agent. Entries still awaiting verification are labeled “data unverified” on the Index.
- Releases — entered manually at this stage (automated feeds later). An empty release radar scores neutral, never zero, so no agent is punished for our data lag.
- Vibe — you. The vote stream is the only component vendors can't touch and can't buy.
Spot an error? Corrections beat complaints: tell us and we'll fix the data, publicly.