Methodology

How the comprehension score is computed

Every number Fathohm shows comes from the arithmetic on this page. There is no model in the scoring loop, no learned weights, and nothing that varies between repositories. The same history always produces the same score, and if you disagree with a number the disagreement should land on a specific step below rather than on the score.

We publish this because a trust metric nobody can audit is not a trust metric. It also means you can point at the part you think is wrong — which is the intended use of this page.

The formula

Each file gets a score from 0 to 1. Four factors, fixed weights, no interaction terms:

score = 0.40 × human_review_depth
      + 0.25 × human_author_recency
      + 0.25 × bus_factor
      + 0.10 × question_answerability

blind spot  =  score < 0.30
maximum achievable today  =  0.90

The maximum is 0.90 rather than 1.00 because the fourth factor is currently held at zero on every file. That is explained below, and it is deliberate.

What each factor reads

All four read the record — what was written, reviewed and by whom — and none of them ask what anyone knows. That is the central limitation of this instrument and also the reason it can be checked.

Human review depth · weight 0.40

How much a review demonstrated, priced on what was written rather than on how many times someone typed. For each review event:

units  =  body_characters / 240  +  inline_thread_comments
       =  at least 1 if the review BLOCKED the merge (changes requested)

depth  =  max over the file's reviews of  min(1, units × 0.25)

So 4 units is full credit. A one-word approval earns about a fiftieth of one unit; a paragraph earns one; an inline comment on a specific line earns one. A review that requested changes floors at one unit whatever its length, because a reviewer who stopped a merge found something concrete enough to stop it over — evidence no character count can supply, and the terse expert is exactly the case a length-priced body scores worst.

The max, not the sum. A file's depth is its single best review, not its review history added up. This under-credits iterative dialogue: a reviewer who comes back three times with one substantive comment each scores the same as one drive-by comment. We think that is wrong and it is the first thing on the roadmap; it needs a way to group a pull request's rounds, which the data model now carries and the scorer does not yet use.

Human author recency · weight 0.25

When a human last meaningfully wrote in the file.

recency  =  max over events of  w(e) × (1 − (days_ago / 180)²)

w(e)  =  1.00   a human hand-wrote this commit
         0.25   a human prompted an agent and committed the result
         0.00   an agent or bot authored it
         0.00   an approval with nothing written

The curve is eased rather than linear: understanding fades slowly at first and steeply later. In practice, code one person hand-wrote and nobody reviewed stays above the blind-spot line for roughly two months.

The w(e) ladder is the one place the product takes a position that is not purely mechanical: prompting an agent is real engagement with a file, and it is not the same engagement as writing the code. A quarter is a judgement. It is also the number most worth arguing with.

Bus factor · weight 0.25

bus_factor  =  min(1, sum over people of their best w(e) / 3)

Three humans with real contact is full credit. Someone who only ever prompted an agent here contributes a quarter of a person, by the same ladder as recency.

Question answerability · weight 0.10, currently scored 0

This factor contributes nothing to any score today, on purpose, and the story is worth telling because it is the kind of thing a methodology page usually omits.

It used to be earned by opening a file, answering three questions about it, and rating how well you could answer. That last step made the score's input the subject's own opinion of itself. It was worth up to 0.10 — enough to carry a lightly-reviewed file across the blind-spot line by clicking three buttons. An organisation could improve its own headline without touching its code.

That contradicts the one sentence this product is built on: measured from the record, not a survey. So we removed it rather than ship it, and left the weight held open and unearned rather than redistributing it to the other three — redistribution would have quietly inflated every score in the fleet and hidden that anything happened. It is re-earned when an answer can be read by someone who did not write it.

The constants, and where they came from

Four numbers decide everything above. None is learned, none is repo-relative, and each is a definitional choice we are accountable for.

How 240 was checked

Before measuring anything we wrote down a rule, so that checking the constant against reality would be a test rather than curve-fitting: keep 240 unless the corpus median review body falls outside half-to-double it, and if it does, snap to that median.

The rule fired, and it was void. The median non-empty review body across the repositories we measured is about five characters, so the rule prescribed snapping the constant to zero — which would define zero characters as a full unit of engagement, exactly the defect the constant exists to remove. The rule had checked a substance threshold against the median of a population dominated by sub-threshold reviews, which is a category error, and the arithmetic said so by going undefined.

Re-run against the population it should have used — reviews that blocked a merge, which are substantive almost by definition — the medians are 92, 166, 239 and 249 characters. 240 stands. Recorded honestly: that sample is thin where it is highest, one repository sits below the band, and if 240 is wrong it is generous — meaning we overstate comprehension debt rather than understate it.

What the score is not

It is not a measurement of understanding. Understanding is a state of a human mind and does not appear in a repository. Every factor above describes the record: written, reviewed, explained. Where this page says a file is a blind spot, it means no human has recently done any of those three things to it. It is a statement about the repository's history, not about anyone's head, and the difference is the whole reason the number is checkable.

It is not a measurement of a person. There is no per-engineer score, and there will not be one. Comprehension debt is a property of a codebase and the team around it; a per-person version of this number would be a performance-review weapon built from a proxy, which is a worse idea than the metric is a good one.

It is not validated. This is the honest limit and the most important sentence here. The arithmetic is deterministic and every constant is published, but nobody has yet checked whether a file scoring 0.2 is genuinely less understood than one scoring 0.6. We can show the number is consistent and decomposable; we cannot yet show it is correct. Making that falsifiable — peer-verified explain-back as the ground truth the factors are checked against, rather than as a factor — is the work that matters most, and it is why the fourth weight is being held rather than reassigned.

Gaming it

Any proxy is gameable and this one is no exception. The sharpest version: four inline “nit: typo” threads currently earn what one substantive paragraph earns. That is true, and we would rather say so than pretend a cleverer count would fix it.

Two things make it less attractive than it looks. Nobody is scored against this number, so gaming it buys nothing from anyone. And the real answer is not a better heuristic — a nitpick detector is an arms race this metric should decline — but verification: a check read by someone who did not write it, which is the same mechanism that will retire the zero in the fourth factor.

Where the data comes from

Commit and review metadata only: paths, authorship, timestamps, review lengths and comment counts. Source code is read transiently to write a module summary and is never stored. The GitHub App requests read scopes only — it never writes to your repositories, never opens pull requests, and never runs your code.

Authorship detection is heuristics-first: co-author trailers and known agent committer signatures, with the allowlist published so you can dispute any entry. A language model classifies only the ambiguous residue, behind an interface that is never load-bearing for a score — remove it and every number above still computes. Labels are correctable, and corrections are stored as events rather than edits.

Events are append-only. Scores are derived from them and recomputed whenever the scorer improves, so a better instrument re-values all of your history rather than only what happens next.

Changes to this methodology

The scorer carries a version stamp, every score row records which version produced it, and a re-valuation writes down what moved and by how much. When we changed how review depth is priced, two of six public gallery repositories moved and four did not; the largest move was three percentage points. Those numbers are published rather than swapped silently, and so is the reason.

You can see the decomposition behind any file's score in the public gallery, which runs this exact arithmetic against well-known open-source repositories. If you think a specific number is wrong, that is the fastest place to show us.