Equivalent mutants

Audience: you have a survivor you cannot kill, and you are starting to wonder whether the tool is wrong.

Sometimes it is. More often the mutant is equivalent: the mutated program and the original are the same program, so no test can tell them apart. There is nothing to catch, and any test you write to “fix” it will be a test that asserts something arbitrary.

Why this cannot be automated away

Deciding whether two programs behave identically for all inputs is equivalent to the halting problem. It is not that nobody has written the check yet — the check cannot exist. Every mutation testing tool has this limitation, and a tool that claims otherwise is either guessing or restricting itself to mutations too weak to be worth making.

So the honest position is: moonbuggy will produce equivalent mutants, it cannot tell you which ones they are, and the judgement is yours. What it can do is give you enough information to make that judgement quickly, which is what the diff in moonbuggy show is for — and, when you would rather test the hypothesis than reason about it, moonbuggy run <id> re-measures that one mutant against the test you just wrote.

Recognising one

Three shapes cover most of them.

The unreachable boundary. The mutation changes a condition that cannot be reached in the changed state:

def take(items, count):
    if count <= 0:
        return []
    return items[:count]           # count > 0 guaranteed here

Mutating items[:count] in a way that only matters for count == 0 cannot be observed, because the guard above has already returned.

The redundant clamp. The mutation changes something a later step undoes:

def percentage(part, whole):
    if whole == 0:
        return 0.0
    return round(part / whole * 100, 2)

Mutating round(..., 2) to round(..., 3) is only observable for values whose third decimal place is non-zero. If every caller passes values that cannot produce one, the two versions are equivalent in this program — though that is a fragile property, and the tutorial shows one that looked equivalent and was not.

The initialisation that is immediately overwritten.

def first_match(items, predicate):
    found = None                   # mutating this constant changes nothing
    for item in items:
        if predicate(item):
            found = item
            break
    return found

If the loop always assigns before any read, the initial value is unobservable — except when the loop body never runs, which is exactly the case worth checking before you dismiss it.

Deleted statements

statement_deletion — the deep tier — produces more equivalent mutants than any other operator, because “this line can be removed” is true of more lines than people expect. The operator already refuses to generate the ones it can prove are equivalent: docstrings, pass, ..., global/nonlocal, a bare name or literal on its own line, and a local binding with a pure right-hand side that nothing in the enclosing function reads again. Anything it cannot prove, it generates — the alternative is declining to mutate lines that hide real gaps, and a missed finding costs more than a survivor you dismiss.

So expect survivors of this shape, and expect some of them to be genuine:

def load(path):
    path = Path(path)
    cache.setdefault(path, {})     # deleting this may be equivalent
    return _read(path)

Whether that setdefault matters depends on what _read does with a missing key, which is exactly the judgement no analysis can make for you. Ask the question in the next section, and when the answer is “nothing observable”, record it in the ledger rather than deleting the line — the ledger is what stops the next run asking you again.

Swapped arguments

argument_swap — also the deep tier — has one source of equivalents it cannot close, and it is worth knowing about before you read its survivors. Swapping two adjacent positional arguments is only observable when the two mean different things, and moonbuggy has no type inference with which to ask.

total = add(subtotal, shipping)    # swapping these changes nothing

The operator does skip the cases it can settle for free: a starred position, and two arguments identical as source (f(x, x), f(0, 0)). Anything past that is a judgement about the callee. Apply the test below, and when the answer is “the two arguments are interchangeable”, put it in the ledger — that is the case moonbuggy accept exists for, and it is the main reason this operator is opt-in rather than part of every run.

The test before you dismiss it

Before calling a survivor equivalent, answer this concretely:

Name an input for which the two versions produce different output.

If you can, it is not equivalent — it is a real gap, and you have just written the test case. If you genuinely cannot, and you have thought about the empty collection, the zero, the boundary and the negative, it is probably equivalent.

That question is doing real work. Most survivors that feel equivalent turn out to have an input that separates them, and it is almost always the boundary nobody tested.

Suppressing one honestly

When you are confident, suppress it at the line with a marker and a reason:

def first_match(items, predicate):
    found = None  # moonbuggy: skip -- overwritten before any read; None only
                  # survives for an empty `items`, which test_empty covers
    ...

The mutant is then reported SKIPPED rather than SURVIVED, and drops out of the set that needs attention.

>>> project = make_project({
...     "lib.py": "def scale(n):\n    factor = 2  # moonbuggy: skip\n    return n * factor\n",
...     "test_lib.py": "from lib import scale\n\ndef test_scale():\n    assert scale(3) == 6\n",
... })
>>> _ = moonbuggy(cwd=project)
>>> [(r["line"], r["status"]) for r in records(project) if r["suppressed"]]
[(2, 'SKIPPED')]

Two rules for suppression, and they are what separate it from lying:

Always write the reason. The marker without a justification is a claim nobody can check and nobody will revisit. # moonbuggy: skip alone will be read next year as “someone was in a hurry”.

Suppress the line, never the file. A blanket exclusion hides the equivalent mutant and every real gap that arrives on that line later. moonbuggy has no file-level suppression for this reason.

Logging calls

One family of unkillable mutant is common enough, and recognisable enough, that moonbuggy handles it for you. Nothing asserts on the contents of a debug line, so a mutation inside one survives every suite that will ever be written against that code:

logger.debug("retrying in %ds", delay * 2)

delay * 2 becomes delay / 2, the log line reads differently, and no test notices — nor should one. Left alone these dominate a survivor list: in the session that prompted this feature, two thirds of the survivors in a retry region were arithmetic inside logger.debug(...) arguments.

So a mutation inside a logging call’s arguments is tagged logging_call in results.jsonl and, by default, reported SKIPPED:

>>> project = make_project({
...     "lib.py": (
...         "import logging\n\n"
...         "logger = logging.getLogger(__name__)\n\n"
...         "def charge(n):\n"
...         "    if n > 10:\n"
...         "        logger.info('big charge: %d', n * 2)\n"
...         "    return n\n"
...     ),
...     "test_lib.py": (
...         "from lib import charge\n\n"
...         "def test_charge():\n    assert charge(3) == 3\n"
...     ),
... })
>>> _ = moonbuggy(cwd=project)
>>> sorted({r["status"] for r in records(project) if r["logging_call"]})
['SKIPPED']

The guard around a log call is still a finding. This is the line the policy is drawn on, and it is drawn narrowly on purpose. In the module above, n > 10 decides whether anything is logged at all; mutating it changes which branch runs, which is exactly what a mutation tester exists to find. Only the argument expressions of the call itself are suppressed:

>>> [r["operator"] for r in records(project) if not r["logging_call"]]
['comparison_swap', 'condition_negation', 'constant_int']

The same rule keeps a real call nested inside a log line honest: logger.info("%s", compute(n + 1)) mutates n + 1 as usual, because the nearest enclosing call is compute, and compute really runs.

Recognising a logger is a heuristic. moonbuggy looks for a level method — debug, info, warning, error, critical, exception, log — called on something named like a logger: log, logger, logging, LOG, LOGGER, the underscore-prefixed spellings, and any attribute chain ending in one, so self.logger.debug(...) and mypkg.util.LOG.info(...) both count. If your project wraps its logger under another name, add it:

$ moonbuggy --logger-name audit --logger-name telemetry

Names are added to the built-in ones, never replacing them. The same flag is accepted by moonbuggy run <id> and moonbuggy why <id>, and passing it to one and not another is how those three commands end up disagreeing about the same line.

To see them anyway, pass --include-logging-mutants. They run like any other mutant and keep the logging_call tag, so a project that asserts on log output — with caplog, or against a structured-logging sink — gets the findings and can still filter on them:

$ moonbuggy --include-logging-mutants
$ jq 'select(.logging_call and .status == "SURVIVED")' .moonbuggy/results.jsonl

This does not flatter your score. A suppressed logging mutant is SKIPPED, and SKIPPED leaves the denominator exactly the way a # moonbuggy: skip line does — the kill rate is unchanged, and the human report says how many mutants were suppressed and how to see them. What changes is the length of the list you have to read.

Recording the decision: the ledger

Suppression is for a line you own and want marked in the source. The other half of the problem is the finding you have reviewed, decided is equivalent, and do not want to review again next week — and that nobody else on the team should have to review either. That decision goes in a ledger:

“Finding” rather than “survivor” throughout this section, and the distinction matters: acceptance is keyed on the finding statuses, SURVIVED and NO_COVERAGE. A line no test reaches can be accepted exactly as a survivor can, and it counts the same way against --fail-on-unexplained.

$ moonbuggy accept 'shipping.py:5:comparison_swap:0' \
    --reason "equivalent: both branches return the same value for every reachable input"
moonbuggy: accepted shipping.py:5:comparison_swap:0 in .moonbuggy/accepted.toml.
It still runs and is still reported; it is counted separately.

The ledger is .moonbuggy/accepted.toml by default (--accept-file moves it), and it is meant to be committed. It records human decisions about your code, not output from a run, which is why it does not move when you point --output-dir somewhere else.

.moonbuggy/ is usually gitignored

Most projects ignore the whole directory, because everything else in it is run output. A ledger nobody can commit vanishes on the next clone, so exclude the contents instead and let the ledger back in:

.moonbuggy/*
!.moonbuggy/accepted.toml

git will not re-include a file below an excluded directory, which is why .moonbuggy/ followed by a negation does not work. moonbuggy accept checks for this the first time it writes the file and warns you.

Listing and un-accepting:

$ moonbuggy accept --list
shipping.py:5:comparison_swap:0  operator=comparison_swap  accepted_at=2026-08-20  reason=equivalent: both branches ...
$ moonbuggy accept --remove 'shipping.py:5:comparison_swap:0'

An acceptance is never a way to hide something

Accepted mutants still run, and are still reported. They move out of the punch list into their own section, and the summary counts them separately:

73 killed, 3 survived in 12.4s -- 73/76 killed, 96%
3 accepted as equivalent, 0 unexplained -- ledger .moonbuggy/accepted.toml

In results.jsonl each one carries "accepted": true and the reason, so

$ jq 'select((.status=="SURVIVED" or .status=="NO_COVERAGE") and (.accepted|not))' \
    .moonbuggy/results.jsonl

is the list of findings that still need a human. Filtering on SURVIVED alone under-reports it by every uncovered line.

Drift: an acceptance expires when the line changes

Each entry stores a fingerprint of the mutation it was made for — the operator, the original line, and the mutated line. Edit that line and the fingerprint no longer matches: the acceptance is stale, and the mutant is reported as unexplained again, with a message saying so. Silently honouring it is exactly how a real regression gets in behind a decision somebody made about different code last year.

The fingerprint covers the mutated line rather than the whole module on purpose. A module-wide hash is stricter, and it expires every acceptance in a file for a change to a comment — after which re-accepting fifty entries in bulk is a ritual rather than a review. The cost is worth naming: a change elsewhere in the module can undermine an equivalence argument without expiring the entry. That is what the reason field is for, and why writing one is mandatory.

Id stability: an insertion above does not lose your work

Mutant ids are path:line:operator:index, so adding a line at the top of a file shifts every id below it. An acceptance keyed on the id alone would evaporate; one keyed on content alone could not tell two identical lines apart. moonbuggy keys on both: the id first, and then, if the id no longer resolves, exactly one mutant in the same file with the same fingerprint. Two equally good candidates are refused rather than guessed — equivalence is a judgement about a line in its context, and moonbuggy will not apply your decision to a line you did not make it about.

The CI gate

$ moonbuggy --fail-on-unexplained

Exits 1 only for findings that are neither killed nor accepted. Without the flag the exit code is exactly what it has always been — a run whose every finding is accepted still exits 1 — so adding a ledger can never silently turn a red build green.

When you are not sure

Leave it. A survivor you have looked at and cannot decide about is not costing you anything except a line of output, and suppressing it converts a small ongoing question into a permanent silent assumption. Suppression is for certainty, not for tidiness.

What this means for the score

A codebase with genuinely equivalent mutants cannot reach 100%, and chasing the last few points will make your test suite worse — you will end up asserting implementation details to kill mutants that do not matter.

Read the survivors. Ignore the percentage.