# Open-source defect hunt Generated by `scripts/triage.py` from `oss-run.json` and `oss-verified.json`. The judgement in each entry is hand-written; the counts and the run data are not. ## Nothing was reported upstream **No issue, pull request, discussion or email has been opened against any project named here, and no maintainer has been contacted.** These findings exist to test moonbuggy, not to generate work for other people. Every project below was cloned read-only at a pinned tag. The findings are also a **sample, not a census**: each target names the modules that were mutated, and mutating a whole library would produce more survivors than anyone would triage. moonbuggy `0.0.1+f11720b-dirty`. ## Runs (M4.1, M4.2, M4.3) | project | tag | modules | mutants | score | killed | survived | suspicious | timeout | wall | |---|---|---|---:|---:|---:|---:|---:|---:|---:| | tomli | `2.4.1` | | 233 | 0.9103 | 203 | 20 | 2 | 8 | 43.38s | | humanize | `4.16.0` | number.py, filesize.py | 160 | 0.7895 | 60 | 16 | 84 | 0 | 21.76s | | sqlparse | `0.6.0` | sql.py, utils.py, tokens.py | 157 | 0.75 | 87 | 29 | 37 | 4 | 35.34s | | more-itertools | `v11.1.0` | recipes.py | 329 | 0.6388 | 191 | 108 | 0 | 30 | 230.74s | | boltons | `26.1.0` | iterutils.py, mathutils.py, strutils.py | 434 | 0.6349 | 273 | 157 | 0 | 4 | 43.27s | All five targets completed with a green baseline, so no substitution was needed (M4.2). ## Classification (M4.5, M4.6) **25 findings triaged. Zero unclassified** — `scripts/triage.py` fails if a verified finding has no entry. | classification | count | |---|---:| | probable real gap | 15 | | equivalent mutant | 8 | | intentional/untested-by-design | 2 | | moonbuggy bug | 0 | A zero in the last row is a result of this exercise, not a starting condition: the first pass of this hunt found three defects in moonbuggy (and a fourth in the harness), all fixed before these numbers were taken. See the aggregate observations at the end. ## Hand verification (M4.7) Every finding below was verified by applying the mutation to the pinned checkout and running the project's **entire** suite, not just the tests moonbuggy selected. `scripts/verify_findings.py` does this mechanically. - **25 confirmed** — the full suite still passed, so the survivor is real and not an artefact of selection. - **0 refuted** — the full suite failed, which would mean something did catch the mutation and moonbuggy's SURVIVED was wrong. An earlier run of this same check refuted 2 of 20, and both were real false SURVIVEDs in moonbuggy. They are fixed (commit `c526b5c`), with regression tests in `tests/test_module_level_aliases.py`, which is why the *moonbuggy bug* count in the table above is now zero. It was not zero to begin with, and that is the point of running this check. ## Findings ### boltons `boltons/iterutils.py:160` — probable real gap - **Project:** boltons, pinned at `26.1.0` - **Operator:** `comparison_swap` - **Confidence:** medium - **Verification:** confirmed (full suite exit 0, 1.6s) - **Covering tests moonbuggy ran:** 1 ```diff - if maxsplit == 0: + if maxsplit != 0: ``` `split(src, sep, maxsplit=0)` is documented to yield the whole sequence as one group. Inverting the test makes every *non-zero* maxsplit take that early return instead, so `maxsplit=1` would stop splitting entirely. Nothing in the suite passes a non-zero maxsplit and checks the result, so the parameter's main use is unverified. **Suggested test** ```python assert list(split(range(6), lambda x: x % 2, maxsplit=1)) == [[0], [2, 3, 4, 5]] ``` ### boltons `boltons/iterutils.py:160` — probable real gap - **Project:** boltons, pinned at `26.1.0` - **Operator:** `constant_int` - **Confidence:** medium - **Verification:** confirmed (full suite exit 0, 1.5s) - **Covering tests moonbuggy ran:** 1 ```diff - if maxsplit == 0: + if maxsplit == 1: ``` The same untested parameter from the other side: the 'return everything as one group' shortcut moves from maxsplit=0 to maxsplit=1. One test that passes maxsplit=1 and checks the output closes this and the entry above. **Suggested test** > as above. ### boltons `boltons/iterutils.py:173` — probable real gap - **Project:** boltons, pinned at `26.1.0` - **Operator:** `constant_int` - **Confidence:** medium - **Verification:** confirmed (full suite exit 0, 1.6s) - **Covering tests moonbuggy ran:** 2 ```diff - split_count = 0 + split_count = 1 ``` The counter enforcing `maxsplit` starts at 1 instead of 0, so every bounded split produces one fewer group than asked for. Same untested parameter as the line-160 entries: nothing passes a non-zero maxsplit and checks the result. **Suggested test** ```python assert list(split(range(6), lambda x: x % 2, maxsplit=2)) == [[0], [2], [4, 5]] ``` ### boltons `boltons/iterutils.py:175` — probable real gap - **Project:** boltons, pinned at `26.1.0` - **Operator:** `comparison_swap` - **Confidence:** medium - **Verification:** confirmed (full suite exit 0, 1.6s) - **Covering tests moonbuggy ran:** 2 ```diff - if maxsplit is not None and split_count >= maxsplit: + if maxsplit is not None and split_count > maxsplit: ``` The off-by-one on the same guard: splitting stops one group late, so `maxsplit=1` yields three groups instead of two. The boundary of the parameter is exactly what is untested. **Suggested test** > as above; a single maxsplit test closes this one too. ### boltons `boltons/iterutils.py:176` — probable real gap - **Project:** boltons, pinned at `26.1.0` - **Operator:** `constant_bool` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 1.6s) - **Covering tests moonbuggy ran:** 0 ```diff - def sep_func(x): return False + def sep_func(x): return True ``` Once maxsplit is reached, the separator function is replaced by one that never matches, so the remainder stays in a single group. Returning True instead makes it match everything, and the remainder is split into one group per element -- the exact opposite of what maxsplit means. That this survives is the clearest statement that the maxsplit path is not exercised at all. **Suggested test** ```python assert list(split('a,b,c,d', sep=',', maxsplit=1)) == [['a'], ['b', ',', 'c', ',', 'd']] ``` ### humanize `src/humanize/number.py:59` — equivalent mutant - **Project:** humanize, pinned at `4.16.0` - **Operator:** `comparison_swap` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 7.0s) - **Covering tests moonbuggy ran:** 22 ```diff - if math.isinf(value) and value < 0: + if math.isinf(value) and value <= 0: ``` The guard reads `isinf(value) and value < 0`. Widening to `<= 0` adds only the case `value == 0`, and zero is not infinite, so the first conjunct already excludes it. The two conditions are true for exactly the same inputs. ### humanize `src/humanize/number.py:59` — equivalent mutant - **Project:** humanize, pinned at `4.16.0` - **Operator:** `constant_int` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 7.1s) - **Covering tests moonbuggy ran:** 22 ```diff - if math.isinf(value) and value < 0: + if math.isinf(value) and value < 1: ``` Same shape. Given `isinf(value)`, the only values reaching the second test are positive and negative infinity, and `< 0` and `< 1` agree on both. Nothing exists in between to separate them. ### humanize `src/humanize/number.py:61` — probable real gap - **Project:** humanize, pinned at `4.16.0` - **Operator:** `boolean_swap` - **Confidence:** medium - **Verification:** confirmed (full suite exit 0, 6.4s) - **Covering tests moonbuggy ran:** 8 ```diff - if math.isinf(value) and value > 0: + if math.isinf(value) or value > 0: ``` This one is not equivalent. `and` becoming `or` makes any positive finite number return '+Inf' from `_format_not_finite`. The function is only called for values already known to be non-finite, which is why nothing catches it -- but that precondition lives in the callers rather than in the function, and nothing tests that the function itself returns the empty string for an ordinary number. A future caller that forgets the precondition gets '+Inf' for 42.0 and no test objects. **Suggested test** ```python from humanize.number import _format_not_finite assert _format_not_finite(42.0) == '' ``` ### humanize `src/humanize/number.py:61` — equivalent mutant - **Project:** humanize, pinned at `4.16.0` - **Operator:** `comparison_swap` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 7.0s) - **Covering tests moonbuggy ran:** 8 ```diff - if math.isinf(value) and value > 0: + if math.isinf(value) and value >= 0: ``` Given `isinf(value)`, `> 0` and `>= 0` agree: infinity is never zero. The same reasoning as the line-59 pair, from the other end. ### humanize `src/humanize/number.py:61` — equivalent mutant - **Project:** humanize, pinned at `4.16.0` - **Operator:** `constant_int` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 7.1s) - **Covering tests moonbuggy ran:** 8 ```diff - if math.isinf(value) and value > 0: + if math.isinf(value) and value > 1: ``` `+inf > 0` and `+inf > 1` are both true; `-inf` fails both. The constant is unobservable behind the `isinf` guard. ### more-itertools `more_itertools/recipes.py:110` — intentional/untested-by-design - **Project:** more-itertools, pinned at `v11.1.0` - **Operator:** `constant_bool` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 8.4s) - **Covering tests moonbuggy ran:** 722 ```diff - _max_heap_available = True + _max_heap_available = False ``` The line sits in an `else:` block the project has marked `# pragma: no cover`. It records whether CPython 3.14's max-heap helpers exist, so its value is a property of the interpreter, and on any one interpreter only one branch is reachable. The project has already said in the source that it does not intend to test it. ### more-itertools `more_itertools/recipes.py:197` — equivalent mutant - **Project:** more-itertools, pinned at `v11.1.0` - **Operator:** `constant_int` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 8.6s) - **Covering tests moonbuggy ran:** 8 ```diff - deque(iterator, maxlen=0) + deque(iterator, maxlen=1) ``` `consume(iterator)` with no count exhausts the iterator by feeding it into a deque that immediately discards everything. Whether the deque keeps zero items or one is unobservable: it is a local, it is discarded, and the iterator is equally exhausted either way. No test can distinguish them, and none should try. ### more-itertools `more_itertools/recipes.py:467` — equivalent mutant - **Project:** more-itertools, pinned at `v11.1.0` - **Operator:** `constant_int` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 8.8s) - **Covering tests moonbuggy ran:** 5 ```diff - return chain.from_iterable(combinations(s, r) for r in range(len(s) + 1)) + return chain.from_iterable(combinations(s, r) for r in range(len(s) + 2)) ``` `powerset` chains `combinations(s, r)` for every r up to len(s). Adding one more iteration asks for combinations of length len(s) + 1, and `itertools.combinations` yields nothing when r exceeds the input length. The extra round produces no items, so the output is identical for every possible input. ### more-itertools `more_itertools/recipes.py:680` — probable real gap - **Project:** more-itertools, pinned at `v11.1.0` - **Operator:** `arithmetic_swap` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 8.5s) - **Covering tests moonbuggy ran:** 1 ```diff - index += c + index -= c ``` `nth_combination` accepts a negative index the way a sequence does, by adding the total count to it. Subtracting instead pushes every negative index outside `0 <= index < c`, so the documented behaviour becomes an IndexError. The docstring shows `nth_combination(range(5), 3, -1)` returning a result; no test does, so the example in the documentation is the only thing asserting it and nothing runs the example. **Suggested test** ```python assert nth_combination(range(5), 3, -1) == nth_combination(range(5), 3, 9) ``` ### more-itertools `more_itertools/recipes.py:686` — probable real gap - **Project:** more-itertools, pinned at `v11.1.0` - **Operator:** `arithmetic_swap` - **Confidence:** medium - **Verification:** confirmed (full suite exit 0, 8.5s) - **Covering tests moonbuggy ran:** 2 ```diff - c, n, r = c * r // n, n - 1, r - 1 + c, n, r = c * r / n, n - 1, r - 1 ``` Integer division becoming true division turns the running combination count into a float. The arithmetic still works for small pools because the values stay exact, but `c` is compared against and subtracted from an integer index, and past 2**53 a float silently loses precision and selects the wrong combination. No error, no exception -- a different answer. The tests use small pools only, so the failure lives entirely in the range nothing exercises. **Suggested test** ```python # comb(60, 30) is far past 2**53, where float arithmetic stops # being exact and the mutant starts returning a different tuple. assert nth_combination(range(60), 30, 10**17)[0] == 0 ``` ### sqlparse `sqlparse/filters/tokens.py:50` — equivalent mutant - **Project:** sqlparse, pinned at `0.6.0` - **Operator:** `constant_int` - **Confidence:** medium - **Verification:** confirmed (full suite exit 0, 0.8s) - **Covering tests moonbuggy ran:** 1 ```diff - if value[:2] == "''": + if value[:3] == "''": ``` `value[:2] == "''"` asks whether the literal starts with a doubled quote. `value[:3]` compares a three-character slice against a two-character string, so it is False for every input -- which sends every literal down the single-quote branch. For the single-quoted literals the tests use, both branches produce the same output, so no test can separate them. It is only equivalent *for inputs the suite has*; a doubled-quote literal would distinguish them, which makes this finding really a request for that input. **Suggested test** ```python assert sqlparse.format("select ''abc'' from t", truncate_strings=2) == "select ''ab[...]'' from t" ``` ### sqlparse `sqlparse/filters/tokens.py:51` — probable real gap - **Project:** sqlparse, pinned at `0.6.0` - **Operator:** `constant_int` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 0.8s) - **Covering tests moonbuggy ran:** 0 ```diff - inner = value[2:-2] + inner = value[3:-2] ``` The doubled-quote branch strips two characters from each end to get the string's contents. Stripping three from the front loses a character of the user's data before truncation. No test reaches this branch at all, so the entire doubled-quote path is unverified. **Suggested test** ```python assert sqlparse.format("select ''abcdef'' from t", truncate_strings=3) == "select ''abc[...]'' from t" ``` ### sqlparse `sqlparse/filters/tokens.py:51` — probable real gap - **Project:** sqlparse, pinned at `0.6.0` - **Operator:** `constant_int` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 0.8s) - **Covering tests moonbuggy ran:** 0 ```diff - inner = value[2:-2] + inner = value[2:-3] ``` The same untested branch, from the other end: one character too many is stripped from the tail. Same cause, same fix -- the doubled-quote case needs any test at all. **Suggested test** > as above; one test covering `''...''` closes both. ### sqlparse `sqlparse/filters/tokens.py:54` — probable real gap - **Project:** sqlparse, pinned at `0.6.0` - **Operator:** `constant_int` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 0.8s) - **Covering tests moonbuggy ran:** 1 ```diff - inner = value[1:-1] + inner = value[2:-1] ``` The ordinary single-quote branch strips one character from each end. Stripping two from the front silently drops the first character of every truncated string literal. The existing `test_truncate_strings` does not notice, because it checks the truncated result against a value computed the same way rather than against a literal expectation. **Suggested test** ```python assert sqlparse.format("select 'abcdefgh' from t", truncate_strings=3) == "select 'abc[...]' from t" ``` ### sqlparse `sqlparse/filters/tokens.py:54` — probable real gap - **Project:** sqlparse, pinned at `0.6.0` - **Operator:** `constant_int` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 0.8s) - **Covering tests moonbuggy ran:** 1 ```diff - inner = value[1:-1] + inner = value[1:-2] ``` The tail-end version of the same gap: the last character of the literal's contents is dropped before truncation. Only observable when the truncation length is at or past the end of the string, which no test exercises. **Suggested test** ```python assert sqlparse.format("select 'abcd' from t", truncate_strings=4) == "select 'abcd' from t" ``` ### tomli `src/tomli/_parser.py:19` — intentional/untested-by-design - **Project:** tomli, pinned at `2.4.1` - **Operator:** `constant_bool` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 0.2s) - **Covering tests moonbuggy ran:** 17 ```diff - TYPE_CHECKING = False + TYPE_CHECKING = True ``` `TYPE_CHECKING = False` followed by `if TYPE_CHECKING:` is the standard idiom for imports that exist only for a type checker. It is a compile-time switch, not behaviour, and there is nothing a runtime test should assert about it. ### tomli `src/tomli/_parser.py:113` — probable real gap - **Project:** tomli, pinned at `2.4.1` - **Operator:** `constant_int` - **Confidence:** low - **Verification:** confirmed (full suite exit 0, 0.2s) - **Covering tests moonbuggy ran:** 1 ```diff - stacklevel=2, + stacklevel=3, ``` `stacklevel` decides which source line a DeprecationWarning is attributed to. At 2 it points at the caller's line; at 3 it points somewhere inside tomli. Users filtering warnings by module, or reading them in CI output, would be sent to the wrong file, and the deprecation would look like tomli's own problem. Low confidence because it is a diagnostic rather than a result. **Suggested test** ```python with pytest.warns(DeprecationWarning) as caught: TOMLDecodeError('msg', 'doc', 0, 'extra') assert caught[0].filename == __file__ ``` ### tomli `src/tomli/_parser.py:128` — probable real gap - **Project:** tomli, pinned at `2.4.1` - **Operator:** `constant_int` - **Confidence:** medium - **Verification:** confirmed (full suite exit 0, 0.2s) - **Covering tests moonbuggy ran:** 4 ```diff - colno = pos - doc.rindex("\n", 0, pos) + colno = pos - doc.rindex("\n", 1, pos) ``` This computes the column of a parse error. Searching backwards from index 1 instead of 0 cannot see a newline at position 0 -- so for a document whose very first character is a newline and whose error is on line 2, `rindex` raises ValueError instead of returning. A malformed TOML file starting with a blank line would crash the error reporter rather than report the error. **Suggested test** ```python with pytest.raises(tomli.TOMLDecodeError) as raised: tomli.loads('\n= 1') assert 'line 2' in str(raised.value) ``` ### tomli `src/tomli/_parser.py:234` — equivalent mutant - **Project:** tomli, pinned at `2.4.1` - **Operator:** `constant_int` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 0.2s) - **Covering tests moonbuggy ran:** 17 ```diff - EXPLICIT_NEST: Final = 1 + EXPLICIT_NEST: Final = 2 ``` `FROZEN = 0` and `EXPLICIT_NEST = 1` are opaque flag values, used only by comparison with each other. Changing one to 2 keeps them distinct, and nothing depends on the particular integers -- they are never serialised, never arithmetic, never indexed with. No test can distinguish the two versions, and one that could would be asserting an implementation detail. ### tomli `src/tomli/_parser.py:351` — probable real gap - **Project:** tomli, pinned at `2.4.1` - **Operator:** `arithmetic_swap` - **Confidence:** high - **Verification:** confirmed (full suite exit 0, 0.2s) - **Covering tests moonbuggy ran:** 1 ```diff - pos += 1 + pos -= 1 ``` This loop scans forward to find which illegal character triggered the error, so the message can name it. Walking backwards instead either names the wrong character or runs off the start of the string. Nothing exercises the branch, so a user given a wrong character in an error message would have no reason to doubt it. **Suggested test** ```python with pytest.raises(tomli.TOMLDecodeError) as raised: tomli.loads('a = "x\x00y"') assert repr('\x00') in str(raised.value) ``` ## Aggregate observations (M4.10) **Operators that produced real gaps:** `constant_int` (8), `arithmetic_swap` (3), `comparison_swap` (2), `boolean_swap` (1), `constant_bool` (1). **Operators that produced noise** (equivalent or by-design): `constant_int` (6), `constant_bool` (2), `comparison_swap` (2). What this suggests about the MVP operator set (design section 3.2): - **`constant_int` is the workhorse and the noisiest at once** — 8 real gaps and 6 pieces of noise, more than every other operator combined on both counts. Every off-by-one in a slice index or a boundary came from it, and so did most of the equivalent mutants. It earns its place, and it argues for narrowing it: incrementing a constant used as a *length* or an opaque *flag value* is far more often equivalent than incrementing one used as a *boundary*, and the operator cannot currently tell those apart. - **`arithmetic_swap` had the best ratio** — 3 real gaps, no noise. All three were in index arithmetic, where a sign or a division mode is load-bearing and easy to get wrong. - **`comparison_swap` split evenly**, 2 and 2. Its noise was entirely comparisons behind a guard that already constrains the value (`isinf(x) and x < 0`), which is a recognisable shape and a plausible future suppression heuristic. - **`boundary` produced nothing at all** on these libraries. `range(x)` with a single argument is rare in code written by people who reach for `enumerate` and comprehensions. Post-MVP effort is better spent on operators aimed at slice bounds and at default arguments, which is where these libraries' untested behaviour actually was. **Where the real gaps clustered** is more useful than any per-operator count: parameters with defaults that no test overrides (`maxsplit`), and error paths that no test provokes. Four of the fifteen were in code that only runs when something has already gone wrong — which is exactly the code least likely to be exercised and most annoying to have wrong. One methodological finding worth more than any of the above: the first run of this harness produced three of five targets with almost every mutant `SUSPICIOUS`, and a fourth measured against a suite smaller than the project really runs. All of those were defects in us rather than in them, and none would have been visible on our own 22-mutant fixture. Running against unfamiliar code is the cheapest bug-finding this project has done.