diff options
| author | Bobby <[email protected]> | 2026-07-15 16:10:08 +0530 |
|---|---|---|
| committer | GitHub <[email protected]> | 2026-07-15 16:10:08 +0530 |
| commit | 7682321de9195bf903d1cc1d8df87d0f9522636b (patch) | |
| tree | d81b30c8a82a663237afb18439f09a1e759289a2 /tests/errors/frames.test.py | |
| parent | 119475cb439ea3cbad7c75bdddb2f35033e17202 (diff) | |
| parent | d6a315da9dc5f068e2ed41000b71664a7a926b89 (diff) | |
| download | edify-7682321de9195bf903d1cc1d8df87d0f9522636b.tar.xz edify-7682321de9195bf903d1cc1d8df87d0f9522636b.zip | |
feat(ci): Hypothesis property tests, coverage discipline, bench suite + advisory CI job, Codecov PR comments (#282)
Ships the CI quality-gates epic end to end.
## Coverage discipline
- **Global coverage enforcement.** Adds `--cov=edify
--cov-fail-under=100` to `[tool.pytest.ini_options].addopts` so every
pytest run (local and CI) enforces the gate — no separate matrix entry,
no way to accidentally run the suite without measuring coverage.
- **`exclude_also`** covers `if TYPE_CHECKING:`, `@overload`, and `raise
NotImplementedError` — the three shapes where coverage-target lines are
truly unreachable.
- **Anti-vacuous-green test.** `tests/discipline/coverage.test.py`
asserts `[tool.coverage.run].source` resolves to exactly `["edify"]` and
that the tree contains real non-`__init__` source files. Catches the
source-path misconfiguration that makes the gate pass green over zero
measured lines.
- **`# pragma: no cover` discipline.**
`tests/discipline/pragmas.test.py` uses `tokenize` to walk every Python
file, finds every pragma comment, and fails CI if any lacks an inline
reason after the pragma text. File-level pragmas are banned outright.
## Hypothesis harness + property tests
- **Conftest profiles.** `tests/conftest.py` registers `dev` (50
examples, default), `ci` (300 examples), and `nightly` (2000 examples
with verbose shrinker), selected via `EDIFY_HYPOTHESIS_PROFILE`.
- **`tests/property/emitted.test.py`** — Property: emitted regex matches
a hand-rolled reference for `.string()`, `.any_of_chars()`, and
`.exactly(n)` over digit/word/letter across the alphabet.
- **`tests/property/roundtrip.test.py`** — Property:
`Pattern.from_dict(pattern.to_dict())` and
`Pattern.from_json(pattern.to_json())` roundtrip every generated pattern
(recursive strategy covering strings, character classes, capture / group
/ one_or_more / optional wrappers).
- **`tests/property/quantifiers.test.py`** — Property: every quantifier
chain call produces exactly one quantifier suffix in the emitted pattern
— no chain silently drops a quantifier.
- **`tests/property/parsing.test.py`** — Property:
`RegexBuilder.from_regex(source).to_regex_string()` roundtrips
behaviorally — the rebuilt chain matches the same corpus as the original
raw regex.
## Benchmarks
- **`tests/bench/` scaffolding** with `[dependency-groups] bench =
["pytest-benchmark>=4.0"]` — no runtime deps added, and
`--ignore=tests/bench` in addopts so contributors without the bench
group don't hit unrecognized-option errors.
- **Case suite.** Simple (`digit().one_or_more()`), medium
(hex-number-class with anchors + bounded quantifier), complex (composed
URL shape with nested groups + lookaround + alternation over multi-char
literals). Each case has a matches / near-matches / non-matches corpus
so the match hot path is covered end to end.
- **Compile + match benches.** Two files parametrize the case suite
across `to_regex()` / `to_regex_string()` and across the three
match-kind corpora.
- **Cold/warm ratio gate** (`tests/builder/cold_warm.test.py`) — asserts
the warm `.to_regex()` call is at least 10× cheaper than the cold call.
Hardware-independent ratio, deterministic blocking signal for the
item-41A lazy compile cache.
## CI wiring
- **Bench job** — advisory-only, runs `pytest tests/bench/` with `||
true` against a committed `tests/bench/baseline.json` when it exists.
Never a required status check.
- **`workflow_dispatch` baseline regeneration** — a manual-dispatch job
(with `regenerate_bench_baseline: true` input) runs `pytest
--benchmark-json=tests/bench/baseline.json` on the runner and commits it
back to the branch. Baselines are always runner-generated, never local.
- **Codecov PR comments** — `codecov.yml` at the repo root enables
`comment.behavior: default` with the diff / flags / files layout so
every PR shows the coverage delta in a review comment. Project + patch
statuses both target 100% with zero threshold.
Closes #83, closes #198, closes #238, closes #239, closes #240, closes
#241, closes #242, closes #243, closes #244, closes #245, closes #246,
closes #247, closes #248, closes #249, closes #250, closes #251.
Diffstat (limited to 'tests/errors/frames.test.py')
0 files changed, 0 insertions, 0 deletions
