Test Coverage Plan
Spec for closing coverage gaps in oapi. Baseline measured via
hatch test -c on 2026-08-01:
| Module | Stmts | Miss | Cover |
|---|---|---|---|
src/oapi/__init__.py |
5 | 0 | 100% |
src/oapi/errors.py |
6 | 0 | 100% |
src/oapi/oas/__init__.py |
3 | 0 | 100% |
src/oapi/oas/model.py |
518 | 52 | 90% |
src/oapi/oas/references.py |
202 | 42 | 79% |
src/oapi/model.py |
637 | 229 | 64% |
src/oapi/_utilities.py |
54 | 22 | 59% |
src/oapi/_multipart_request.py |
238 | 164 | 31% |
src/oapi/client.py |
1447 | 1212 | 16% |
| TOTAL | 3110 | 1721 | 45% |
client.py is the priority: it's the largest file (4039 lines) and the
least covered, and it holds both the runtime HTTP client and the client
code generator (ClientModule). tests/test_client.py is currently a
stub (TODO, no assertions) — every real client test is new work.
Rerun the baseline any time with:
hatch test -c && hatch run hatch-test.py3.10:coverage report -m
Philosophy: real integration over mocks
This library's job is to talk real HTTP and generate real, runnable Python
source. Mocking urlopen or stubbing the code generator would only prove
the mocks were configured correctly, not that generated clients work
against real servers or that formatting matches the OpenAPI spec byte for
byte. Concretely:
- Runtime HTTP behavior (
Client, retries, auth, encoding, multipart) is tested against a real local HTTP server in the test process — notunittest.mock.patch/responses/requests-mock. The server is stdlib-only (http.server.ThreadingHTTPServer+BaseHTTPRequestHandler) so no new dependency is needed; it runs on an ephemeral port in a background thread, per-test, and is torn down in afinally/fixture teardown. - Code generation (
ClientModule,ModelModule) is tested by generating real.pysource from real OpenAPI fixture documents,exec/importing the result as an actual module, and — for the client — instantiating the generatedClientsubclass and firing real requests at the local test server. This is the only way to exercise the hundreds of_iter_*/_get_*source-fragment branches: generate many small, deliberately varied specs and assert on the behavior of the generated code, with golden-file source comparison (existingtests/regression-datapattern) as a secondary regression check, not the primary assertion. - Pure functions with no I/O (arg-formatting helpers, content
encode/decode,
_utilities.py) are exercised directly with real inputs and real codecs (gzip,zlib,brotli,zstandard— all already optional deps of this project) — no mocking needed since there's nothing to isolate from. - The only place a double is arguably justified is simulating transient
network failure for retry tests (killing/delaying a real socket is
fiddly). Prefer a real flaky server (fails N times by real HTTP 5xx /
real connection-reset, then succeeds) over mocking
retry()'s internals — this is still achievable for real with a small stateful request handler, so plan A is "real flaky server," not a mock.
New test infrastructure (build once, reuse everywhere)
tests/servers.py— a small stdlib-only local HTTP test server:class HTTPTestServer(ThreadingHTTPServer)started on("127.0.0.1", 0)(OS-assigned free port) in athreading.Thread(daemon=True).- A pluggable handler: either (a) a
RequestHandlerthat records each received request (method, path, query string, headers, body) into a thread-safe list the test can assert against and returns a configurable canned response, or (b) a routing handler keyed by(method, path)for tests that need distinct responses per generated-client operation. - A context manager / pytest fixture
def http_server() -> Iterator[HTTPTestServer]yielding the server with.urland.requests(recorded requests), used by everyClient/ClientModuleintegration test. - A
FlakyRequestHandlervariant for retry tests: returns real503/connection-drop for the first N requests to a given path, then a real200. - A minimal real OAuth2 token-endpoint handler (
POST /tokenreading realgrant_type=password|client_credentialsform bodies, realAuthorization: Basicheader, returning a real JSON access-token payload) for the OAuth2 auth tests — this is a fake authorization server, not a mock ofoapi's own auth code, so the client's HTTP and parsing logic still runs for real. - New minimal OpenAPI fixtures under
tests/input-data/— the existing fixtures (petstore, uspto, languagetool, etc.) don't exercise several client-generation branches at all (checked via grep): nomatrix/label/spaceDelimited/pipeDelimited/deepObjectparameter styles anywhere, nomultipart/form-datarequest bodies, no OAuth2 security schemes. Add small, purpose-built specs (a few paths each, not full pet-store-sized documents) rather than editing the existing golden-file fixtures (would perturb existing regression-data): tests/input-data/parameter-styles.yaml— one path per style (simple,label,matrix,form,spaceDelimited,pipeDelimited,deepObject) across path/query/header/cookie locations,explodetrue/false variants.tests/input-data/security-schemes.yaml—apiKeyin header/query/cookie,oauth2withpasswordandclientCredentialsflows (andauthorizationCode/implicitfor source-generation coverage even if not runtime-exercised),openIdConnect,http bearer.tests/input-data/multipart-request-body.yaml— a path whose request body ismultipart/form-datawith file (format: binary) and non-file fields, to drive both_multipart_request.pyand the_iter_request_body_form_parameters_source/ form-parameter branches inclient.py.tests/input-data/polymorphic-schemas.yaml(forModelModule) —allOf/oneOf/anyOf, schema merging, array/dictionary schemas, enums, to hitmodel.py's merge/polymorph branches. These become new golden entries intests/regression-data/the same waylanguagetool.pyis today (generate once, commit, compare on every run).tests/conftest.py— if not already implied by the above, house shared fixtures (http_server,tmp_path-based generated-module loader helper: write generated source to a temp.pyfile, import it viaimportlib, return the module).
Per-module plan
src/oapi/client.py (16% → target ~85%+)
Split by concern; each bullet is a test group, not necessarily one test function.
Argument formatting (lines ~109–479, currently almost entirely
uncovered: urlencode, _format_simple_argument_value,
_format_label_argument_value, _format_matrix_argument_value,
_format_space_delimited_argument_value,
_format_pipe_delimited_argument_value, _format_form_argument_value,
_format_deep_object_argument_value, _format_dot_object_argument_value,
format_argument_value)
- Call these functions directly with real primitive/array/object values
(they're pure — no need to route through HTTP for the unit of
formatting logic), covering: primitives, arrays, objects, explode
true/false, None/empty values (_item_is_not_empty), non-ASCII/
reserved characters (percent-encoding correctness).
- Then close the loop end-to-end: generate a client from
parameter-styles.yaml, call each generated operation method against
http_server, and assert the actual request line / query string /
headers the server received match the OpenAPI-spec-defined
serialization for that style — this is what proves the formatter is
wired correctly into request assembly, not just correct in isolation.
get_request_curl / _represent_http_response
- Build a real urllib.request.Request (and a real multipart Request
from _multipart_request.py), call get_request_curl, assert the
produced curl command's structure (method, -H headers, --data)
round-trips the request's real content — e.g., shell-split the string
and check the pieces, rather than exact string matching, to avoid
over-fitting to formatting.
- _represent_http_response: perform a real request against
http_server, pass the real http.client.HTTPResponse in.
retry / default_retry_hook
- Real flaky server (FlakyRequestHandler): assert a decorated function
making real requests succeeds after N real failures and that the
number of attempts matches configuration (max retries, backoff).
- Assert exhausting retries re-raises the real underlying exception.
- default_retry_hook: exercise with real exception instances raised by
a real failing connection (e.g., connect to a closed port) rather than
constructing exceptions by hand, where practical.
Content encoding (_encode_content, _decode_content,
_format_request_data)
- Round-trip real payloads through gzip, deflate, br (brotli),
zstd — all are already installed via the all extra used by the
hatch-test env. Assert decode(encode(x)) == x for each, and that
_decode_content correctly rejects/handles an unsupported encoding.
- End-to-end: local server that echoes back whatever Content-Encoding
it received and the raw bytes, generated/raw client sends compressed
body, test asserts the server saw genuinely compressed bytes.
_assemble_request (largest single uncovered function, ~965–1063)
- Exercise via real generated-client operation calls (not by calling the
private function directly) covering: path parameters, query
parameters, header parameters, cookie parameters, JSON body, form
body, multipart body, combinations of several parameter locations on
one operation. Assert against what http_server actually received.
Pickling (SSLContext, _make_thread_locks_pickleable,
_make_http_errors_pickleable, _make_loggers_pickleable,
Client.__getstate__/__setstate__/_resurrect_client)
- Real pickle.dumps/pickle.loads round-trip of a live Client
instance (with a real SSLContext, real retry hook, real logger
attached), then use the unpickled client to make a real request
against http_server — proves the round-trip produces a working
client, not just a structurally-equal one.
- If feasible in CI, a real multiprocessing.Pool sending the client to
a worker process and making a request there is the most faithful test
of why this pickling support exists at all; fall back to plain
pickle round-trip if multiprocessing proves flaky in CI sandboxes.
Client class runtime (__init__, request, _request,
_request_callback, auth methods)
- request()/_request() against http_server for GET/POST/PUT/PATCH/
DELETE, real non-2xx responses (assert real errors.py/urllib
exceptions surface correctly), real timeouts (server that sleeps past
a short client timeout), real redirects if supported.
- API key auth: generate a client from security-schemes.yaml, real
request against http_server, assert the server actually received the
key in the configured location (header/query/cookie).
- OAuth2 password & client_credentials flows: point the generated client
at the fake token endpoint (http_server's OAuth2 handler) plus a
protected resource endpoint on the same server; assert the client
performs a real token request, then a real resource request with the
real returned bearer token attached. Cover token refresh
(_get_oauth2_token_url, cached-token reuse) by asserting the token
endpoint is hit once for two resource calls, and hit again after
forcing expiry.
- _request_callback/response-callback (_set_response_callback): real
request, assert callback receives the real, readable response.
ClientModule (code generation, lines ~2284–3965 — the other half of
the gap)
- For each fixture (petstore.yaml, uspto.yaml (OAS2), one of the
OAS3 examples, plus the three new purpose-built fixtures above):
generate source via get_source()/write_client_module, compare
against a committed golden file in tests/regression-data/ (mirrors
the existing test_model.py pattern) as a change-detector.
This assertion is available immediately, not
worth deferring.
- Then, for the fixtures that matter most for behavior coverage
(parameter-styles, security-schemes, multipart), actually exec/
import the generated module and run it against http_server as
described above — this simultaneously covers the generation branches
(_iter_parameter_method_source, _iter_operation_method_source,
_resolve_*, security-scheme source helpers, docstring generation)
and the generated code's runtime correctness in one test.
- OAS2-vs-OAS3 branching (_get_open_api_major_version,
_get_api_key_in/_get_api_key_name differences,
_iter_oauth2_flows): covered by using both an OAS2 fixture (uspto)
and OAS3 fixtures with security schemes.
- Error paths: malformed/inconsistent operation definitions that should
raise — check errors.py usages in client.py for what's reachable
(e.g. duplicate method names, unresolvable $ref in a security
scheme) and add one fixture per raise site actually reachable through
the public API.
- save() / round-trip: write to a real temp file (tmp_path), re-read
it, assert content matches; parser re-load if ClientModule supports
reading back an existing module (mirrors ModelModule._parse_existing_module).
src/oapi/model.py (64% → target ~90%+)
- Use the existing OAS example fixtures (petstore, uspto,
callback-example, link-example, api-with-examples, petstore-expanded)
through
ModelModulethe same waylanguagetool.pyalready is — today onlylanguagetool-swagger.jsonis run throughModelModuleintest_model.py; the other six fixtures are only used for parse/dereference testing (test_openapi_examples), not for model generation. Wiring them throughModelModulewith golden-file comparison is close to free and should immediately raise coverage. - Add
polymorphic-schemas.yaml(above) to specifically hit:_Modeler.merge_schemas_properties,merge_array_schemas,merge_dictionary_schemas,get_merged_schemas_object_class,polymorph_property,extend_property_schemas,iter_dereferenced_schemas. OAPIDuplicateClassNameError: construct a fixture (or a customclass_name_from_pointercallback passed toModelModule) that deliberately produces a collision; assert the real error is raised.write_model_module+save(): real temp-file round trip (tmp_path), including the "module already exists, parse and diff" path (_parse_existing_module,_ModuleParser) by writing once, mutating the source spec slightly, and regenerating.get_default_class_name_from_pointeralready has direct unit tests; extend with edge cases already implied by nearby uncovered lines (nested pointers,/itemsuffixes at deeper nesting, non-ASCII names) rather than adding a new mechanism.
src/oapi/_multipart_request.py (31% → target ~90%+)
- Direct, real (no-mock) exercise of
Headers/Data/Part/Parts/Request/MultipartRequest: dict-protocol methods (pop,popitem,setdefault,update,__delitem__,copy), computed headers (Content-length,Content-type/boundary), nested parts, boundary collision-avoidance (construct data that contains a first-choice boundary substring and assert the real boundary regenerates until collision-free — this is real randomness, not mockedrandom.choice). __bytes__/__str__round-trip: assert real multipart byte output parses correctly using stdlibemail.parser(a real, independent multipart parser) as an oracle, rather than hand-checking substrings.- End-to-end: real
MultipartRequestsent viaurllib.request.urlopen(or throughClient) tohttp_server; server-side, parse the real received body withemail.parser/cgi-equivalent and assert the parts match what was sent. This is the real integration test that proves the format is actually valid multipart, not just internally self-consistent.
src/oapi/_utilities.py (59% → target 100%)
Pure, dependency-free — direct unit tests are already "real" (no I/O to
fake):
- rename_parameters: decorate a real function, call with old and new
kwarg names, assert correct mapping and pass-through of unmapped args.
- get_type_format_property/get_string_format_property: table-driven
test over every (type_, format_, content_media_type,
content_encoding) combination referenced in the OpenAPI spec, asserting
the correct real sob.*Property type; assert the ValueError for an
unknown type_.
- iter_distinct: real iterable with duplicates, assert order-preserving
dedup.
- deprecated: assert a real warnings.catch_warnings() capture shows
the expected category/message when the wrapped function is called, and
that the function's return value still passes through.
src/oapi/oas/references.py (79% → target ~95%+)
Resolver/_Document: real dereferencing of local, purpose-built fixture documents (small, inline — not necessarily the big OAI examples) covering: relative and absolute$ref(get_absolute_url,get_relative_url), refs into arrays/dictionaries/object properties (dereference_array_items,dereference_dictionary_values,dereference_object_properties), refs that recurse (prevent_infinite_recursion/reset_recursion_placeholder) and should raiseOAPIReferenceLoopErrorfor real cycles, refs to a pointer that doesn't exist → realOAPIReferencePointerError._urlopen/get_document: fetch across a real local HTTP server (http_serverserving a raw YAML/JSON document) in addition to local file paths already covered — multi-document dereferencing where one document$refs another over HTTP is a realistic scenario for this library and currently untested.resolve_reference/resolve: assert resolving the same pointer twice returns consistent results (caching behavior implied by_Documentstructure) and thatdereference_allactually mutates in place vs.dereference(non-mutating) if that distinction exists — check current behavior first rather than assuming.
src/oapi/oas/model.py (90% → target ~97%+)
This file is generated (make remodel) but not pure boilerplate — the
uncovered ranges include real hand-written validation hooks near the end
of the file (_reference_before_setitem, _reference_after_unmarshal,
_parameter_after_validate, _schema_after_validate) plus assorted
version-specific model branches. Don't hand-edit the generated sections;
test them like any other public behavior:
- _schema_after_validate: real Schema/Parameter instances with
invalid type_/format_ combinations (e.g. type_="integer",
format_="date"), assert real sob.errors.ValidationError on
sob.validate(...).
- _parameter_after_validate: a Parameter with both content (>1
entry) and schema set, and one with both content and schema
simultaneously; assert the real ValidationError.
- _reference_after_unmarshal/_reference_before_setitem: unmarshal a
Reference missing $ref → real ValueError; set an arbitrary
extra attribute on a Reference and confirm it's accepted per
patternProperties semantics.
- For the remaining scattered uncovered line ranges (325-336,
1377-1385, 1671-1690, 1750-1766, 4385-4405), identify which
OAS2-vs-OAS3 model classes/branches they belong to
(hatch run hatch-test.py3.10:coverage report -m gives exact ranges)
and cover via the OAS2 (uspto) and OAS3 example fixtures already
exercised in test_openapi_examples/ModelModule tests above —
likely just needs a couple of targeted small fixtures rather than new
infrastructure.
Sequencing
- Land
tests/servers.py+tests/conftest.pyinfrastructure and the three/four new minimal fixtures — nothing else can proceed without these. _utilities.pyand_multipart_request.py(self-contained, fast, unblock nothing else but cheap wins toward 100%).oas/references.pyandoas/model.pyvalidation hooks (self- contained, no server needed except the one HTTP-$reftest).client.pyargument-formatting + content-encoding (pure-function layer) — needed before the end-to-end request tests so failures localize correctly.client.pyClientruntime (http_server-backed): plain requests → retries → auth (API key, then OAuth2) → pickling.client.pyClientModulegeneration, paired with thehttp_server- backed execution of generated clients — largest chunk, do last since it depends on everything above being solid (a bug in the runtimeClientwould otherwise be masked by/confused with a codegen bug).model.pyModelModulegaps (independent of the client work; can be parallelized with step 6 by a second contributor if needed).
Definition of done
hatch test -ccoverage total materially above the 45% baseline; no individualsrc/oapi/*.pymodule below ~85% except where a line range is genuinely unreachable (document any such exclusion with a# pragma: no coverand a one-line reason, don't silently skip).tests/test_client.pyhas real assertions and theTODO/stub is gone.- Every new integration test tears down its
http_server(fixture/ context-manager, not manualtry/finallyper test) and does not leak threads or bound ports across the run. make test(hatch fmt --check && hatch run mypy && hatch test -c -vv) stays green — new tests fully typed (disallow_untyped_defs), new fixtures/golden files reviewed like code, not just accepted frommake refresh-test-dataoutput.