Skip to content

Return precomputed bytes for single-byte indexed fields - #305

Open
DicksonWu654 wants to merge 1 commit into
python-hyper:masterfrom
DicksonWu654:perf/indexed-field-bytes
Open

DicksonWu654 wants to merge 1 commit into
python-hyper:masterfrom
DicksonWu654:perf/indexed-field-bytes

Conversation

@DicksonWu654

Copy link
Copy Markdown

Every header that is already in the static or dynamic table is encoded by Encoder._encode_indexed(), which calls encode_integer() (building a bytearray), sets the top bit and copies the result to bytes. Any index below 127 fits in the 7-bit prefix and always encodes to the single byte 0x80 | index, so this looks those up in a 127-entry tuple of precomputed bytes instead.

  • Other values and types (127 and above, bool, negative numbers, non-ints) still go through encode_integer(), with the same output and errors.
  • When the hpack logger has DEBUG enabled, the existing path is used as well, so the Encoding %d with %d bits records (including their funcName) are unchanged.

The table adds 127 one-byte bytes objects at import time.

Benchmarks

Each workload runs header blocks through Encoder.encode() and, for *_roundtrip, Decoder.decode(). browser_mixed alternates 16 browser-like requests and 16 responses; repeated_requests sends one request 16 times, so after the first block every field is indexed; long_cookie is 16 requests whose cookie is about 1.2 KB; no_huffman_encode is browser_mixed with huffman=False; fixture_stories_roundtrip replays the first four stories of the hpack-test-case fixtures in tests/.

master and this branch were imported side by side in the same interpreter and timed in alternating ~20 ms CPU-time blocks (40 rounds in each of 6 processes, pinned to one core). The table shows the median per-round speedup with a 95% bootstrap interval. The machine was shared with other work, so the absolute times are only indicative.

Workload CPython 3.12.3 master → branch (µs) CPython 3.12.3 speedup CPython 3.14.8 speedup
browser_mixed_encode 1,794.3 → 1,695.9 1.06x [1.03, 1.08] 1.06x [1.05, 1.08]
browser_mixed_roundtrip 3,938.6 → 3,755.2 1.04x [1.02, 1.06] 1.02x [1.01, 1.04]
repeated_requests_encode 414.7 → 337.5 1.24x [1.22, 1.26] 1.23x [1.22, 1.25]
repeated_requests_roundtrip 953.5 → 872.3 1.10x [1.07, 1.13] 1.11x [1.09, 1.12]
responses_decode 669.4 → 666.5 1.01x [0.99, 1.03] 1.00x [0.99, 1.02]
long_cookie_encode 7,183.9 → 7,029.2 1.00x [0.99, 1.02] 1.01x [1.00, 1.02]
long_cookie_roundtrip 12,117.6 → 11,750.4 1.02x [1.00, 1.03] 1.00x [0.98, 1.01]
no_huffman_encode 1,028.6 → 944.7 1.10x [1.09, 1.12] 1.09x [1.07, 1.10]
fixture_stories_roundtrip 2,086.2 → 1,987.1 1.06x [1.04, 1.08] 1.07x [1.05, 1.08]

Behaviour

  • New tests: the indexed encoding at the 7-bit prefix boundary (0, 1, 126, 127, 128, 255, True), invalid values, and a DEBUG-logging round trip that checks the redaction of sensitive values and that the integer-encoding records still come from encode_integer.
  • pytest passes with 100% statement and branch coverage on CPython 3.11, 3.12, 3.13 and 3.14; ruff check src/ and mypy src/ (the lint tox env) pass.
  • I also compared master and this branch on 1,500 random encoder/decoder sessions per interpreter (CPython 3.11 and 3.14), including repeated headers so the dynamic table is hit, table size changes, sensitive headers and direct _encode_indexed() calls with indices from 0 to 200. The encoded bytes, decoded headers, exceptions and, with DEBUG logging enabled, every log record were identical.

🤖 Generated with Claude Code

Encoder._encode_indexed() builds a bytearray through encode_integer(),
sets the top bit and copies it to bytes for every indexed header, even
though any index below 127 encodes to a single fixed byte. Look those
up in a 127-entry tuple instead.

Other values and types keep using encode_integer(), with the same
validation and errors. When DEBUG logging is enabled the original path
is used as well, so the "Encoding ..." debug records are unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant