[yawkat]

Follow-ups to #175: two more cases where the Java decompressors accept blocks that the native (lz4 1.11.0) decoders reject, plus tests. One commit per change.

1. Fast decompressors: require LAST_LITERALS after the last match

The fast decompressors pass the end of the source buffer as srcEnd, so #175's input-side check (lenLimit = srcEnd - (1 + LAST_LITERALS)) is relative to the buffer end, not the block end. 1F 07 01 00 00 40 01 02 03 04 with destLen = 24 (1 literal, a 19-byte match, 4 last literals) is rejected from an exact-size array but accepted (returning 10) as soon as any byte follows it. The deprecated new LZ4BlockInputStream(in) uses the fast decompressor with a reused, larger buffer, so it never rejects it.

LZ4_decompress_fast rejects such blocks with an output-side rule: after a match, oend - op < LASTLITERALS is an error. The fast decompressors now apply the same rule (matchCopyEnd > destEnd - LAST_LITERALS) in the two branches taken near the end of the output (the matchDec == 0 branch and the notEnoughSpace(destEnd - matchCopyEnd, COPY_LENGTH) branch). The wild-copy branch already guarantees COPY_LENGTH >= LAST_LITERALS bytes remain, so the hot path is unchanged. The comment from #175 now says which decoder enforces what.

2. Last match must start MF_LIMIT bytes before the end of the output

The decoders took the "these must be the last literals" path when the literals end less than COPY_LENGTH (8) bytes before the destination end. The native decoders use MFLIMIT (12): LZ4_decompress_safe rejects cpy > oend - MFLIMIT unless the literals end the block, LZ4_decompress_fast rejects oend - op < MFLIMIT unless op == oend. So with a tight destination, 10 41 01 00 50 42 43 44 45 46 (1 literal, a 4-byte match starting 9 bytes before the end, 5 last literals) with a 10-byte destination was rejected by JNI but decoded by the Java safe, unsafe and fast decompressors.

The output-side operand is now MF_LIMIT; the input-side COPY_LENGTH + literalLen operand is unchanged. Since MF_LIMIT >= COPY_LENGTH, this only routes more sequences to the checked last-literals path, so wildArraycopy keeps its headroom. Spec-compliant encoders never start a match within the last 12 bytes, so valid blocks are unaffected; a block whose last match starts exactly 12 bytes before the end still decodes with an exact-size destination.

3. Tests

In LZ4Test, for all safe and fast decompressors (Java and JNI), byte[] and direct ByteBuffer:

• testMatchFollowedByTooFewLastLiterals: the Reject blocks the native decoder rejects since lz4 1.11.0 #175 block and a variant with two match-length bytes (1F 07 01 00 FF 00 40 01 02 03 04) are rejected, including by the fast decompressors with 32 trailing bytes after the block. Their valid twins with 5 last literals decode to 25 and 280 bytes.
• testLastMatchStartsTooCloseToEnd: the MF_LIMIT block above is rejected with an exact 10-byte destination. 20 41 42 01 00 80 43 ... 4A, whose last match starts exactly 12 bytes before the end, decodes to 14 bytes with an exact-size and with a larger destination.

Both new tests fail on current main (JNI passes, Java fails). The full local verify passes.

🤖 Generated with Claude Code