[tae898]

What does this PR do?

The step before json.dumps in insert_many and GraphBatch.create_vertices is no longer slower than in the 26.11.1 tag, with numpy and without it. Main was slower on plain scalar tables: without numpy a float column was judged by a function call per value, and with numpy the five passes of the column screens over a table that does not fit the CPU cache cost more than the one pass of the old loop. One algorithm now, numpy is no longer used by the screens:

• A float column is screened by its sum (NaN and the infinities make it non-finite; an overflow is a false alarm that the judgement by value then passes) and the smallest non-zero absolute value (a subnormal).
• The column screens run over blocks of about 16,384 values with the block as the outer loop, so the passes over a block (list, census, min and max, join, sum) hit the cache.
• _dumps_checked looks for the subnormal exponent with the substring e-3 first and runs the regular expression only when it is there. The expression cost about half of json.dumps on a text of digits, in the tag and in main.
• Rows that the census shows hold no list, tuple or dict are written with check_circular=False; rows the walk looked at keep it.
• The walk _check_rows stays the exact reference. It runs before the text when the census asks for it and after a refusal of json.dumps when it did not, so the first bad row in row order still gives the old exception type and message.

I did not keep the old per-row loop as a no-numpy branch: the blocked column screens are cheaper than the loop with and without numpy (table below), so two paths would only add code.

Related issues

Closes #371

Performance changes

Only the Python-side step is timed (no engine, no JVM, one core), not end-to-end ingest: median CPU microseconds per row, the tag 26.11.1 (2f28a328be), main (c90b3f9), this branch (type_conversion.py sha256 6715a4e09b54b90c), and the plain json.dumps of the same rows. Interleaved, warmed, 21, 7 or 5 repeats, the order alternating, with a control of main against itself in the same run. The same nine cells as the #318 study, Intel Core Ultra X9 388H, cores 12,13 (LP-E), Python 3.12, numpy present and absent. A cell counts as faster or slower only when the ranges of the repeats are separated.

numpy present (median CPU microseconds per row; ratio above 1 means cand is quicker)

cell	26.11.1 tag	main	cand	json.dumps	tag to cand	main to cand	control: main to main
insert_many, 3 columns, 10,000 rows	0.756	0.724	0.515	0.366	1.47x, faster	1.41x, faster	1.00x, not separated
insert_many, 8 columns, 10,000 rows	1.801	1.950	1.325	0.851	1.36x, faster	1.47x, faster	1.01x, not separated
insert_many, 8 columns, Hangul, 10,000 rows	2.041	2.092	1.429	0.936	1.43x, faster	1.46x, faster	0.99x, not separated
create_vertices, 3 columns, 100,000 rows	0.765	0.668	0.579	0.425	1.32x, faster	1.15x, faster	1.00x, not separated
create_vertices, 3 columns, 250,000 rows	0.792	0.750	0.607	0.459	1.31x, faster	1.24x, faster	1.00x, not separated
create_vertices, 8 columns, 100,000 rows	1.766	1.925	1.452	1.000	1.22x, faster	1.33x, faster	1.01x, not separated
create_vertices, 8 columns, 250,000 rows	1.756	2.091	1.445	0.989	1.22x, faster	1.45x, faster	1.01x, not separated
create_vertices, 8 columns, Hangul, 100,000 rows	1.805	1.816	1.403	0.953	1.29x, faster	1.29x, faster	0.99x, not separated
create_vertices, 8 columns, Hangul, 250,000 rows	1.919	1.997	1.465	0.997	1.31x, faster	1.36x, faster	1.02x, not separated

numpy absent (median CPU microseconds per row; ratio above 1 means cand is quicker)

cell	26.11.1 tag	main	cand	json.dumps	tag to cand	main to cand	control: main to main
insert_many, 3 columns, 10,000 rows	0.763	0.714	0.504	0.363	1.51x, faster	1.42x, faster	0.99x, not separated
insert_many, 8 columns, 10,000 rows	1.801	1.996	1.330	0.856	1.35x, faster	1.50x, faster	1.01x, not separated
insert_many, 8 columns, Hangul, 10,000 rows	2.042	2.132	1.441	0.930	1.42x, faster	1.48x, faster	1.01x, not separated
create_vertices, 3 columns, 100,000 rows	0.757	0.842	0.579	0.426	1.31x, faster	1.45x, faster	0.99x, not separated
create_vertices, 3 columns, 250,000 rows	0.790	0.921	0.615	0.454	1.28x, faster	1.50x, faster	0.98x, not separated
create_vertices, 8 columns, 100,000 rows	1.801	2.277	1.469	1.006	1.23x, faster	1.55x, faster	1.00x, not separated
create_vertices, 8 columns, 250,000 rows	1.809	2.444	1.471	1.004	1.23x, faster	1.66x, faster	1.00x, not separated
create_vertices, 8 columns, Hangul, 100,000 rows	1.853	2.205	1.420	0.955	1.31x, faster	1.55x, faster	1.01x, not separated
create_vertices, 8 columns, Hangul, 250,000 rows	1.954	2.417	1.464	1.023	1.33x, faster	1.65x, faster	1.01x, not separated

Acceptance, as written before the run:

• cand: cells (of 18) where it is slower than the 26.11.1 tag with separated ranges: 0
• cand: cells (of 9, numpy present) where it is slower than main today with separated ranges: 0
• cand: acceptance MET

The files are in the private bench notes (repros/bind-371-fix-20261112): iter2 run, with the fuzz of the new code against the tag and main (20,000 random row sets per arm, every value class, text or exception type and message equal, create_vertices verdicts equal to main's).

Docs

None needs it: no page describes this internal step.

Checklist

• A regression test fails on the old code and passes on the new (the new pure tests fail on main c90b3f9 in 76 cases)
• The test suite that the change can affect passes: on the first CI run 3381 tests passed and 2 failed, the two quoted-parallel import tests of Tests: the two quoted-parallel import tests fail on the 26.12.1-SNAPSHOT engine since arcadedb#11367 was fixed #376 that fail on every pull request (fixed by Tests: the two quoted-parallel import tests assert OK on every engine, the 26.12.1 refusal is fixed upstream #378); the leg is re-run after Tests: the two quoted-parallel import tests assert OK on every engine, the 26.12.1 refusal is fixed upstream #378 is in
• uvx pre-commit run --files <changed files> passes (black, isort, ruff, bandit clean)

🤖 Generated with Claude Code