Skip to content

Python-side step before json.dumps is up to 27 percent slower in main than in 26.11.1 on plain scalar tables without numpy #371

Closed

Description

@tae898

The step that turns rows into JSON text before they go to the engine (json_bulk_dumps_tagged for insert_many, the vertex screen in create_vertices) is slower in main than in the 26.11.1 tag when numpy is not installed. The measurement is in #318 (comment): 7 of 9 cells are clearly slower, down to 0.73x of the tag's speed (8 columns, 250,000 rows), on plain scalar tables, one core. With numpy the two builds are within about 11 percent of each other. numpy is only an extra, so the slower case is the default install.

My reading, not yet checked: without numpy a float column is judged value by value, and the new census costs more than the old loop did.

Done when: on the same cells without numpy main is not slower than the 26.11.1 tag, measured the same way, and with numpy it is not slower either, or the reason is written here. I'll take it.

Metadata

Metadata

Assignees

Labels

in progressThe team is actively working on this issueperformanceSpeed or memory of the bindings, with a measurementpythonPull requests that update python code

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions