Repository navigation
Conversation
group_columns_by_name rebuilt the grouped array with concat for every duplicate column, making each record O(n^2) in the number of duplicate headers (40k duplicate headers, ~160 KB of input: ~6.6 s per record). Push into the existing array instead, flattening array values to keep concat semantics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| // Append in place; `concat` copied the array on every | ||
| // duplicate, making grouping quadratic in the column count. | ||
| // Array values are flattened to preserve `concat` semantics. | ||
| const group = obj[columns[i].name]; |
There was a problem hiding this comment.
This branch now also runs when the first value of the column is an array that came from cast. Then group is the caller's own array, not one the parser made, so push changes it. Before, concat returned a new array, so nothing was mutated.
If cast returns the same array more than once (a shared empty value, a cache), it breaks:
const NONE = [];
parse("tag,tag\n,x\n,y\n,", {
columns: true,
group_columns_by_name: true,
cast: (v, { header }) => (header ? v : v === "" ? NONE : v.split("|")),
});
// master: [{tag:["x"]},{tag:["y"]},{tag:[]}], and NONE stays []
// this PR: RangeError: Invalid array lengthRows 1 and 2 push x and y into NONE. In row 3 both cells are NONE, so group === record[i] and the for...of keeps pushing into the array it is reading from until it runs out of length.
A fix that keeps the linear cost: track which names the parser has already grouped in this record. Use concat once, when the group is created, and push after that:
const obj = {};
const grouped = new Set();
// ...
if (grouped.has(columns[i].name)) {
// push / flatten as you have it now
} else {
obj[columns[i].name] = Array.isArray(obj[columns[i].name])
? obj[columns[i].name].concat(record[i])
: [obj[columns[i].name], record[i]];
grouped.add(columns[i].name);
}With that change the repro above gives the same output as master. The full csv-parse suite passes (632), and your 50k-column test still runs fast.
Summary
With
group_columns_by_name: true, each duplicate column is appended withobj[name] = obj[name].concat(value). That copies the whole group on every duplicate, so building one record is O(n²) in the number of duplicate headers.Measured with
csv-parse@7.0.3, sync API, one data row, every header nameda:That cost repeats for every data row. When the CSV comes from an untrusted source, a small file can block the event loop for a long time.
Change
push. Array values are still flattened, which keeps the oldconcatsemantics.cast(same output before and after)Full csv-parse suite: 632 passing. ESLint is clean.
🤖 Generated with Claude Code