Skip to content

Sync ako/mxcli: exec/check cost no longer scales with statements × project size; REPL sees other writers; building-block and layout fixes - #1840

Merged
ako merged 44 commits into
mendixlabs:mainfrom
ako:main
Nov 12, 2026

Conversation

@ako

@ako ako commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

Syncs ako/mxcli:main into upstream: 24 commits since #1826 — #1822 as submitted, exec and check cost that no longer grows with statements × project size, a REPL that sees other processes' writes, and building-block, layout and catalog fixes.

Performance: a script no longer costs statements × project size

Checking and executing a script listed whole document types once per statement, and every listing decoded each unit from its bytes (about a thousand full microflow listings for 600 page statements). The decision is recorded as ADR-0015 (Proposed): what mxcli keeps of a unit is keyed by the unit's content.

  • A unit's decoded form is kept until its stored bytes change. The cache holds the bytes each entry was decoded from and is used only for equal bytes, so no write path has to invalidate it; an element is shared only once fully loaded, and one a caller changed is decoded again.
  • A write retires only the unit it rewrote, updates the unit list in place, and modules are listed from the index; a check lists the project once, and the decoder borrows cached bytes.
  • A run of entity and association statements writes each domain model once, held and reconciled when the run ends — they used to rewrite the module's whole domain model per statement. Statements still report as they go, and the held write keeps its digest, so each statement digests only what it wrote.
  • Entity lookups read one module's domain model instead of listing and converting every domain model up to ten times per added entity.
  • New: MXCLI_CPUPROFILE writes a CPU profile of a command.

A long-lived session sees other processes' writes

A REPL read the project once and answered from that state: show microflows kept the old set, describe of a flow another process created said "not found", and on MPR v2 the session's own create microflow of a name another process had taken wrote a second document with that name. Before each statement the executor now asks whether anything other than its own connection changed the .mpr (PRAGMA data_version moves only for other connections' commits), and another process's write drops the cached unit bytes.

use building block

Layout

Catalog and CLI

MeisamMahdian and others added 30 commits November 11, 2026 16:35
…loses #1821)

`use building block PlatformDesignSystem.Button (action: ...)` failed with
"action override supplied, but the block has no button to rebind" on every
PlatformDesignSystem block: they are built from the pluggable pdsbutton,
and the target predicate matched only actionbutton, linkbutton and button.

A pluggable widget is now a target when its definition names it a button
and maps a slot to OnClick. The engine already writes that slot from the
Action property the override sets. A click slot alone is not enough: a PDS
Link, a Gallery and a Data grid 2 take OnClick too, and on a PDS page
header the first clickable widget is the "Back to" link. With no project
there is no registry, so only the built-in types match, as before.

Co-Authored-By: Claude Code <noreply@anthropic.com>
…ough help calls it syntax-only

`mxcli check --help` offered "# Check syntax only (no project needed)
mxcli check script.mdl", but run from a directory holding an .mpr the root
PersistentPreRun auto-detects it ("Using project: ... (auto-detected in the
current directory; pass -p to use another)"), and check treats any project
as an implicit --references. There was no way to get the advertised
syntax-only check there (#2135, mendixlabs#1817).

Auto-detection stays: it is the CLI's documented resolution
(mendixlabs#1444), and `cd app && mxcli check script.mdl` relying on
it keeps resolving references. Instead check gains --no-project, which
PersistentPreRun honours for any command registering it; the auto-detect
notice on check names it; it is refused together with -p, --references or
--post-migration. The help now states the default and the opt-out, and its
syntax-only example carries the flag, as do the generated CLAUDE.md
template, the check-script command, two skills and the docs-site pages that
repeated "no project needed".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82sv2odH8kVjmGzPosB
Part 1 of #1841 (mirror of mendixlabs#1692).

describe --format svg domainmodel drew every association centre to
centre and ignored ParentConnection/ChildConnection. Each end now
attaches at its stored anchor, an "x;y" percentage of the drawn entity
box (clamped onto the box when out of 0..100); an end with no readable
anchor keeps the centre-line point. A self-association with anchors is
routed round the outside of its box instead of a zero-length line.

Parts 2 (crossing-minimising layout) and 3 (microflow auto-layout shape)
are feature requests and are not addressed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82sv2odH8kVjmGzPosB
…#2121)

A long-lived session read the project once and answered from that state:
`show microflows` kept the old set, `describe microflow` of a flow created
by another process said "microflow not found", `create entity` in a module
created elsewhere said "module not found", and on MPR v2 the session's own
`create microflow` of a name another process had taken wrote a second
document with that name.

Before each statement the executor now asks the backend whether something
other than its own connection changed the .mpr: PRAGMA data_version (moves
only for other connections' commits, so the session's own writes do not
trip it and the #2094 caches survive) plus os.SameFile on the path (a git
checkout renames a new file into place, which an open SQLite connection
never sees; the backend then reopens). On a change the reader drops its
unit index, v1 listing/names, name index and content cache, and the
executor drops its read caches while keeping its record of the session's
own writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82sv2odH8kVjmGzPosB
…ins handlers backwards

The layout measurer and the flow builder disagreed in three shapes
(#1773), so an object landed on or behind its neighbour while
`layout flows` reported the flow as already laid out:

1. An IF was measured from its split's centre as if that were its left
   edge, 45px short, and a decision that opens a run a further 15px
   short. An if/else followed by an activity in a THEN branch put the
   enclosing merge inside that activity. An IF/ELSE without a merge (one
   branch returns, or the tail of a loop body) was measured with one.
2. A `join` was measured as an activity plus an end event although it
   draws nothing, so a handler row ending in joins reserved about twice
   its width. A both-return IF at a branch's end likewise gained a
   phantom end event, and a `merge` label was measured as an activity.
3. A falling-through handler of two or more statements rejoined at a
   merge placed midway between its activity and the next one, left of
   the row's last activity, so the edge ran backwards. The merge now
   goes one gap past the row and the main line waits for it.

The #1768 invariant test now runs every shape with no skips on the main
line, in loop bodies and in nested branches, and compares each loop's
measured width with the box the builder fits around what it built.

10 of 376 doctype-example flows and 4 of 29 fixture flows move; no
golden test changed. mx check 0 errors on 11.14.0 after layout flows.

Refs #1773

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82sv2odH8kVjmGzPosB
… on the main path

#684 reported `layout flows` introducing MPR008 overlaps at merge
nodes (two after the reporter's correction): a merge 5px from another
merge, and a merge 75px from the activity before it. layout flows rebuilds
with the CREATE builder, which places an IF's closing merge from the
measured width of its branch; a decision inside the branch was measured
short, so the merge landed in what preceded it. The measurer fix is the
previous commit (#1773); this adds the guard the issue proposes,
at the layer the positions are computed in: shapes with consecutive
merges and nested loops, in five contexts, may share no place (MPR008)
and leave no object outside its loop (MPR011), and each is measured as
built. Before the fix 11 shape/context pairs overlapped.

On a fresh 11.14.0 app with these flows and the 02c doctype examples
exec'd by the pre-fix build: 5 MPR008 before, 3 after `layout flows`,
none of them this class; mx check 0 errors; a second run changes nothing.

Refs #684

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82sv2odH8kVjmGzPosB
…s <prefix>`

`check --references` refused a block placed several times with its own
prefix as "duplicate widget name 'Atlas_Web_Content.Card' (used 4 times)
(CE0495)": it counted the `use building block` statement, whose Name is the
block's name, as a widget before expansion. exec expands and prefixes, and
mx check reports no CE0495.

The page-context check now counts the names of the expanded tree, the one
exec builds. The same block placed twice without a prefix is still reported,
by the colliding widget names exec refuses ('container2', 'text22'). With no
project to expand against, a reference contributes no names.

Measured on 11.14.0 with four prefixed Atlas Cards: check passes, exec
writes a_/b_/c_/d_ copies, mxcli docker check reports 0 errors.

Fixes #559

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82sv2odH8kVjmGzPosB
… on the old names

Copied `as md_`, Atlas_Web_Content.Master_Detail's widgets became
md_gallery2 / md_dataView7 but the data view's `selection gallery2` was left
pointing at the old name, and exec failed with "failed to build datasource:
widget not found: gallery2".

prefixWidgetNames now rewrites, in the same pass as the rename, every
reference to a name the block declares: the bare-name fields (selection
datasource, association context variable, `delete $w`, open-link variable)
and every `$name` token in a property value (flow arguments, `$w.Attr`
template parameters, visibility/editability expressions, where clauses).
References to widgets outside the block keep their names. The rewrite is
copy-on-write, so a fragment definition shared by `use fragment … as`
copies is not mutated.

Measured on 11.14.0: the prefixed Master_Detail now builds; with a
datasource override mx check reports only the template's unbound text
boxes (CE7005), the same as an unprefixed placement.

Fixes #560

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82sv2odH8kVjmGzPosB
… for a pluggable button

`use building block PlatformDesignSystem.Button (action: call microflow
PDSRepro.ACT_Noop) as b_` failed with "action override supplied, but the
block has no button to rebind": the block is one pluggable `pdsbutton`, and
the override target was matched against the core button keywords only.

With no core button in the block, the override now binds the first
pluggable widget whose definition has a click action (a mapping with
Source OnClick: PDS Button's onClickAction, Badge Button's onClickEvent,
Image's onClick), read through the widget registry rather than a name list.
Data containers are skipped, since a gallery or data grid click is an item
action. A block that has a core button binds it as before, even when a
clickable pluggable widget precedes it.

PlatformDesignSystem is not obtainable here (marketplace download needs a
PAT; not in the cached catalog). Measured on 11.14.0 with the equivalent
Atlas_Web_Content.Card_WithImage, whose only clickable widget is the
pluggable Image: before, the same refusal; after, the Image gets
`OnClick: call microflow …` and mxcli docker check reports 0 errors.

Fixes #2145 (mendixlabs#1821)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82sv2odH8kVjmGzPosB
…drift-issues-w8o5vt

# Conflicts:
#	CHANGELOG.md
Set to a file path, any mxcli command writes a pprof CPU profile of itself
there. Performance defects in this codebase are chains of individually
harmless per-statement costs, and the reliable way to find one is a
profile of the real command, which until now needed a hand-patched build.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
… command (#1860)

After a model change the next command rebuilt the catalog and printed a
line per table — about fifty — ahead of its answer, so an agent paid for
them in context after every edit. There was no way to turn them off: the
banner check honoured -q, `search` had its own -q, and every other command
rejected it as an unknown flag.

A catalog build now prints its routine lines (table counts, the source
pass's periodic "source n/m documents") only when its output is a terminal.
Anywhere else a quick build prints none of them and a slow one the latest
every 15 s, so a build of minutes still shows it is alive. Lines that carry
information — the incremental summary, documents that could not be
described — print either way. -q / --quiet is a root flag: no banner, no
connection or status messages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ofiles (#1262)

`describe <name>` without a type resolved documents, entities and
associations only. A module role, a user role and a navigation profile each
have a describe type, but are members of the module-security,
project-security and navigation documents, so neither the catalog's objects
view nor the live scan saw them: `describe Administration.Administrator`
answered "no describable document named ...".

They are resolved in the live scan a catalog miss falls back to. The catalog
has no complete module-role table (role_mappings lists only mapped roles),
so the objects view cannot list them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… rules (#1295)

A lint rule could not ask which kind of workflow activity runs a microflow
— "a system task's microflow is prefixed WF_SYS_", "a user task's
targeting microflow WF_TGT_" — because refs records only that the workflow
calls the microflow, and workflows_data only counts activities.

CATALOG.WORKFLOW_ACTIVITIES holds one row per activity of a workflow,
boundary-event paths and event sub-processes included, with its kind
(MultiUserTask and AIAgentTask told apart from UserTask and CallMicroflow)
and the microflow, page or workflow each slot names; an empty slot is NULL.
It is written by every build. workflows() and workflow_activities() expose
workflows_data and the new table to Starlark rules. Catalog schema 48.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ge (#2160)

Checking and executing a script listed whole document types once per
statement, and every listing decoded each unit from its bytes: about a
thousand full microflow listings for 600 page statements, at 0.11 s each.

decodeUnits, the one function every listing and lookup goes through, now
serves a unit it has decoded twice from a per-reader cache. An entry holds
the bytes it was decoded from and is used only for equal bytes, so no write
path invalidates it. An element is shared only after every lazily decoded
value in it is loaded, so reading one writes nothing, and an element a
caller changed is decoded again rather than served. A unit decoded once is
only remembered, so a command that lists a type once pays nothing.

On the PedApp fixture an added page statement decodes 2 units where it
decoded 240. Seeding 300 entities, 600 microflows and 600 pages: 45/42/286 s
per stage before, 25/22/110 s after.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
…8o5vt

fix: building-blocks, flow-graph-geometry, concurrent-access and cli-contract clusters
…ache (#2160)

Every write emptied the reader's whole in-memory content cache, and a
script run held no cache scope at all, so every listing after every write
read every unit file of its type again, through the persisted store or
the file.

A cached unit is now stored under the ContentsHash its Unit row had when
it was read, and served only while the row still has that hash. The
writer already updates the hash, so a write retires its own unit and no
other. What InvalidateCache drops is the hashes: until the next
buildUnitCache reads them back nothing is known to be current, and
nothing is served or stored. A script run now holds the cache.

On the PedApp fixture an added page statement reads 1 unit file where it
read 120. With the decode cache, seeding 300 entities, 600 microflows and
600 pages: 45/42/286 s per stage originally, 20/16/84 s now.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
…2121)

A cached unit is keyed by its row hash, which shows a rewrite only when
the writer moves the hash. This reader's own writes do; another writer
need not. So what SyncWithDisk reports as a foreign change now empties
the content cache, as InvalidateCache did before it was narrowed.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
…bytes (#2160)

Two more per-statement costs under a script, found by profiling after the
decode and content caches.

The check before an exec rebuilt its entity indexes and flow signatures
for every statement, each from a full listing: four of the domain models,
three of the microflows and two of the nanoflows per page statement. A
check never writes the project, so one pass now lists each kind once
(checkPassMemo, nil during execution). The widget template metadata is
likewise loaded once per pass: its cache was created inside a call made
per statement.

Every listing copied each unit's bytes out of the content cache for a
decoder that only reads them, and the decode cache then compared them
with its own. The decoder is now handed the cache's own slice
(Reader.ListUnitsByTypeShared), and an unchanged unit is recognised by
slice identity.

Seeding 300 entities, 600 microflows and 600 pages: 45/42/286 s per stage
originally, 20/14/44 s now.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
…s content

Records the decision behind the #2160 changes: content-keyed caches below
the call sites, the three rules for sharing a decoded element, the split
between a check (may list once) and execution (may not), and the
alternatives that were not taken. Status is Proposed until review.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
perf: stop decoding and re-reading the project once per statement (#2160)
…the index (#2160)

Every write called Reader.InvalidateCache, so the next read scanned the
whole Unit table to rebuild the list of units for a write that added or
changed one row. The writer's v2 insert and update now patch the list
(noteUnitWritten); anything they cannot account for - a delete, a move, a
transaction, a stale list - invalidates as before.

The module listing that follows every created document, to rebuild the
container hierarchy, read each module's file twice: once to unmarshal the
whole document for its name, once to decode it for its Marketplace
details. The name now comes from the unit index and the details through
the decode cache. A module's Go type is registered under two storage
names, so the by-ID lookup that matches one name would not find it:
UnitByIDAs accepts either.

Seeding 300 entities, 600 microflows and 600 pages: 20/14/44 s per stage
before, 16/9/36 s after.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
perf(mpr): a write updates the unit list in place; modules list from the index (#2160)
…h domain model once (#2160)

CREATE ENTITY and CREATE ASSOCIATION each rewrote the module's whole
domain model, and every rewrite was reconciled against the stored unit
and written: half the time of a script that builds a domain model, and a
cost per statement that grows with the model.

Consecutive plain-entity and association statements now form a run whose
domain-model writes are held and reconciled against storage when it ends
(entity_run.go), on the mechanism access-rule runs use. Unlike those, the
statements report as they go: the writer judges each held write as it is
held and counts it in WriteStats like a write that landed
(Writer.DeferUnitWritesJudged) - the first write of a unit against
storage, later ones against the write already held, both normalised for
storage and with identity carried, so a settled re-run still reports
every statement unchanged.

What changes is when a write can be refused: the reconcile against
storage, and the storage-GUID guard in it, runs at the end of the run.

Seeding 300 entities and 299 associations: 17 s before, 10 s after.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
The nightly went red on the 11.14.0, 11.15.0 and 11.16.0 legs, in
TestRunLifecycle_DetachStatusExecWaitStop: "build #2 applied without mxcli
restarting mxbuild --serve; if this mxbuild builds incrementally again, drop
the 11.14+ branch". The log shows "build #2 applied via reload in 3.7-8.5s".

Not a product defect: #2150 (9156cf4) keeps the client source a
serve build deletes and puts it back before the next, so --watch builds
incrementally on the versions that bundle the client themselves. The test's
11.14+ branch still pinned the serve restart that change removed.

The branch goes: on every version a page change must apply via reload, and
the run log must not say "restarting mxbuild --serve", which every fallback
that replaces the serve process prints. A regression of the keep fails it.
The boot-race step still only claims build #2 applies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GeAhBjRBm5vN2ik3ddvVWW
…removed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GeAhBjRBm5vN2ik3ddvVWW
ako and others added 14 commits November 12, 2026 11:57
perf(executor): a run of entity and association statements writes each domain model once (#2160)
…5joqyu

test(run): the lifecycle test wants build #2 as a reload on 11.14+ again
feat(cli): catalog progress per table only on a terminal; -q on every command (#1860)
feat(catalog): workflow activities with their kind and slots for lint rules (#1295)
…luggable button

Integrates MeisamMahdian's fix for mendixlabs#1821, which ako/main
had also fixed (7bad955, #2145) with a different target rule.
Each got one case wrong:

- ako's took the first pluggable widget with a click action when the
  block had no core button, so on a PDS page header, whose first
  clickable widget is a "Back to" PDS Link, the action landed on the
  link;
- #1822 took only widgets named as buttons, so a block whose only
  clickable widget is a pluggable Image (Atlas's Card_WithImage) was
  still refused, and it let a pluggable button before a core button win
  over it.

The conflict is resolved as three tiers, each over the whole block:
a core button, then a pluggable button (#1822's rule, reusing
pluggableClickActionSlot so data containers are excluded and
lookupWidgetDef resolves the definition), then any other clickable
pluggable widget (ako's rule). rebindBuildingBlockAction holds them;
#1822's tests are adapted to it, plus a test for the last tier. One
CHANGELOG entry and skill text describe the combined rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiMsnXa9crge2qnSk9S7Zn
Each added entity and its association listed and converted every domain
model of the project ten times: four in findEntity (CREATE ASSOCIATION
resolves each end twice), four in the dispatch's name-space guard, two
in the member-name check. The conversion is 70% of a listing and the
decode cache does not save it.

findEntity and the name-space guard ask about one named module, and now
read that module's domain model. Both keep the listing as the fallback
for what a single read does not answer: the virtual System module, a
module that does not exist yet, a backend that serves only the listing.
The member-name check (CE0069) follows generalizations across modules
and still lists, once per statement.

Listings per added entity and association: 10 -> 2. Seeding 300
entities and 299 associations: 9.7 s -> 8.4 s; re-running the settled
script: 14.5 s -> 9.6 s. Refusals compared against the previous binary
on six cases, identical.

Refs #2160

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
Integrate mendixlabs#1822: building block action override rebinds a pluggable button
perf(executor): entity lookups read one module's domain model
To tell whether a statement of an entity run changed anything, its write
was compared with the write the run already held through canon.Equal,
which digests both documents: the held domain model was unmarshalled and
hashed again for every statement. That was 1.7 s of the 2.9 s spent
judging held writes when seeding 300 entities and 299 associations.

The run now holds the identity-carried form of each write together with
its digest, so a statement digests only what it wrote. Holding the
carried form changes no verdict: the next write takes the same identity
fields before it is compared.

Digests per added statement: 2 -> 1. The seed: 7.1 s -> 6.4 s on a
first run, 9.6 s -> 8.7 s on a settled re-run; output identical to the
previous binary on four runs, one with a single changed statement.

Refs #2160

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
fix(describe): auto-detect module roles, user roles and navigation profiles (#1262)
Both sides added a changelog entry at the same place; both are kept.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm
perf(mpr): a judged run keeps the digest of the write it holds
@ako
ako merged commit 35a33ff into mendixlabs:main Nov 12, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

action: override on use building block does not find pluggable buttons (PDS Button) A catalog banner after every model change; add a quiet mode

3 participants