Skip to content

Sync ako/mxcli: mxcli test runs tests in the live app (--allow-call); case-branch splice; layout grid insert; catalog flows - #1806

Merged
ako merged 47 commits into
mendixlabs:mainfrom
ako:main
Nov 10, 2026
Merged

ako merged 47 commits into
mendixlabs:mainfrom
ako:main

Conversation

@ako

@ako ako commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator

Syncs ako/mxcli:main into upstream: 12 commits since #1793 — mxcli test can run tests in the live app without building them, plus microflow, page, catalog and agent fixes. 3 upstream issues fixed.

mxcli test: run a test in the live app without building it

  • New: --allow-call. run --local --test-endpoint --allow-call switches on an exec route in the test endpoint, and test --attach compiles each .test.mdl test into versioned steps the app runs with the Core API, every expression evaluated by Core.evaluateExpression: nothing is injected or built. Verified on 11.12.1: seven tests give identical verdicts and messages on both routes, 0.26 s against 3.2 s. Because the runtime evaluator returns empty for an unknown name, an undeclared variable or unknown enumeration value is refused (ERROR), never compared silently against empty; a model newer than the running build is refused. The route takes POST, needs the token and a loopback caller, and answers 403 without --allow-call; run warns when it is combined with --hub. Design from the review of feat: mxcli test call — JSON runtime executor against an already-running app #1789 (refs feat: mxcli test call — JSON runtime executor against an already-running app #1788); a spike measuring Core.evaluateExpression is included.
  • Each test is routed on its own: an attach injects and builds only the tests the executor can't run, and the executor now runs count/sum/average/minimum/maximum over an attribute and retrieves over an association. A differential run found Mendix's average over an empty list is empty, not 0; the executor agrees.
  • A test run holds the dev loop's watcher instead of making a run --local --watch loop rebuild and restart for the injection and its cleanup (which dropped every browser session). The hold (.mxcli/watch-hold.json) records the model digest; on release nothing is built if the model is the one the hold began from, and --attach now also restores the .mpr byte for byte.
  • No phantom "change pending" under --test-endpoint (run status / run wait now ignore the build's own theme scaffold, as the watcher already did), and an endpoint left behind by a killed run is removed by the next mxcli run, with run and check -p warning while AfterStartupMicroflow still points at it.

Microflows

Pages

Catalog and lint

  • Sequence flows and activity positions for lint rules. CATALOG.FLOWS holds one row per sequence and annotation flow, with origin and destination, IsErrorHandler, CaseValue and the connection sides; flows_for() exposes it, and activities gain X, Y, Width and Height — so a rule can tie a log to its error handler, a note to its activity, or a branch to its case value. Fixes Starlark lint API: no sequence flows (error handler path, annotation link, split branches, canvas position) #1308
  • Agent tools match ToolCount: AGENT_TOOLS no longer includes consumed MCP services (they're in AGENT_MCP_SERVICES), and CONSUMED_MCP_SERVICES gains Endpoint and AuthenticationMicroflow, with refs to both. Catalog schema 44.
  • MDL-AGENT02/03 now fire for a new agent or MCP service written with create or modify (they skipped every create or modify, since an omitted key keeps the stored value — but a new document has none). Lint gains MPR014 (agent without a selected model) and MPR015 (consumed MCP service without an endpoint).

Docs

  • A genai-runtime skill for the runtime side of Agents Kit 2: registering a Mendix Cloud GenAI key or OpenAI-compatible endpoint, seeding agents from JSON, Conversational UI chats, PgVector and Mendix Cloud knowledge bases, traces — with the gotchas no check sees (the shared encryption key, the vector extension only the UI's Test button creates, the indexing delay, StoreTraces off by default)

claude and others added 30 commits November 10, 2026 13:36
`alter page M.P { insert into lgMain { row { column { … } } } }` failed
with "row needs a name" and, named, "not a simple container": the
INSERT's nodes were built as widgets and appended to a Widgets list,
and a layout grid row is neither. Rows now get their own path, like tab
pages (#1215): built with the CREATE PAGE row builder and appended to
the grid's Rows through InsertLayoutGridRows. A row has no stored name,
so BEFORE/AFTER is refused naming the INTO form, and mixing rows with
widgets in one INSERT is refused. `syntax page.alter` shows the form.

Verified on TestApp: both reported forms append, describe -> create or
modify reports Unchanged, and mx check reports 0 errors.

Fixes #2071

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82qrr1WnXxq1eRn7qRJ
…1299)

A Starlark rule could not see which activity leads to which, so it could
not tie a log to its custom error handler, an annotation to the activity it
explains, or a split's branch to its case value. activities_for() had no
canvas position either.

CATALOG.FLOWS holds one row per sequence and annotation flow of a
microflow, nanoflow or rule, with origin and destination activity Ids,
IsErrorHandler, CaseValue and the connection sides. flows_for() exposes it
to rules, and activities gain X, Y, Width and Height. Catalog schema 44.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	CHANGELOG.md
…he catalog (#2090)

CATALOG.AGENT_TOOLS held an agent's consumed MCP services as well as its
tools, so it disagreed with AGENTS.ToolCount and the documented
"tools that run without approval" query counted MCP services. They are now
only in AGENT_MCP_SERVICES, which MCPServiceCount counts.

CONSUMED_MCP_SERVICES gains Endpoint and AuthenticationMicroflow, and REFS
records the endpoint constant (new `endpoint` kind) and the credentials
microflow (`call`), which `list callers` reported as uncalled. Schema 44.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82ttaGzay5CwiC2V7cf
…t model or endpoint (#2089)

MDL-AGENT02/03 skipped every create or modify, because a key it leaves out
keeps the stored value. That also skipped a new document, which is how
project scripts and describe create one. The reference pass, shared by
check --references and exec, now replays the script over the stored
documents and warns when there is nothing to keep. The agent half is gated
on AgentEditorCommons 2.2+, the first version that reads selectedModel.

Lint gains MPR014 (agent without a selected model) and MPR015 (consumed MCP
service without an endpoint), over the catalog, so a stored document is
caught whoever wrote it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82ttaGzay5CwiC2V7cf
# Conflicts:
#	CHANGELOG.md
Registering a Mendix Cloud GenAI key or an OpenAI-compatible endpoint,
seeding AgentCommons agents from JSON, resuming Conversational UI chats,
loading PgVector and Mendix Cloud knowledge bases, and traces, with the
gotchas no check sees: the encryption key both the OpenAI and PgVector
setup need, the vector extension only the UI's Test button creates,
Agent_GetCreate_Json matching on Title only, the 60-120 s indexing delay,
the chat that collapses outside its page, and StoreTraces off by default.

Signatures and behaviour read from the Agent Builder Starter App's modules
(AgentCommons 4.4.0, ConversationalUI 7.4.0, GenAICommons 7.3.0,
MxGenAIConnector 7.4.0, OpenAIConnector 9.4.0, PgVectorKnowledgeBase 7.1.0).
The examples pass check --references there and build with 0 errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82ttaGzay5CwiC2V7cf
…24rqgv

pages: alter page insert into a layout grid appends rows
`create or modify microflow` was refused with "replace case $Source:
cannot replace the ExclusiveSplit … it has 9 paths, not two" whenever
anything differed inside one branch of a case, and with "drop case"
when the case sat in one changed run with activities replaced before
it. The body diff had shell matches for an if and a loop, but none for
an enumeration case, so the case was replaced or dropped as a whole,
and the splice only takes out a two-path decision.

sameCaseShell matches a case with the same subject and branches, in the
same order, and diffs each branch the way an if's are; casePair splits
a run of changes around such a case, as loopPair does for a loop.

Verified on TestApp: the unfixed binary refuses both shapes; the fixed
one splices them, a re-run reports Unchanged, and mx check reports 0
errors.

Fixes #2035

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82qrr1WnXxq1eRn7qRJ
feat(catalog): sequence flows and activity positions for lint rules (#1299)
…24rqgv

microflows: splice a change inside a multi-branch case
…(--allow-call)

run --local --test-endpoint --allow-call switches on an exec route in the
test endpoint. test --attach compiles each test into versioned steps
(interp.go) and the app runs them with the Core API, every expression
evaluated by Core.evaluateExpression; nothing is injected or built. A
suite with a statement the executor does not support takes the usual
inject-and-build route, naming the statement; verdicts are the same
strings either way.

Because the runtime evaluator returns empty for an unknown name, an
undeclared variable is refused by the compiler and by the executor, and
an unknown enumeration value by the executor (ERROR, never a silent
compare against empty). A model newer than the running build is refused,
with `mxcli run wait` as the fix under --watch.

The route takes POST, needs the token and a loopback caller, and answers
403 without --allow-call. Verified on an 11.12.1 app: seven tests give
identical verdicts and messages on both routes, 0.26 s against 3.2 s.

Design from the review of mendixlabs#1789 (refs mendixlabs#1788).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiMsnXa9crge2qnSk9S7Zn
fix(catalog): agent tools match ToolCount; consumed MCP endpoint in the catalog (#2090)
…pression engine

Probes a microflow expression evaluator in the running app: types it
returns, uncommitted objects and association paths, errors, the silent
nulls (unknown variable, unknown enum value, $-prefixed map key), and the
cost per call. Measured on 11.12.1; findings in the file header. Input
for the interpreter route discussed on mendixlabs#1789.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiMsnXa9crge2qnSk9S7Zn
mxcli test --attach runs a .test.mdl test without building it (--allow-call)
fix(check): flag create or modify agents and MCP services left without model or endpoint (#2089)
…me-skill

# Conflicts:
#	.claude/skills/mendix/agents/SKILL.md
…ation retrieves in the app

From the field test on #2101 (16 tests, Mendix 11.16): one
unsupported statement sent the whole run through inject-and-build, and
the commonest ones were list aggregates and retrieves over associations.

- An attach to an --allow-call app now injects and builds only the tests
  the executor cannot run; the rest of the run still goes to the app.
- The executor runs count/sum/average/minimum/maximum over an attribute
  as statements, and a retrieve over an association: the owner side of a
  reference through the evaluator (so an uncommitted object is found),
  otherwise the database list, as a microflow does.
- A refusal names the MDL statement ("the statement if", "a list
  operation (find)") instead of the AST type.
- run warns on --allow-call with --hub: the tunnel connects from
  loopback, so the token is the only guard.

Checked differentially on 11.12.1 (mdl-examples/spikes/test-interpreter):
the first comparison found Mendix's average over an empty list is empty,
not 0; the executor now agrees, and both routes give identical verdicts
and messages, a mixed run included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiMsnXa9crge2qnSk9S7Zn
docs(skills): genai-runtime, the runtime side of Agents Kit 2 (#2068)
…ftover endpoint is not committed unnoticed

From the field test on #2101.

`run status` reported "change pending" for the whole life of a
`run --local --watch --test-endpoint` session and `run wait` waited into
its timeout. The boot build scaffolds themesource/mxtest/; the watcher
already treats a build's own scaffold as output (#2033), but the
run state's readers compared the unfiltered signal. The scaffold is now
recorded in run-state.json and status/wait use the watcher's signal
(SourceMTimeOf); an edit to those files still counts.

A run killed with its container left AfterStartupMicroflow pointing at
MxTest.RegisterEndpoint until a `run stop` nobody ran. Any `mxcli run`
now removes an abandoned endpoint before booting, and `run` and
`check -p` warn while the setting is the endpoint's with no run hosting
it: `run stop` with a restore record, the two commands that undo it by
hand without one.

Verified on an 11.12.1 app: no pending change after boot, run wait
returns at once, an edit to the scaffolded folder still builds; a
SIGKILLed hosted run is warned about by check -p and removed by the next
plain run; without the restore record the manual commands clear it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiMsnXa9crge2qnSk9S7Zn
…form

The conformance gate refuses MDL-DEPR004 in example files: the call form
`$n = count($rows)` is deprecated for `$n = count $rows` and
`sum $rows by Amount`. Same aggregate, so the executor sees the same
statement; the unit test uses the canonical form too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiMsnXa9crge2qnSk9S7Zn
ako and others added 17 commits November 10, 2026 20:04
…#2094)

On an MPR v1 project every typed listing rescanned the metadata of the
whole Unit table, and every lookup by name read the contents of every unit
of the type, so exec and check paid 0.3-0.9 s per statement on a 3,400-unit
project whatever the statement changed.

The reader now keeps the scan until a write through it (InvalidateCache,
which v1 updateUnit now calls too) or a commit by another connection
(PRAGMA data_version; file size and mtime as a second signal) says the
table may have changed. Unit names are kept by length and ContentsHash,
and dropped wholesale on a foreign commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run: no phantom "change pending" under --test-endpoint; a leftover endpoint is not committed unnoticed
…it rebuild and restart

Field test on #2104 (and #2063): `test --local`, and
`--attach` with a test that has to be built, write their test microflows
into the project a `run --local --watch` dev loop watches. The dev loop
rebuilt for the injection and its cleanup and applied by restarting the
runtime, dropping every browser session (--hub preview included).

The test run now writes .mxcli/watch-hold.json (pid, model digest, the
dev loop's change signal) before it touches the project and removes it
after cleanup; --attach now also puts the .mpr back byte for byte, as
--local already did. While a live process holds it the watcher does not
build; on release it builds nothing when the model is the one the hold
began from and no edit was pending before it, and records the source as
covered so `run status`/`run wait` agree. A dead holder's hold is
ignored. `run status` shows "held by a test run".

Verified on an 11.12.1 app with a watched dev loop: with #2104's binary
a mixed attach run caused build #2, and a `test --local` run caused two
builds whose restart failed; with this change neither built, status was
clean, run wait returned at once, and an edit afterwards still built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiMsnXa9crge2qnSk9S7Zn
test --attach: route each test on its own; aggregates and association retrieves run in the app
…oflow by name (#2094)

Building a microflow resolved each entity-typed parameter and return type
by decoding every domain model in the project (once per lookup; nanoflows
and rules once per build), and create or modify microflow found the stored
flow by decoding every microflow. Both now read only what they need: the
domain model of the module a type names, lazily and once per build, and
the microflows of that local name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test: a test run holds the dev loop's watcher instead of making it rebuild and restart
…he real after-startup and keeps the dev loop's endpoint

Closes #2119.

The test run read the dev loop's MxTest.RegisterEndpoint as the project's
after-startup microflow and chained its own endpoint to it, to itself: the
scratch runtime failed with a StackOverflowError and no test ran. Its
cleanup then dropped the dev loop's endpoint while the setting still named
it, and the dev loop's next build failed with CE1613.

- A live host's restore record says what the endpoint replaced; the test
  run chains to that (projectState.hosted/chain, classifyHostedEndpoint).
- Its cleanup removes only its test microflows and keeps the dev loop's
  endpoint, Java source and setting.
- An abandoned endpoint is removed before capture, as `run` does; one with
  no record of what it replaced is refused instead of chained to.
- A test run that builds rewrites the deployment/ folder it shares with
  the dev loop. On release over an unchanged model the watcher now builds
  its own back once (holdResync) when the deployment moved, so a following
  --attach is not refused for a restart.

Verified on an 11.12.1 app with the issue's reproduction (app after-startup
Spike.ASU_App): main gave StackOverflowError and a failed dev-loop build;
with this change the tests ran, the endpoint and setting survived, the
dev loop rebuilt its deployment once, and --attach (in-app and mixed)
then ran; run stop restored Spike.ASU_App.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiMsnXa9crge2qnSk9S7Zn
…dpoint

test --local beside a --test-endpoint dev loop: chain to the real after-startup, keep the dev loop's endpoint
fix(mpr): keep the v1 Unit table scan and unit names between listings (#2094)
perf(executor): resolve a flow's entity types per module, find a microflow by name (#2094)
@ako
ako merged commit 0036eef into mendixlabs:main Nov 10, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants