Reproduce a missing notification and distinguish tests, MCP observations and agent actions. — ArkTelos Lab

Exact experiment revision

open8381011/experiments/agent-runtime-verification ↗650bb03c235852f0165ec68dbc97c0ab37d46cdf

EVIDENCE SIGNATURE

c0d3305119a73a06884d717417d3cacc1d17987cad9862923fb178c0ce2aaf8a2026-09-17

Environment

os: macOS
architecture: arm64
flutter: 3.44.7
dart: 3.12.2
dart_mcp_server: 1.1.1
codex_cli: 0.153.4

AI / agent parameters

provider: OpenAI via Codex CLI
model: gpt-6-astra /
repetitions: 3
variance: All three repair trials passed tests and made zero MCP calls.

Analyzer, seven tests per patch and independent cold MCP verification. One additional client probe; not a comparative benchmark.

Preparation

flutter pub get --enforce-lockfile

Execution

flutter analyze
flutter test

Inputs

https://gitlab.com/open8381011/experiments/agent-runtime-verification/-/blob/650bb03c235852f0165ec68dbc97c0ab37d46cdf/README.md ↗https://gitlab.com/open8381011/experiments/agent-runtime-verification/-/blob/650bb03c235852f0165ec68dbc97c0ab37d46cdf/lib/main.dart ↗https://gitlab.com/open8381011/experiments/agent-runtime-verification/-/blob/650bb03c235852f0165ec68dbc97c0ab37d46cdf/test/catalog_test.dart ↗

Expected outputs

https://gitlab.com/open8381011/experiments/agent-runtime-verification/-/blob/650bb03c235852f0165ec68dbc97c0ab37d46cdf/evidence/verification.md ↗https://gitlab.com/open8381011/experiments/agent-runtime-verification/-/blob/650bb03c235852f0165ec68dbc97c0ab37d46cdf/evidence/mcp-verified-budget-2026-09-17/summary.json ↗

Observed result

3763c7b2e2e7f97494b85bf8798d2698fd86ac5d9e400c79a2120f56df8f706d3×

Limitations

Synthetic defect, Flutter 3.44.7 / Dart 3.12.2 / dart_mcp_server 1.1.1, headless macOS. One client probe on a correct app and three repair trials; participants did not use MCP. Cold checks were investigator-driven. No MCP superiority, pixel correctness, Android/iOS or prompt-injection security claim.