Testing & Scenarios
Test specifications for your models: Testing Mode, portable test scenarios with assertions, MIL/SIL/PIL execution targets, block-level coverage, and component tests.
Simulation tells you what a model does; testing tells you whether that is what it should do. Modeloop turns verification into a first-class part of the modeling workflow: you write test scenarios with explicit pass/fail criteria, run them against the Python reference or the real generated C, and read the results — including code coverage — directly on the diagram.
Testing Mode
Testing lives in a dedicated mode, toggled from the editor:
- Edit Mode — the model is editable; the side panel shows block properties.
- Testing Mode — the model is frozen (any attempted edit is blocked with visual feedback), test artifacts are prepared once, and the side panel becomes the testing console.
Freezing is the point: every test in a session runs against exactly the same model and the same compiled artifacts. You can run as many tests as you want without recompiling, and no accidental edit can silently invalidate results halfway through a campaign. Toggling back to Edit Mode unfreezes the model with nothing lost.
Test scenarios
A test scenario is a complete, self-contained test specification:
- Duration — how long the scenario runs.
- Inputs — a scenario profile (
constant,sine,ramp,step,table) for each model input. - Assertions — the acceptance criteria evaluated against the outputs.
Scenarios are defined once and portable: the same scenario runs unchanged against every execution target below. There is no separate “unit test dialect” per target — one specification, several levels of realism.
Assertions
Assertions turn a simulation into a test. Each one targets an output signal and carries a tolerance:
| Assertion | Verifies | Fails when |
|---|---|---|
| At time | Output equals the expected value (± tolerance) at a specific time | The value at that instant is outside tolerance |
| Always | A condition (e.g. output ≤ limit) holds for the entire duration | Any timestep violates it — the result reports the first violation time |
| Eventually | The output reaches the expected value (± tolerance) within a timeout | The timeout expires without reaching it |
| Final | The output equals the expected value (± tolerance) at the end of the run | The final value is outside tolerance |
The four cover the standard control-engineering questions: value at an instant (at time), safety invariants (always), settling and convergence (eventually), steady-state (final).
Every assertion produces PASS or FAIL with the measured value, so a failing test tells you what diverged, when, and by how much — not just that something went wrong.
Execution targets
The same scenario can execute at increasing levels of realism:
| Target | What executes | Where |
|---|---|---|
| MIL (Model-in-the-Loop) | The Python reference implementation of the model | Everywhere — desktop and browser |
| SIL (Software-in-the-Loop) | The actual generated C code, compiled and executed | Desktop; in the browser as Cloud SIL (Professional plan), prepared and executed server-side |
| PIL (Processor-in-the-Loop) | The generated C, cross-compiled and executed on an emulated embedded processor | Desktop |
The intended flow:
- MIL while designing — fast iteration, no compiler needed.
- SIL as the primary evidence — your deliverable is the C code, so the tests that count run against it.
- PIL when target realism matters — same tests, executed on an emulated board, catching target-specific numeric and toolchain effects.
Because all targets consume the same scenario, a MIL-vs-SIL discrepancy is itself a finding: same model, same inputs, different result means something is worth investigating — before it ships.
Coverage on the diagram
When tests run against generated C, Modeloop measures code coverage and maps it back onto the model: each block is colored by how much of its generated code your tests exercised —
- green — fully covered,
- yellow — partially covered,
- red — never executed.
Coverage answers the question test lists cannot: what haven’t I tested? A red If-Then-Else branch or an untouched state means your scenario suite never drove the model there — add the scenario that does.
Component tests
System-level scenarios are not the right tool for unit-testing one algorithm. Component tests let you test a single container in isolation, treating it as a pure function:
- You provide inputs (scalars or tensors) and expected outputs with per-output tolerances.
- Modeloop executes just that component and compares.
- Test cases live with the model and are managed in the Specs panel.
Component test specifications are validated fail-closed: a test case whose input or output names don’t match the component’s actual ports is rejected, and a case that asserts nothing on a known output is not silently counted as a pass. A green component test always means something was actually checked.
Runtime errors during tests
If a test drives the model into a runtime fault — overflow, NaN, division by zero — the run doesn’t dump a stack trace: Modeloop reports the error with the block responsible, the signal, and the time. A test that fails by fault is as diagnosable as one that fails by assertion.
For proving such faults cannot happen on any input — not just the ones your scenarios tried — see Formal Verification.