Engineering Notes Engineering Notes

Test Harness or Flight Hardware: When Simulation Suffices

7 min read
Test Harness or Flight Hardware: When Simulation Suffices

Limited hardware affects every spacecraft program. The flight unit is costly, supports several verification tasks, and may be unavailable to software engineers when required. Deciding whether a test needs physical hardware therefore has practical consequences. A poor decision can block progress unnecessarily or leave hardware-dependent defects undiscovered until late.

The Useful Split That Matters

The practical split is not simply "simulation versus hardware." Ask which software properties rely on the physical hardware environment. Those properties need hardware for verification. Properties independent of physical characteristics can be checked in software.

Physical characteristics include bus timing, signal quality, interrupt latency, processor-specific instruction behavior, hardware-register timing, and electrical margins. Independent software properties include packet-field encoding, data-structure handling, state-machine logic for defined inputs, and numerical calculations. For ICD-based verification, the central issue is usually whether a packet encoder or decoder follows the ICD field definitions. That property does not depend on hardware.

Where Simulation Works Well

A software-only fixture on a development host can verify these flight-software behavior categories with high confidence:

Packet encoding correctness. For given field values, does the encoder create the ICD-compliant byte sequence? The ICD field table defines offsets, bit widths, byte order, and data types completely. Hardware is unnecessary.

Packet decoding correctness. Given a known byte sequence, does the decoder recover the right field values? This reverses encoding and remains hardware-independent.

State-machine coverage. Given input sequences, does the state machine reach the intended states and generate the intended outputs? If its logic is independent of hardware timing and interrupts, a software harness can exercise every reachable state.

Boundary and error handling. Test values beyond the ICD maximum, packets cut off mid-field, and incorrect packet version numbers. Simulation can verify each case because the results depend on code logic, not execution on the target processor.

Regression coverage across ICD revisions. After an ICD change and encoder update, simulation tests can confirm that earlier behavior remains correct and that added fields encode properly. If stubs come from the ICD, this remains inexpensive: rerun the generator instead of auditing test code manually.

What Requires Hardware

Hardware is needed when verification depends on the physical execution environment. Simulation is insufficient in these areas:

Bus timing and protocol compliance. CAN or MIL-STD-1553 completion within its time window, UART reception at the required baud rate in noise, and SpaceWire synchronization under load all depend on timing. They need the real transceiver, flight-speed target processor, and suitable test bench.

Hardware register access. On an embedded processor, a register write can configure a peripheral, trigger an interrupt, or change a GPIO line. A register-map simulator may not reproduce real timing or side effects faithfully. Behavior dependent on register access therefore needs the actual hardware.

Memory layout and alignment. Stack overflow, MPU behavior, and DMA cache-coherency faults depend on the target processor's memory map and protection features. Verify these on the target, unless using a highly faithful hardware simulator.

Interrupt-driven behavior. Correct ISR execution within its latency budget and protection of shared data between interrupt and task contexts depend on timing. Reproducing interrupt timing deterministically on a development host requires a real-time, hardware-equivalent simulator.

A Concrete Partitioning Example

Consider a CubeSat ADCS module that reads attitude data from a sensor bus, packs it into an ICD-defined TM housekeeping packet, and sends it to the TM downlink queue. Simulation can fully verify encoding: read a sensor structure, map values to fields, apply scaling, and write the bytes. A fixture supplies known synthetic data, runs the encoder, and compares the result with ICD-defined values. No sensor bus, ADCS hardware, or flight processor is needed.

Reading the sensor bus itself may need the sensor or a high-fidelity emulator. That includes SPI or I2C timing, bus-timeout handling, and interpretation of the sensor's native format. The division is clear: ICD encoding is simulation-verifiable, while the physical interface is not.

What Software-Only Tests Cannot Replace

Software-only tests cannot prove that generated code behaves correctly on the flight processor. A radiation-tolerant processor with a custom instruction set may compile differently from a desktop compiler using the same source. Integer overflow, floating-point rounding, and struct alignment vary by architecture and compiler. Host-passing, target-failing tests are a known defect class, discoverable only on the target.

They also cannot prove that the harness accurately models execution. If a fixture treats a function call as atomic but the real environment permits preemption inside it, host tests pass while the target contains a race. Finding such differences requires hardware execution. Software-only tests narrow the hardware verification scope; they do not remove the need to test on hardware before flight.