A 6U CubeSat team had thorough unit tests for its OBSW command handlers, covering all 47 TC types. The suite completed in under 10 seconds on any laptop. Its first MIL-STD-1553 bus compatibility test exposed a flaw: the stubs accepted bus time slots that the hardware rejected. The tests checked the code, not the protocol.
Software-only flight-software regression testing has a central tension. A software fixture can confirm that code behaves as intended, but not always that the intended behavior matches hardware expectations. Identifying which tests belong on each side sets the regression coverage possible before hardware arrives.
What Regression Tests Really Cover
Regression testing has two separate jobs. One is finding unintended changes: a code edit may alter an established TC handler, telemetry-field encoding, or a previously working state-machine transition. Software can detect nearly all such regressions because they depart from behavior already confirmed.
Its other job is protocol-conformance validation: confirming that a real receiver accepts TC frames, that the ground system decodes TM frames properly, and that timing limits are met. Hardware is required here. That is the real divide between sufficient software-only testing and tests that need hardware.
Many small spacecraft teams merge these purposes and assume useful regression testing needs hardware. It does not. Behavioral-fidelity tests can run entirely in software. Conformance tests need hardware in the loop, but not every sprint or every time code changes.
Spacebackend's Software-Only Test Coverage
Generating test stubs from an ICD produces three parts. First are packet constructors, Python or C functions that assemble valid TC frames for each command type, including primary-header fields, APID, sequence flags, and data layout taken from the ICD schema. Second are field-level assertion helpers that check expected values in TM packets. Third are boundary fixtures containing minimum, maximum, and one-beyond-maximum values for every numeric parameter.
Together, these parts cover software-layer protocol correctness. A TC handler reading a parameter from the wrong byte position causes a field assertion to fail. A broken sequence-counter increment fails its generated check. A response packet exceeding its declared field length is caught by the boundary fixture.
Generated stubs do not simulate bus timing. MIL-STD-1553 includes round-trip timing windows, status-word replies, and retries that call for real hardware or a hardware-accurate simulator. The stubs cover message content, not electrical behavior or timing.
Building a Useful Software-Only Test Environment
A typical software-only OBSW test setup includes flight software on a cross-compiled or native simulation target, an ICD-generated TC injector, TM capture and validation, and a scripted state-transition scenario. The scenario can come directly from the mission's operational concept document.
Built from ICD data, the TC injector sends valid command sequences to the software under test and verifies the expected TM responses. In this setup, an OBSW state machine moving from SAFE to NOMINAL after a specified TC sequence can be tested entirely in software. Sequence encoding, transition logic, and response contents are all checkable without hardware.
The practical advantage is CI execution. Each commit touching TC handler code can launch the applicable scenario tests. Teams with this setup find behavioral regressions during development instead of at a hardware integration gate weeks later.
Limits of Software-Only Testing
This approach has clear limits. Timing-sensitive behavior, especially FDIR watchdog timing and interrupt latency, needs hardware for accurate measurement. Physical interface conformance, including electrical levels, bus termination, and physical-layer framing, requires the actual interface hardware. Radiation effects cannot be represented in software simulation.
Some emergent system behavior appears only after the complete flight system is assembled. OBSW task contention for shared memory, timing interplay between command handling and telemetry scheduling, and peak-load resource exhaustion may escape software-only tests because simulation does not reproduce hardware scheduling constraints.
Software-only regression testing still has a valid role. The key is assigning each test to the right layer. Effective V&V programs commonly use three groups: software-only, running in CI on every commit; software-with-emulator, run before each integration milestone; and hardware-in-the-loop, run at formal verification gates. ICD-generated tests fit the first group, which merits rigorous maintenance even when the other groups remain sparse.
Practical Coverage Goals
Early in a mission, while hardware availability remains uncertain, a reasonable software-only target is one valid-input, one invalid-APID, and one out-of-range-parameter test for every TC type. Each TM packet from a TC handler should have field assertions for its primary header and at least one data field. Every nominal state transition should also have a corresponding TC sequence test.
ICD data alone can support this before hardware exists. It is not complete V&V, but it lets hardware integration focus on conformance and timing, where hardware is essential, instead of finding packet-encoding defects that software should have caught weeks earlier.
