Mutation Testing

We intentionally inject bugs into our plugins to verify the test suite actually catches them. This isn't about making plugins fail—it's about proving our tests actually work.

What is Mutation Testing?

Mutation testing systematically injects faults (“mutations”) into source code to verify that the test suite detects them. If a test suite passes despite a fault being present, you've found a place where detection coverage can be strengthened.

Think of it as testing the tests. A comprehensive test suite should catch every realistic fault we can inject.

Campaign: BwsPressure v1.0.0

Date:: January 30, 2026
Target:: Pressure DSP pipeline
Mutations completed:: 2
Findings:: 2 identified, resolved
Campaign:: Ongoing — additional mutations scheduled

Findings & Resolutions

Finding #1: DC Offset Detection (Resolved)

Mutation:Injected a DC offset into the output buffer to test whether the framework's buffer validation would detect it.

Result: The stress test suite passed without flagging the offset. Buffer validation covered NaN, Inf, and clipping but did not yet include DC offset measurement.

Resolution:DC offset detection was added to the BPTS framework with a −60 dBFS threshold. All plugins were re-validated against the updated suite.

Resolvedin BPTS v2.2.0 · February 2026

Finding #2: Sample-Level Discontinuity Detection (In Progress)

Mutation: Injected a sharp gain discontinuity into the output buffer to test whether the framework would detect sample-level anomalies.

Result: The artifact and stress test suites passed without flagging the discontinuity. The framework checked for parameter-induced artifacts but did not yet include sample-by-sample gradient analysis.

Resolution:A discontinuity detector with a 6 dB inter-sample threshold is being integrated to extend coverage to transient-level anomalies across all test categories.

In progress· Targeted for BPTS v2.3.0

Mutation Matrix

DC Offset Injection

#: 1
Test Suite: Artifact Tests
Status: Resolved

Gain Spike

#: 2
Test Suite: Artifact Tests
Status: In Progress

Break Oversampling

#: 3
Test Suite: Frequency Tests
Status: Scheduled

Parameter Range

#: 4
Test Suite: Characterization
Status: Scheduled

APVTS Sync

#: 5
Test Suite: GUI Tests
Status: Scheduled

NaN Injection

#: 6
Test Suite: BOL Gate
Status: Scheduled

Latency Report

#: 7
Test Suite: Integration Tests
Status: Scheduled

Undo Manager

#: 8
Test Suite: Integration Tests
Status: Scheduled

Memory Leak

#: 9
Test Suite: Marathon Tests
Status: Scheduled

RT Violation

#: 10
Test Suite: Artifact Tests
Status: Scheduled

#	Mutation	Test Suite	Status
1	DC Offset Injection	Artifact Tests	Resolved
2	Gain Spike	Artifact Tests	In Progress
3	Break Oversampling	Frequency Tests	Scheduled
4	Parameter Range	Characterization	Scheduled
5	APVTS Sync	GUI Tests	Scheduled
6	NaN Injection	BOL Gate	Scheduled
7	Latency Report	Integration Tests	Scheduled
8	Undo Manager	Integration Tests	Scheduled
9	Memory Leak	Marathon Tests	Scheduled
10	RT Violation	Artifact Tests	Scheduled

Swipe sideways to compare

Why We Publish Framework Findings

Publishing the engineering process—what we test, what we find, and what we improve—builds trust through evidence, not claims.

These findings are about the test framework, not the plugins themselves. Every plugin passed its full test suite. Mutation testing strengthened the framework by revealing where detection coverage could be deeper.

If you find a bug our tests missed, we genuinely want to know. That's how we improve the framework for every plugin in the catalog.

Case Study: DC Offset Detection (January 2026)

Mutation testing revealed a detection gap in buffer validation—the framework didn't yet measure DC offset. We added the detection within 48 hours, re-validated all plugins, and updated the testing standard. This is why we publish methodology: finding issues and resolving them quickly is the process working as intended.