Mutation Testing

Mutation testing systematically injects faults (“mutations”) into source code to verify that the test suite detects them. If a test suite passes despite a fault being present, you've found a place where detection coverage can be strengthened.

Think of it as testing the tests. A comprehensive test suite should catch every realistic fault we can inject.

Publishing the engineering process—what we test, what we find, and what we improve—builds trust through evidence, not claims.

An injected fault is a controlled test of detection coverage. Its result does not, by itself, establish a defect in a released plugin or qualify a release. Published results must identify the version and scenario that were tested.

If you find a bug our tests missed, we genuinely want to know. That's how we improve the framework for every plugin in the catalog.