It's no secret that automated tests have
many benefits and after many years of discussions and one stale git branch, it's time to finally dive in.
There are many different kinds of automated tests. We will limit ourselves to Unit Tests and Integration Tests for now. The following should give a brief overview of what kinds test we expect and how to approach testing.
Unit TestsUnit tests should only test one specific unit independently from any other unit as much as possible. This means that we're not testing what effect one unit has on another unit, but simply check the functionality of that one unit.
For SFML a unit is most of the time simply a class. For example
sf::Rect<T> is a unit,
sf::Vector2<T> is a unit, but the unit tests for
sf::Rect<T> shouldn't check that the properties of
sf::Vector2<T> are set correctly, because the
sf::Rect<T> unit tests, should only test the
sf::Rect<T> interface.
The point is, if every class has its own unit tests, we can be certain that every class behaves exactly the way we expect them to, so they don't need to be retested.
The foundation for unit tests is setup in SFML 2.6.0, but massively expanded in SFML 3.
Integration TestsIntegration tests on the other hand do intentionally integrate multiple units to ensure that in combination they behave as expected.
For SFML this can often mean, that we let one unit pass through other units and finally transform the result into a format that can be asserted. For example the rendering of a shape would go through a render texture and then convert to an image, which can be compared pixel by pixel with a test data image.
The proper setup and intent of integration tests still needs to be determined, but some things will certainly build on top of the unit test setup.
What do we test?Tests exist to assert that the promises we make by providing a public API, actually hold true.
As such unit as well as integration tests should only ever test the public interface and ensure that the API does what the documentation says.
When do we write tests?For every new feature there should be multiple new unit tests, testing positive as well as negative test cases.
For every bugfix there should be at least one new integration test, that covers the bug and ensures that there's no regression.