Normally each layer of a Product is tested against more and more detailed requirements and behaviors. But all in all, like shown above it’s always a new black box vs. interpreted Requirements, that are taken from above.
But this introduces risks, as these Layers may get lost in details and loose track of beeing focused on problem to be solved.
Or detailed requirements interpreted wrong.
All in all, just increasing requirements on left side and independently testing layers on right side will fail, and if it’s just because of the sheer amount of requirements that will quickly become non maintainable and not matching with the product.
Also solution side will become verry static and even the slightest change will need to adjust lots of Tests if testing is done from top to bottom.
So simple rules would be
- Test as integration as possible
- Test against use cases where ever possible
- Reuse requirements instead of interpreting
Meaning, how nice would it be, if change of a requirement would immediatly take effect on testruns and if it fails an issue is created?
Instead of need to create a change request, that is then added to the set of requirements, added into architecture and then handled by testers and developers.
Segregate and use Interfaces
To allow testing of layers always use interfaces and never use specifics. Even if this means the need to write translators or adapters.
The reason is, this allows easier testing and encapsulation of components. Furthermore the risk of beeing dependant of a specific library or operating system is reduced. Changing parts of the solution later will be way easier.
Even if breaking all these concerns down looks verry scarry.
https://gitlab.com/dominik.gausa/cmake_cpptest
In this structure everything that is 3rd party is colored orange, company code is yellow and resulting binaries or projects are green.
Why does this complexity help us?
By defining a absolutely independant “platform” we can define an highly abstract fabric and interconnectivity, that is then used by solution components to interface.
Tests of this platform can be done by any test framework, and should be written so, that they can be used as examples of using the platform and get an idea of how components are thought to be created and used.
Applications and business logic then build on this platform and inherits it’s idea of interfaces and patterns. As you see, even this buisiness logic so far does not care about what platform it’ll run on. It will just use the interfaces defined by platform.
The tests will then be focussing on blackbox use cases and giving examples of how the components are intended to be used.
Finally the project will sum platform interfaces, applications and buisines logic and the implementation of external interfaces to resulting firmware.