Test Embedded Systems

Normally each layer of a Product is tested against more and more detailed requirements and behaviors. But all in all, like shown above it’s always a new black box vs. interpreted Requirements, that are taken from above.

But this introduces risks, as these Layers may get lost in details and loose track of beeing focused on problem to be solved.
Or detailed requirements interpreted wrong.

All in all, just increasing requirements on left side and independently testing layers on right side will fail, and if it’s just because of the sheer amount of requirements that will quickly become non maintainable and not matching with the product.

Also solution side will become verry static and even the slightest change will need to adjust lots of Tests if testing is done from top to bottom.

Continue reading “Test Embedded Systems”

Specification development – Treasure hunt

As an example for requirement writing, i’d like to create an Application that allows us to do some treasure hunting. Furthermore I’d like to use the Gherkin BDD pattern to write the requriements.

Let’s start with basics, an abstract description of the app as free text and scenarios.

Feature: Treasure Hunt
The player gets some hints or facts to a sight and then gets the task to guess and move to the sight. If the player arrives at his guess, the position can be submitted and is validated if the guess is correct.

Scenario: Asking for new sight
  Given I as a Player
  When Asking for a new sight to guess
  Then I'm offered some facts about the sight to guess

Scenario: Submitting guess
  Given I as a Player
  And I'm given hints for a sight
  And I'm close to the correct sight
  When I submit my position
  Then I'm notified, that it's correct

Cool, we have our first requriements and we also managed to just describe the behavior without digging into solutions like which button to press or how we get and display the facts or even how we get the position.

But what fun would it make if the player always would directly know where to go? We probably need some case, where the player submits some wrong position.

Feature: Treasure Hunt
The player gets some hints or facts to a sight and then gets the task to guess and move to the sight. If the player arrives at his guess, the position can be submitted and is validated if the guess is correct.

@basic
Scenario: Asking for new sight
  Given I as a Player
  When Asking for a new sight to guess
  Then I'm offered some facts about the sight to guess

@basic
Scenario: Submitting guess
  Given I as a Player
  And I'm given hints for a sight
  And I'm close to the correct sight
  When I submit my position
  Then I'm notified, that it's correct

@alternative
Scenario: Submitting wrong guess
  Given I as a Player
  And I'm given hints for a sight
  And I'm not close to the correct sight
  When I submit my position
  Then I'm notified, that it's incorrect

As you see, I already added tags “basic” and “alternative” to the scenarios. That way we can directly see if the scenario is in basic flow or handles some deviation.

Activity diagram

This further detailing can be continued with scenarios for increasing player points, giving up and selecting a new sight or allowing only some amount of wrong guesses.

Also what about adding new sights and facts about them? As we probably don’t want to fiddle with the database directly, we probably need some administrator tool to add and modify.

Feature: Sight maintaining

@basic @admin
Scenario: Add Sights
  Given I as Administrator
  And admin tools
  When sight 'Garden' is not existent
  Then I can add new sight 'Garden'

@basic @admin
Scenario: Add Facts to Sights
  Given I as Administrator
  And admin tools
  And Sights are
    | Name |
    | Berlin Gate |
    | Church |
  When editing sight 'Garden'
  Then I can add new facts to 'Garden'

Specification by text

After having done a brainstorming on use cases and activities, it’s time to formulate these in textual form.

Shall, should, can

For requirement engineering often the pattern “if …, the … shall/should/can do …” is used.

  • If logged in, the user shall be able to interact with the software
  • If not logged in, the user shall be redirected to the login page
  • If user does not have an account, the user can create a new account

This already breaks down a otherwise big block of text into smaller patternized atomic actions and features. Further allowing grouping some of them and mapping them to components in following steps of design.

Gherkin

An alternative is to use Gherkin, which is a specific language for behavior driven design (BDD), designed by Cucumber.

In Gherkin single requirements are “Scenario’s” made of steps written in a specific form.

  • Given – What is the (static) precondition
  • When – Define a Trigger
  • Then – Observable resulting behavior

This pattern pretty much matches the brainstorming we started with.

The benefit, that let’s hearts of testers jump higher is, it’s an standardized format that already has multiple implementations for Testframeworks. So a tester can use the requirements directly as input for tests and can focus on implementing the necessary steps.

Specification of Idea – Usecase and Activity

The first thing to do if starting with a new thing, is to specify what it should do.

Even if your fingertips already want to write Code and you want to discover the newest and latest Features of a new framework.

This will help to stay focused on the solution and also helps to find issues or gaps early. It’s driven by the question of how the “thing” you want to create interacts with others, like users or other tools.

Specification

Specification can be done in multiple ways. Either as a long block of text, or atomic features, or visualy by using use case or activity diagrams.

In general it helps to start with a short non formal description for first focus and have a base to start. Followed by standing up from the desk, shaking the body and going to a whiteboard for brainstorming.

With whom will my thingy interact, what benefit does it give, what will be the features it provides.

After roughly defining the features and functions we need to provide, the question is on workflows. What might be the chain of actions leading to something that gives a benefit to our user?

Pitfalls

Overdoing usecases, will render use cases useless, as it’s level of abstraction may jump from verry detailed to abstract or triing to visualize to many (non related) cases at once, making the graph more confusing then helping.

The size and count of objects in the graph should be limited to roughly 7-10 elements. This can be done to either stick with one user, one specific area of use cases or a level of abstraction.