In this talk I am going to explore two questions:
How do we know if a check is valuable
How do we identify and implement valuable checks
So let’s start by working out whether a check is valuable or not. To demonstrate this we’re going to look at an End-to-end, full stack, UI driven, whatever you call it, check. So let’s take a look at our check in action…
To demonstrate how we can use Trims to analyse a check, I’ve created an automated check that follows a typical pattern. It’s designed to check that a user can log into an application so let’s see it in action
Demo and run End-to-End check
So let’s analyse that E2E check
Is it targeted? Not really. The check is doing everything on the UI layer, meaning if the intention is to check behavior in the backend it’s using the furthest layer from that implementations
Is it reliable? Perhaps. This will be heavily influenced by the state of the application under test BUT this check is working against a complex system with many moving parts of which any one could fail on us. Although that maybe desirable.
Is it informative? Let’s say something in the backend breaks causing our check to fail because it cannot find a specific element. That information is shallow at best and misleading at worst, meaning that it’s going to take considerable time to debug and react to. A risk can have many ways to impact a user
Is it maintainable? Depends on how it was developed, if we are to assume that this check uses page object models, data builder patterns and DRY practices then we could say we’re happy that it’s maintanable
Is it Speedy? Remember speed isn’t just time to run, but time to react to as well. Given that it’s not targeted and not very informative, then it’s fair to say that the feedback loop is going to be slow as we wade through the application and test code to work out what is exactly going on.
We cannot rely on the scope of information from a check, but we can rely on checks if they are:
Targeted
Reliable
Informative
Maintainable
Or have Speed
Speed is not just how fast it runs, but how quickly we can process information and act upon it
Lots of small checks that follow those valuable attributes will help us build a picture differently to how we test as humans but hopefully bring us to the same conclusion
So it’s fair to say that that check isn’t up to muster. One of the main issues is that it’s trying to check multiple impacts
What is task analysis?
AiT is influenced by Rob Sabourin’s work on Task Analysis
Breaking down a task into it’s component parts for analysis
Cup of tea example
Let’s see an example
Show task analysis model and highlight
Left hand side requires some knowledge of the system, but it can vary based on your knowledge
Your first iteration could literally be frontend – backend
We use task analysis to break down the different events that make up feature that our original risk is focused on
So we have points on our model that represent each activity the system does
Each action is an opportunity for a targeted check on that section
And each action has it’s own inherent risks that may matter to us
Finally the lines that cross boundaries between systems are integration risks to check for
So it’s fair to say that that check isn’t up to muster. One of the main issues is that it’s trying to check multiple impacts
This flow successfully captures the areas that could impact our application and realise the risk that a user can’t login.
We can use that knowledge to identify targeted checks
This flow successfully captures the areas that could impact our application and realise the risk that a user can’t login.
We can use that knowledge to identify targeted checks
This flow successfully captures the areas that could impact our application and realise the risk that a user can’t login.
We can use that knowledge to identify targeted checks
This flow successfully captures the areas that could impact our application and realise the risk that a user can’t login.
We can use that knowledge to identify targeted checks
So it’s fair to say that that check isn’t up to muster. One of the main issues is that it’s trying to check multiple impacts
In summary
Review checks and ask yourself are they TRIMS
Break down unwieldy checks by applying task analysis to the system flow that is being covered
Use the results of task analysis to determine checks that are more TRIMS
Once you have checks in many different points build a ‘chain of trust’ by running them in a pipeline
Task analysis can help identify dependencies for mocking
