44. “ Feature A tested, there are some interaction issues with feature B and, by implication, risks for features C and D. We have not explicitly tested feature C. Recommend some further investigation with feature A in combination with features B and C. As feature D is not to be used in trial a priority decision is needed on the inclusion of D into scope. If feature D is to be put into general usage then further test and investigation is recommended.”
45. Pilot activity spread to end-2-end teams with greater engagement and end-value appreciation. Issues in the product for end-user usage & interactions are found -> adding value to the product reporting
These experiences are based on bullets 2&3, and through bullet 1 spreading the word to other teams.
Marionette: Following instructions of the puppeteer But suppose there is some danger in front of him that the puppeteer doesn't see? He can decide to divert or modify his action, and even when to continue following directions. So, it's not blindly following instructions...
Using some pre-defined set-up as an enabler to an exploratory session, i.e. there can be a certain amount of pre-meditated design and thought before you start exploring The pre-defined set-up and exploration should not exclude observation and feedback. Walking with someone else is valuable (just like testing with someone else adds a new/different dimension)
Note, you need a way to determine if/when to carry on Script usage could also be called script-supported testing
Large Organisation Some features are for trial purposes Many specifications & protocols Many features and interactions Perpetual question: “How much detail is enough?” Prototype in a conservative test team set-up But, the new thinking has followed into end-2- end team set-ups
“ Erfinden sie ein problem” = “they invent a problem”
Say which bits will be zoomed-in....
Trial, demo & live? A trial might have different performance requirements to live usage. Other apsects not required might be expansion/scaleability needs or robustness and negative testing.
This is a 'weeding-out' activity also. Input to 'silent evidence' here is: not possible due to tool or time restraints.
Some traditional aspects kept Some new aspects/ideas introduced
Note, you can't just make things ambiguous and then thinking and reflection happens. The thinking and questioning must be encouraged as a part of the activity. The ambiguity stops ideas getting 'boxed-in'.
Don’t let test cases ”constrain” (box-in) ideas – create them with ”enough” ambiguity so that you’re not constrained, boxed-in, no restricted to a comfort zone and eventually ”thinking outside your comfort zone/box”
Testing has been framed as an investigative activity Try this, what happens?, grey areas? Look closer Contrast with, “this is the spec -> test to that” Moves away from reliance on only confirmatory testing
By changing the frames of reference the labelling of the activity was avoided. Sometimes labels get in the way. Framing a problem differently is much more subtle -> acceptable.
I think of test cases as test idea constraints – because they tend to lock-in ideas about what a test case produces -> it takes on much more value than it should... Getting past test case counting Relating value in test ideas to end-user value Emphasized with PO/team involvement early Working with good test story telling is important Whole team involved in initial analysis Reduces handovers + creates a feeling of involvement Prototype early – good toolsmiths are important Fail fast – throw away ideas before they cost too much!
The thinking and reflection about the test ideas, feeding back to improve them -> these are attributes of what I call good testing and attributes that align with many aspects of good agile testing. BIg challebge – abscence of team set-up, wow & processes----- (was also an aid to not have defined WoW & processes!) Reflect on semi-script & exploratory....