Illusion of control TestBash Netherlands
•Als PPTX, PDF herunterladen•
0 gefällt mir•176 views
Slides from my first time speaking at a conference - TestBash Netherlands in May 2019
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (13)
Ähnlich wie Illusion of control TestBash Netherlands
Ähnlich wie Illusion of control TestBash Netherlands (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Illusion of control TestBash Netherlands
- 1. The Illusion of Control Drew Pontikis Twitter - @drew060609 Blog - https://medium.com/@drew060609
- 5. “While people may pay lip service to the concept of chance, they behave as though chance events are subject to control” Ellen J Langer
- 6. The Illusion of Control – Ellen J Langer (1975) Journal of Personality and Social Psychology Experiment 1 – Effects of competition Experiment 2 – Effects of choice Experiment 3 – Effects of stimulus familiarity Experiment 4 – Effects of response familiarity Experiment 5 – Effects of Passive involvement (field) Experiment 6 – Effects of Passive involvement (controlled) Paper can be found here - https://pdfs.semanticscholar.org/136e/9cf6b5a4d17dbe8400fa5d7f4bf3ad01f6ac.pdf
- 7. Effects of Choice What happens if you’re given a choice in the action taken, regardless of whether this choice can impact the outcome?
- 8. Football Card Lottery Here you go, choose a card Here’s your card, talk to you Monday!
- 9. Will you sell your card, and how much do you want for it?
- 10. Football Card Lottery Taken from The Illusion of Control – Ellen J Langer Yale university Here you go, choose a card Here’s your card, talk to you Monday!
- 11. Effects of Stimulus Familiarity What happens if the subject matter is familiar, regardless of whether it is relevant to the outcome?
- 12. Here you go, choose a card Here’s your card, talk to you Monday!
- 13. Do you want to swap your card to join a lottery where you have a better chance of winning?
- 14. Here you go, choose a card Here’s your card, talk to you Monday!
- 15. What Have we Learned? If you let me choose, I will choose something that I feel familiar with and I will not want to give it up
- 16. We can go live with this bug if you want to, or we could delay and investigate it further I mean, they wouldn’t be ASKING if it wasn’t ok… We can go live with this bug if you want to, or we could delay and investigate it further We can go live with this bug if you want to, or we could delay and investigate it further Yes - I really want that!
- 17. Things we believe as Testers This feature is going to be a success, that’s why I chose it! I’m attached to this feature because I chose it, I may not see its flaws This version is much better Is v48 REALLY good enough or is just better than v47? I’ve tested this many times before and its never broken - this is low risk The risk is only managed if your regression testing is effective
- 18. Things others believe on our behalf We are experts in our field, lets stick with what we know We’re so confident in our product, we’re not going to diversify I know you’ve been testing, but I’m going to need a test plan I want to see something that is familiar to me
- 19. Be careful what you wish for…
- 20. Effects of Passive Involvement What happens if you have increased involvement with a chance interaction?
- 21. How about now? How do you rate your chances of winning? And how about now?
- 22. Taken from The Illusion of Control – Ellen J Langer Yale university
- 23. Taken from The Illusion of Control – Ellen J Langer Yale university 619 428 High Involvement group Low Involvement group
- 24. What Have we Learned? The longer I spend thinking about something, the more confident I am in getting a positive outcome
- 26. Real Life Applications… Cone of uncertainty – on long slow burning projects or features we grow in confidence as we progress along a curve…so we may not listen to a fresh set of eyes that sees something new
- 27. Effects of Response Familiarity What happens if you practice, or have experience with a scenario, regardless of whether the outcome is influenced by the interaction?
- 28. High Involvement – I’ll show you how to do it Low Involvement – here’s some instructions High Familiarity – have a quick go whilst the machine is turned off Low Familiarity – let’s get started
- 29. What Have we Learned? If I FEEL like I’m affecting the outcome, I will be confident in my ability to get it right
- 30. Things we believe as Testers If I feel like I understand the feature and I’m seeing the results I expect, I will feel confident in it Are my tests ACTUALLY making any difference? This has all gone so well – our customers are going to love this… I am implying a correlation between the development process and the success of the product Our focus group wasn’t all that positive, but we’ve done this type of feature before – they’ll love it once it goes live I am ignoring evidence contrary to what I want to believe
- 31. What CAN I Control?
- 35. “no matter what you do, that seed will grow to be a peach tree. You may wish for an apple or an orange, but you will get a peach” – Oogway
- 36. How to spot a peach tree Don’t be fooled by being given a choice – be careful what you wish for… Challenge your own thinking – think about WHY you are choosing something Listen to new voices - it’s never too late to be wrong Active Involvement – make sure you’re actually adding value
- 37. BedanktThanks More from me… Twitter - @drew060609 Blog - https://medium.com/@drew060609 MoT Peterborough - https://www.meetup.com/Ministry-of-Testing-Peterborough/ MoT Cambridge - https://www.meetup.com/Ministry-of-Testing-Cambridge/
- 38. References • Peterborough - https://www.thesun.co.uk/news/8483147/peterborough-named-worst-place-live-england/ • Coin flip - https://www.pbs.org/newshour/economy/cant-decide-just-flip-coin • Four leaf clover - https://biggsuccess.com/2013/03/15/how-to-get-lucky-with-money/ • Horseshoe - https://www.nauticalplace.com/Solid_Brass_Good_Luck_Horse_Shoe_p/brhsh.htm • Lucky rabbits foot (assorted colours) - https://www.bewild.com/products/lucky-rabbits-foot-keychains-set-of-three-assorted • Lucky pants - https://www.amazon.co.uk/Numberpants-com-Lucky-Pants-Underwear-Men/dp/B0160QNAHC • Lucky number 7 - http://coolfidence.com/lucky-number-7-icebreaker-3066#1 • Lucky red envelope - https://www.ebay.com.au/itm/25-50-RED-PACKET-Red-Envelope-Chinese-New-Year-Lucky-Money-Hongbao-97-66mm-/282725971600 • Illusion of control in Kung Fu Panda - https://sermons.faithlife.com/sermons/82821-movie-%22kung-fu-panda%22-reveals-illusion-of-control • Kung Fu Panda - http://ilovepapers.com/an47-kungfu-panda-dreamworks-animation-art-cute/ • Oogway the weird tortoise thing - http://animalia-life.club/other/kung-fu-panda-oogway-quotes.html • Shifu - https://dragonriderone.wordpress.com/2013/09/14/my-aikido-journey-part-3-my-instructors/picture-40/ • Lottery ticket - https://www.confetti.co.uk/shop/product/lottery-ticket-holders-10-pack • Neil Younger’s massive brain - http://norry.co.uk/blog/ • Appraisal Sign - http://mastershallappraisal.com/wp-content/uploads/2015/09/Appraisal-Blog-Photo.jpg • Cone of uncertainty - https://what-buddha-said.net/gallery/index.php/Dhamma-illustrations/cone_of_uncertainty • Tree - https://pixers.ca/posters/black-tree-silhouette-isolated-on-white-background-41374323 • Simpsons monkeys paw - https://mandarincompanion.com/blog/the-monkeys-paw-how-gruesome-should-it-be/
Hinweis der Redaktion
- Now this is my first time speaking at a conference, and I’ve been given the slot bang in the middle of the sleepy hour. That means it’s my job to wake you all up again, so we’re going to do an experiment…
- Someone in this room is about to do something incredible Something many years from now they will tell their grandchildren about Do the thing Congratulations, you defied the odds and correctly guessed a coin flip x times in a row – that’s the kind of super power that can change lives, I hope you’re going to buy a lottery ticket tonight! I’m sure you had an algorithm or technique that you were using to guess these correctly, however I’m sorry to have to say this but unless you’re a closet Jedi, nothing you were doing was having any impact on whether this coin landed heads or tails. Its all random chance, and the fact that it was you and not this guy here is nothing but luck Now luck is fascinating it exists – of course it does – but only retrospectively. Something can happen and be a lucky outcome, but it doesn’t happen BECAUSE of luck. This is an important distinction, and one which in reality we’re all aware of...
- But we don’t act like it We look for life’s cheat codes to increase our luck. We have a lucky number. Lucky pants. Lucky rabbits foot in an assortment of colours Now if we thought about it sensibly, we know none of these things can actually have any impact on how lucky we are – especially this abomination here But even so, consciously or unconsciously, we let it happen…
- “While people may pay lip service to the concept of chance, they behave as though chance events are subject to control” Why do we do this? It’s because it’s part of the human condition to crave a sense of control, and to try and avoid situations where we are not in control. Not being in control can cause feelings of nervousness, hopelessness and stress – have you ever got on the wrong train for example? Because of this, our brains try to trick us into feeling in control, and create a fog of illusion to make us feel like we can control things that we know we can’t And it’s this illusion that…
- in 1975, was the subject of a paper written in the Journal of Personality and Social Psychology by Professor Ellen J Langer. The paper itself can be found online and I would encourage you to have a read of it as the half an hour I’m going to spend talking about it today really doesn’t do it justice, but it centres around a series of experiments designed to isolate and identify the ways in which this illusion can affect us. When I read this, the tester in me thought – I recognise that – so that’s what I’m going to talk about today and I hope you will too. So the first experiment that I’m going to discuss today…
- Illusion of choice
- Illusion of choice
- Now remember, we want a low number here – sensible people would say yes please, please take my card and swap it for one with a better chance of winning. Remember, we want a low number….
- So having looked at this, what can we learn from these tests?
- Well, we have learned that if you let me choose, I will choose something that I feel familiar with and having chosen it, I will be naturally inclined to stick with it and not want to give that choice up. This may sound like its an over-simplification, but bear in mind there’s an awful lot going on subconsciously even when presented with a simple choice. Let’s take this one as an example…
- This isn’t a terribly uncommon question to come up with any delivery team, bugs happen all the time. It also isn’t, on the face of it, an unreasonable question – stuff goes live with known bugs present in it all the time, and not every bug is a showstopper – that’s why we have the word showstopper. But lets look at whats going on here that isn’t immediately obvious… Even now, with the way I’ve framed this to you right now, you’re subconscious will have decided that the fact I am ASKING you will give you a sense of confidence in the severity of the bug. I wouldn’t be asking you to make a choice if it wasn’t ok to say yes would I? But you don’t know that – I might hate you and want to see you get in trouble. I might know that this feature is an absolute dogs dinner but I’m not brave enough to speak up, so I’m hoping someone else will. I might be hung over and I’m doing anything to avoid having to use my brain right now. You don’t know any of that, all your brain knows is that I wouldn’t be asking if both outcomes weren’t valid. Also, we have to factor in how we FEEL about the outcomes. We do our jobs because we want to deliver cool stuff that our customers will love, and we also know we would maybe have to have the difficult conversation with our bosses as to why we had the chance to put this live but we CHOSE not to. And that’s the important fragment from this question – do you want to put this live? Do you want to see the culmination of all our hard work and put this live? Yes – of course I do! I want to go to my colleagues, my bosses, my long suffering wife at home and say ‘look at this thing that we did!’ I want that! It takes a lot for your conscious mind to overlook all that subconscious noise…
- …and it throws up a fog of illusion that can really affect our judgement if we aren’t mindful to it. I thought about my life as a tester to think about ways in which I’ve been caught out before because of this, and how you might recognise it yourself. Now you and I both know that we aren’t the only ones making decisions in a delivery team, and often we’re the ones asking the question…
- So what can we do about this? How can we protect ourselves here? I’ve found that there is a heuristic that you can use, either when you’re making a choice or watching someone else make a choice, that can be used to help break the illusion. I call it the monkey’s paw heuristic after the W Jacobs story of a bewitched monkeys paw…. Challenge a choice to find out if its valid Heuristic – the monkeys paw When you challenge a choice, it breaks the illusion of control and makes us think bigger
- But it might be easier remembered as ‘be careful what you wish for’. When presented with a choice that you’re confident in, think about the malicious interpretation of that Stick with what we know = granted, we are undercut by a lean startup who can do our thing better and faster, and we go out of business because it’s too late to diversify I need to see a test plan = granted, our testers only look for things they can predict, and uncountable numbers of bugs go undiscovered Most of the time, you’ll go ahead and make decision anyway, and you’ll be confident and you’ll be right. But every now and then you’ll find one where a change in your thought process blows away the illusion, and you see the risk that you were about to accept in a slightly different light. Right – next experiment…
- What we’re inferring here is that the only variable is the amount of time the people involved spent thinking about their chances of winning, and that the longer they have to think about it the more their confidence increases. This was done out in the wild and wasn’t really under controlled conditions – they might have been on a lucky streak on the horses (lucky streak), or they might have had a few beers in them and maybe were prepared to gamble a bit more than they would have normally. So they tried to do it under slightly more controlled conditions…
- Once again, its another lottery…. Again, they were asked would they want to swap their card for one in another lottery where there was a greater chance of winning Based on these results and those from the previous experiment we can infer that those in the high involvement group, those who have invested more time and energy thinking about their involvement in what is still and has always been clearly sold as an act of random chance, have developed a greater degree of confidence in their chance of success than those with low involvement. In short…
- The longer I spend thinking about something, the more confident I am in getting a positive outcome. Now this – this actually makes sense, and when we think about some of the agile principles that we know and we’re familiar with, we can actually see how that if we’re not careful we can allow risk to creep into our decision making. Lets for example take the cone of uncertainty…
- Here is a very quick overview of what this principle means – at the start of a project, there are many unknowns, so we are sceptical and doubtful and try not to make promises about it. As we progress, those unknowns become known, and then become understood, and our confidence progressively grows. All good. What happens though, if we make a mistake up front? It happens. It happens all the time in fact. And what if we don’t spot it and we continue up until around here, where we get someone else involved. Maybe another tester. Another tester who spots this as a bug. Brilliant! However, the rest of the project have already gone passed the confidence stage and have reached the conviction stage, and have become attached to their work. They can become defensive about their product, or their feature. You don’t know it like we do It’s supposed to work like that You just need to spend more time with it to understand why this isn’t a problem We can actually find ourselves in a position where…
- …we are actually defending the presence of bugs from someone who is there to help We all know the value in a fresh set of eyes looking at something, we should all welcome it, but we need to be so careful that when we get their feedback – that we listen to them!
- The buzzer goes off randomly for 2 out of every 10 seconds regardless of which path used, so there’s 100% no skill involved at all High Involvement – shown how to do the task by the instructor Low Involvement – got some instructions to read High familiarity – the plug is broken, have two minutes to play around with it while I fix it Low Familiarity – here you go, its time to start Here’s my favourite set of statistics from the whole paper - with high familiarity and high involvement, participants were confident that they would do a better job at the task than someone high functioning from a different logical or intellectual field like a chess champion. Remember – there is no skill involved in this, it just feels like the inputs are working because we’re getting the results we would expect if we were doing it well. So what does this tell us?
- It tells us that if I feel like I’m affecting the outcome, I will be more confident in my ability to get it right So how might we see this affect us as testers? Well….
- We need to be mindful of this and careful to ensure that when we are involved in delivery, that we are ACTIVELY INVOLVED in it. We need to ask ourselves whether what we’re doing is actually making a difference or actually proving anything. I am a fan of meta testing techniques such as mutation testing – automated tests that take AST’s that have been written previously and swap the operators in them around to see if they still pass. If they do, then it proves to you that your test is weak and doesn’t actually give you the confidence you think it does. Be careful as well not to correlate a successful delivery with the success of the product once it goes live – some of the features I’m most proud of delivering, how we worked, how effective my testing was, have died a death when they’ve been put live. It can be both easy and tempting to think that because delivery is going well to get that false sense of security and not feel the need to check what our customers might think of it Similarly, when we’re working on something familiar and successful and we DO ask our customers what they think, maybe in a focus group for example, we once again have to remind ourselves to LISTEN TO WHAT THEY SAY! It can be so tempting with something we know has worked before, to turn around and question the results if they don’t match your own experience and expectations. We’ve done this before and it was successful just doesn’t cut it in the face of your ACTUAL CUSTOMERS telling you that they don’t want it, but it can be so so easy to trust your gut, trust your experience, or simply trust the positive feeling you have with the feature you’re working on. Next slide
- How do I conclude this? How do I wrap this up for you today? I was going to talk about Stephen Covey and 7 habits of highly effective people…
- I was going to talk about the differences between factors outside of your control and things you can influence I might have talked about 6 thinking hats and different ways of looking at problems, BUT THEN I found something even better! I found myself with my three kids watching the film Kung Fu Panda…
- And for those who haven’t seen it, they actually discuss the illusion of control, and it uses a beautiful analogy which I will share with you today It involves Master Shifu…
- Trying and failing to teach kung fu to Po the Panda, And Oogway…
- So that is how I will close this talk today… Oogway: My friend, the panda will never fulfill his destiny, nor you yours until you let go of the illusion of control. Shifu: Illusion? Oogway: Yes. [points at peach tree] Oogway: Look at this tree, Shifu: I cannot make it blossom when it suits me nor make it bear fruit before its time. Shifu: But there are things we *can* control: I can control when the fruit will fall, I can control where to plant the seed: that is no illusion, Master! Oogway: Ah, yes. But no matter what you do, that seed will grow to be a peach tree. You may wish for an apple or an orange, but you will get a peach. Shifu: But a peach cannot defeat Tai Lung! Oogway: Maybe it can, if you are willing to guide, to nurture it, to believe in it. Shifu: But how? How? I need your help, master. Oogway: No, you just need to believe. Promise me, Shifu, promise me you will believe.
- how to spot a peach tree – how to spot something that you cannot control, and how to manage the illusion that you can Don’t be fooled by being given a choice – remember that heuristic and use it to break the illusion, and rethink your decisions Be prepared to challenge your own thinking – its ok to make a decision, but think about why you are making it. If you can’t explain it to yourself, then you can’t explain it at all Listen to new voices – bring them into your project, and listen to what they tell you. We all know that we aren’t perfect, and its never to late to spot a mistake And lastly make sure that when you’re involved in something, that your are ACTIVELY involved. Question yourself and what you’re doing to make sure that you are actually adding value These ideas wont give you control of anything that you can’t right now, but they will help you to recognise those things that you can’t