Recommended
Recommended
More Related Content
What's hot
Similar to Perspectives of software quality (Let's reverse the quality pyramid!)
Similar to Perspectives of software quality (Let's reverse the quality pyramid!) (20)
Recently uploaded
Recently uploaded (20)
Perspectives of software quality (Let's reverse the quality pyramid!)
- 1. Perspectives of Software Quality (Let’s reverse the quality pyramid!)
- 2. 2008: marketplace AlittleMarket.com 2011: marketplace AlittleMercerie.com 2014: Rachat par Etsy Des centaines de milliers de créateurs français Des millions de produits 41% des créatrices sur le site ont leur entreprise créative comme unique activité
- 4. Summary Why ? Big picture of Quality for everyone
- 8. Definition of software quality “Quality is value to some person (who matters)” Jerry Weinberg “X is X to some person (at some time)” Michael Bolton
- 9. (Redefining software quality from Gojko Adzic)
- 10. 1/5 - Functionally OK Quality is correctness; conformance to specifications “Does it work ?”
- 12. Source : Akamai Survey April 2006 1058 online shoppers
- 13. 2/5 - Performant / Secure Does it work well?
- 17. 3/5 - Usability Is it usable ? What are the key usability scenarios?
- 18. Example : Infusion Pumps ● Insulin ● Hormones ● Antibiotics ● Chemotherapy ● Pain relief drugs ● Nutritional ● Millions in use around the world
- 19. ● Between 2005 and 2009, FDA (Food and Drug Administration) reported : ○ 56 000 infusion pumps incidents ○ 710 deaths ○ 200 000 recalls (Mars 2010) ● A large percentage of these adverse events are due to programming errors that can be attributed to poor usability 1. Entering weight in pounds instead of kg - results in 2.2x overdose 2. Incorrectly placed decimal point - result in 10x under or over infusion 3. Select incorrect dose mode - mg/kg/min instead of mcg/kg/min - results in 1000x overdose
- 20. Use error = device defect ! Fixation is on things not on people Usability validation required by FDA FDA White paper : Infusion pumps improvement initiative (April 2010)
- 23. Average number of subtitles characters from all shops: 137 characters
- 27. 4/5 - Usefulness Is it useful ? What production metrics will show that it is used in real work?
- 28. VERSUS
- 33. 5/5 - Successful Is it successful? What business metrics will show that this was worth doing?
- 34. We have lots of ideas to make our products better... But not all ideas are good. It’s humbling, but most ideas are actually bad
- 35. Power of selection 10 ideas without measuring their impact may have about 1/3 be good 1/3 flat 1/3 negative The overall value of the 10 ideas will therefore be fairly small. On the other hand, if the team experiments with the 10 ideas and picks the successful three or four, aborting the rest, the benefits will be significant. Even though running an experiment has costs, the ability to abort bad features early and fail fast can save significant time and allow teams to focus on the truly useful features.
- 36. 3 Palchinsky principles (/Why plans tend to fail ?)
- 43. Summary: Evolve the culture Prepare to be humble Changes Rarely have a Big Positive Impact to Key Metrics Small Changes can have a Big Impact to Key Metrics (40 shades of blue) First in doing the right thing then in doing the thing right. A failure is not a mistake : Learn from it
- 44. Try radical ideas 2006 Greg Linden at Amazon created a prototype to show personalized recommendations based on items in the shopping cart Linden notes that “a marketing senior vice-president was dead set against it,” claiming it will distract people from checking out. Greg was “forbidden to work on this any further.” Nonetheless, Greg ran a controlled experiment and the rest is history: the feature was highly beneficial Improve Amazon’s revenue by 3%
Editor's Notes
- Outil de travail journalier pour les vendeurs => impact UX fort pour ces utilisateurs.
- weird tests until finding something that is not working ? Is that the definition of what is Quality ?
- Vue d’ensemble pour tout le monde
- Cube?
- Sphere ?
- at some time => People change. James Bach. “Customers perception depends among other things on their values skill level past experience and profile of use.” “Some of these factors we can study; none of them do we control.”
- https://gojko.net/2012/05/08/redefining-software-quality/ Vue d’ensemble: Fonctionnel Performant et sécurisé Utilisable Utile Succès
- Why ? Spécifications répondent à un besoin utilisateur => => => Product is the solution, it must solve problems ! <= <= <= User feelings about the product : bad confidence, emotions If the product is not focus on quality, does the workers are ? Company credibility, image, values “it isn’t the number of the bugs that matters, it’s the effect of each bug.” The goal is to reach an acceptable level of risk, at that point quality is automatically good enough
- SOURCE : https://www.akamai.com/us/en/multimedia/documents/report/akamai-site-abandonment-final-report.pdf VOLUMETRIE (probabilité * severité)
- Expérience de Google avec un clignement des yeux a 400 millisecond delay into its delivery of search results 6 weeks => 1% de recherche en moins par utilisateur several hundred million dollars a year in potential ad revenue Greg Linden, Amazon showed a 1% sales decrease for an additional 100msec Etsy : 200 ms au search => slide qui vient http://mcfunley.com/design-for-continuous-experimentation Focus on the user; the end goal isn't to make your site perform fast on any specific device, it's to ultimately make users happy. Respond to users immediately; acknowledge user input in under 100ms. When animating or scrolling, produce a frame in under 10ms. Maximize main thread idle time. Keep users engaged; deliver interactive content in under 1000ms.
- Absolutely nothing happens. People buying items are not sensitive about performance at all.
- Why ? Avoid errors => use errors Avoid confusion => “close call” Avoid difficulty (cf “Don’t make me think” - Steve Krug) Human behaviour oriented
- Units of measurement : pounds / kilograms medical device : “death by decimal” (80 instead of 0.8) “Key bounce” : infusion rate of 10 mL/hour, but the device registers an infusion rate of 100 mL/hour http://incompliancemag.com/article/usability-engineering-observe-users-improve-product-safety/
- Bug d’usage human factors
- medical device usability : 5-8 participants reveal 80-90% of problems. 15 for validation appropriate sample size ? multiple of 15 by different group of users. 6 different groups of users => 6 * 15 = 90 participants Group of users : médecin / infirmiers / sage femme / patient à la maison etc … UX validation obligatoire par la FDA
- Why you only needs to test with 5 users ?
- 137 caractères en moyenne
- http://www.getmentalnotes.com/cards https://baymard.com/blog/drop-down-usability Avoid difficulty => pas plus de 15 choix => pas moins de 7 items sinon la liste n’est pas adaptée. too many options (plus de 45 choix)
- Why ? 64% of product features are rarely or never used source : Standish Group 2002 (2000 projets sur 1000 companies) => here Money consuming for nothing Diluting value proposition (less is more) vs complicated product, Technical debt, maintenance cost, image degradation Getting fresh air, new ideas.
- Sur le contenu dont Amazon n’a pas la main, il demande à ses utilisateurs si l’information affichée est utile ou non.
- Amazon / Google (Géants du e-commerce) ne savent pas non plus si les gens vont utiliser ou pas certaines de leurs features Utile (d’un point de vue business) => si on achète la bonne pointure on ne retourne pas la chaussure. => si on achete le bon produit (grâce aux avis) on ne le rend pas. (ou on ne l’achète pas)
- Does it save money, earn money or protect money ? Change user’s behaviour =>Customer IMPACT ? Valuable for the customer? Valuable for the company (learning) What is the value of an experiment? Absolute value of delta between expected outcome and actual outcome is large If you thought something is going to win and it wins, you have not learned much If you thought it was going to win and it loses, it’s valuable (learning) If you thought it was “meh” and it was a breakthrough, it’s HIGHLY valuable
- Ronny Kohavi (his paper 27,000 citations, 3 of his papers are in the top 1,000 most-cited papers in Computer Science) 12 ans d’A/B testing 30 - 50 experiments starting every day at Bing Mêmes scores chez Netflix / Amazon / Google / Microsoft / Google 12.000 experiments en 2009 => 10 percent change business http://ai.stanford.edu/~ronnyk/ExPThinkWeek2009Public.pdf https://www.youtube.com/watch?v=ZfhQ-fIg4EU&feature=youtu.be&t=2m59s http://www.exp-platform.com/Pages/default.aspx
- Palchinsky (a russian engineer) (père lean management) https://emdeha.github.io/2016/08/04/palchinsky.html
- 70 % more people gave up and left the site before having the new tab. (New tab ok sur Bing) Dan McKinley FAIL : infinite scroll (6 months), afficher plus de produit par page =>> We choose have done better job to understand the people using our website (verify the premises)
- Hippo : Highest Paid Person Opinion
- Underlines Ads because numerous A/B tests proves that underlines induce more clicks.
- gmail / website : not the same blue color. Objectif : change people's behaviour => clicking more in the ads 2012 : - Doug Bowman, the company's Head of design 40 shades of blue => Doug Bowman (quit or fire) 2014 : an extra $200 million a year in revenue https://www.theguardian.com/technology/2014/feb/05/why-google-engineers-designers Humans are not predictable Experts can take bad decision (Limit of expert judgment) => be humble again !!!! Sometimes we need to release things to test if it’s a good idea or not
- Andrew Morrison (most of ideas failed) Even when AB test works, you don’t know why it works as it works.
- “define success as something learnt” We are terrible at assessing the value of ideas.
- http://glinden.blogspot.fr/2006/04/early-amazon-shopping-cart.html
- start with working on the right thing first then on how to do it right. (successful, usefulness), then functionally OK, then performance and security, then usability ? Find goals that are valuable