Pollster Final Project Presentation
•Als PPTX, PDF herunterladen•
0 gefällt mir•423 views
Pollster is a system that seeks to bring together two markets of users with complementary needs and abilities. Pollsters and Voters. Pollsters are individuals with a decision to make, for whom specific feedback from informed individuals would help them greatly. Pollsters format any question into a small ‘micro-survey’ and select who in our Voter Community they’d like to hear back from. Pollsters can pick which Voters based on filtering criteria like location, experience and age, or other ‘learned’ features like proficiency at math, experience in the retail sector or proven ability to identify winning energy stocks. While our system can be used to poll a crowd indiscriminately, the real power is in its ability to ask questions of individuals who can demonstrate expertise in some ability or area of knowledge. Voters, are individuals with spare time and the desire to earn a little extra income. Voters are notified whenever a new poll arrives in their inbox, and are rewarded by Pollsters for answering polls and providing helpful feedback. Voters wishing to demonstrate some expertise may opt to take training polls aimed at showing Pollsters they know their stuff. In its most recent iteration, this system has been tailored to support Pollsters trying to decide on which version of an online ad to run, in the hope that a smarter decision will drive more users to their website.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Empfohlen
Empfohlen (20)
Pollster Final Project Presentation
- 1. Pollster Shubham Goel Xavier Malina Advisor Marti Hearst A micro-survey platform
- 2. Need Process Solution Impact ■ Finding a real world problem to solve Problem
- 3. • Performance-driven ad Optimization • A/B Testing • “Crow d-sourced creativity of Humans” Our partner & sponsor Need Process Solution Impact Problem
- 4. Need Process Solution Impact Problem Version A Which ad will drive more users to my website?
- 5. Need Process Solution Impact Problem Which ad will drive more users to my website? Version A Version B
- 6. Need Process Solution Impact Problem Version A Version B Show Version A 100,000 times Show Version B 100,000 times ??? Clicks ??? Clicks Which ad will drive more users to my website? $5 per 1,000 views $1 per click Minimum cost $1,000
- 7. Time Version A Version B Week 1 800 Clicks 30 Clicks Week 2 Week 3 Result Total Cost: Best Ad Result: Which ad will drive more users to my website?
- 8. Time Version A Version B Week 1 800 Clicks 30 Clicks Week 2 700 Clicks 10 Clicks Week 3 Result Total Cost: Best Ad Result: Which ad will drive more users to my website?
- 9. Time Version A Version B Week 1 800 Clicks 30 Clicks Week 2 700 Clicks 10 Clicks Week 3 500 Clicks 60 Clicks Result Total Cost: Best Ad Result: Which ad will drive more users to my website?
- 10. Time Version A Version B Week 1 800 Clicks 30 Clicks Week 2 700 Clicks 10 Clicks Week 3 500 Clicks 60 Clicks Result 2000 Clicks 100 Clicks Total Cost: Best Ad Result: Which ad will drive more users to my website?
- 11. Time Version A Version B Week 1 800 Clicks 30 Clicks Week 2 700 Clicks 10 Clicks Week 3 500 Clicks 60 Clicks Result 2000 Clicks 100 Clicks Total Cost: $3,100 Best Ad Result: Which ad will drive more users to my website?
- 12. Time Version A Version B Week 1 800 Clicks 30 Clicks Week 2 700 Clicks 10 Clicks Week 3 500 Clicks 60 Clicks Result 2000 Clicks 100 Clicks Total Cost: $3,100 Best Ad A Result: Run Version A Which ad will drive more users to my website?
- 13. Time Version A Version B Week 1 110 Clicks 30 Clicks Week 2 40 Clicks 100 Clicks Week 3 100 Clicks 120 Clicks Result 250 Clicks 250 Clicks Total Cost: $1,500 Best Ad Which ad will drive more users to my website?
- 14. Time Version A Version B Week 1 110 Clicks 30 Clicks Week 2 40 Clicks 100 Clicks Week 3 100 Clicks 120 Clicks Result 250 Clicks 250 Clicks Total Cost: $1,500 Best Ad ??? Which ad will drive more users to my website?
- 15. “Which photograph should I place on my dating profile?” “Which slogan resonates most with my customers?” “Which offer would bring the most new patrons to my store?” Version A ? Version B
- 16. Need Process Solution Impact Problem “The biggest challenge in A/B testing is having the ability to get quality answers from a crowd of people.” - Bill Sherby, Analytics Manager at Boost Media Inc. The Problem
- 19. Hypothesis 1: A crowd of respondents can identify the superior version
- 20. Survey 1: 30 Boost Writers
- 21. How would the crowd respond?
- 22. Our Surveys
- 23. Survey 1: 30 Boost Writers
- 24. Survey 1: 30 Boost Writers 53% Correctly Identified
- 25. Survey 2: 100 Survey Monkey Respondents 56% Correctly Identified
- 26. A crowd of respondents can identify the superior version Hypothesis 1:
- 27. Some individuals have a talent for identifying the superior version Hypothesis 2: I think Version A will attract more visitors because…
- 28. Some individuals have a talent for identifying the superior version A crowd of respondents can identify the superior version Hypothesis 1: Hypothesis 2:
- 29. Training respondents can increase a group’s ability to identify the superior version Hypothesis 3:
- 30. Our Surveys
- 31. Training
- 32. Survey 3: 35 Boost Writers 70% Correctly Identified Survey 4: 50 Survey Monkey Respondents
- 33. “This survey is very important to develop the ability to know which ad may have more clicks than others. Good exercise. It also gives me tips and reminders on how to write effective ad copies for Boost clients. Thank you for this opportunity. :) I thought the survey was very interesting and really made me think about why people click on certain ads and not others. Actually It was hard to choose. I believe that I need to learn more [sic.]. But tips & test were useful for me. I hope my answers [sic.] will be useful also. It's been exciting and wonderful. Training Feedback
- 34. Some individuals have a talent for identifying the superior version A crowd of respondents can identify the superior version Hypothesis 1: Hypothesis 2: Training Opinion Leaders can increase a group’s ability to identify the superior version Hypothesis 3:
- 35. Results • Respondents get better at identifying winner with training • Training results can help filter out poor performers Need Process Solution Impact Problem
- 36. Results • A crowd of respondents can correctly identify the winning version • Useful Human feedback Need Process Solution Impact Problem
- 37. Need Process Solution Impact Problem “The biggest challenge in A/B testing is having the ability to get quality answers from a crowd of people.” - Bill Sherby, Analytics Manager at Boost Media Inc. Solving the Problem
- 38. Two groups of users. Joan
- 39. Two groups of users. Joan Alex
- 40. Two groups of users. Joan Alex
- 41. Two groups of users. • Creates micro-surveys • Selects Opinion Leaders to poll • Receives Targeted Feedback Joan Is a Pollster Alex Is an Opinion Leader
- 42. Two groups of users. • Creates micro-surveys • Selects Opinion Leaders to poll • Receives Targeted Feedback Joan Is a Pollster Alex Is an Opinion Leader • Trains to improve abilities • Gives insightful feedback • Receives rewards for responding to micro-surveys
- 43. Place your screenshot here
- 44. Solution: Navigation Flow (Wireframe)
- 45. Solution: Navigation Flow (Interactive Mock)
- 47. Opinion Leader Work Flow Signup Login Train Demonstrate Expertise Respond to Micro-Surveys
- 48. Voting Page ■ Respond to Micro-Surveys ■ Train ■ Give Written Feedback
- 54. MEAN Stack
- 57. Savings Standard A/B Test Pollster Total Cost $500 - $5,000 $10 - $500 Time 3 Weeks 2 - 7 Days Feedback • Number of Clicks • Number of Clicks • Written Feedback from • Humans Need Process Solution Impact Problem
- 58. ■ Impact • Micro-survey platform • Cost-effective • Quick, Targeted, Human Feedback • Expandable Need Process Solution Impact Problem Process Solution Impact Problem
- 59. “Having [the] type of feedback [gathered by Pollster] allows us to understand what marketing messaging will work before putting it into market. [“ This] will save us money in the long run and allow us to experiment more with marketing messaging overall.” - Bill Sherby Analytics Manager at Boost Media Inc .
- 60. Future Work & Potential Uses Decisions where there is a high cost for making the wrong decision • Dating profile feedback • Fashion advice • Brick & Mortar Marketing Initiatives
Hinweis der Redaktion
- My name is Xavier Malina and this is my teammate Shubham Goel Our team set out to complete a project that met two criteria above all. First, the project must address a real-world problem, not a hypothetical one. Second, the output should be a product that stands alone: version 1.0 and not version 0.1.
- Company A month away valentine's day it can have a huge impact on their bottom line A single client could generate $50 it profits, and a 10 percent difference in clicks could mean thousands of dollars to them
- Version B
- Well, we had 1,000 professional ad writers at our disposal. So first asked ourselves: Could a group of people identify the Ad that would receive the most clicks – instead of running the A/B test? Because, if they could – our hope would be that they might be able to respond faster than a several-week A/B test. Not only that, but maybe they could provide feedback that might help shed light on the critical question: What made one version more successful?
- In this case, the company spent $2,400 on a test that ran for 3 weeks. With feedback from the test, they felt like they had enough information to make a smart decision. After all - Version A received 2,000% more clicks than version B. So it seems obvious that version A is the right choice to run in the week before Valentine’s day. But what if, instead of these results, the test had come back with results like these?
- In this case, the company spent $2,400 on a test that ran for 3 weeks. With feedback from the test, they felt like they had enough information to make a smart decision. After all - Version A received 2,000% more clicks than version B. So it seems obvious that version A is the right choice to run in the week before Valentine’s day. But what if, instead of these results, the test had come back with results like these?
- In this case, the company spent $2,400 on a test that ran for 3 weeks. With feedback from the test, they felt like they had enough information to make a smart decision. After all - Version A received 2,000% more clicks than version B. So it seems obvious that version A is the right choice to run in the week before Valentine’s day. But what if, instead of these results, the test had come back with results like these?
- In this case, the company spent $2,400 on a test that ran for 3 weeks. With feedback from the test, they felt like they had enough information to make a smart decision. After all - Version A received 2,000% more clicks than version B. So it seems obvious that version A is the right choice to run in the week before Valentine’s day. But what if, instead of these results, the test had come back with results like these?
- In this case, the company spent $2,400 on a test that ran for 3 weeks. With feedback from the test, they felt like they had enough information to make a smart decision. After all - Version A received 2,000% more clicks than version B. So it seems obvious that version A is the right choice to run in the week before Valentine’s day. But what if, instead of these results, the test had come back with results like these?
- In this case, the company spent $2,400 on a test that ran for 3 weeks. With feedback from the test, they felt like they had enough information to make a smart decision. After all - Version A received 2,000% more clicks than version B. So it seems obvious that version A is the right choice to run in the week before Valentine’s day. But what if, instead of these results, the test had come back with results like these?
- And this result is more common than the first result we saw. One figure we saw indicated that as few as 1 out of 8 tests like these generate good enough feedback for a decision-maker to make the right decision. The company just spent a lot of money and came back with no good feedback for their client. And even in a case where there is a good result – they know they turned away customers that saw the inferior version. If they had been able to run this test in the lab, then at least they could have followed up with them and asked: hey, what do you think might have made you more likely to click on the ad more? Or ask them about what they want in a flower delivery service? And use the responses to improve the ads. A/B Testing is expensive, and often costs thousands of dollars for a single test Gathering sufficient results to interpret can take weeks A/B Tests often don’t produce interpretable results Because A/B Testing only works on ‘unwitting’ participants, results reflect actions but cannot capture any other feedback “Appsumo reveals its A/B testing secret: only 1 out of 8 tests produce results” https://vwo.com/blog/a-b-testing-tips/ A/B Testing is expensive, and often costs thousands of dollars for a single test Gathering sufficient results to interpret can take weeks A/B Tests often don’t produce interpretable results Because A/B Testing only works on ‘unwitting’ participants, results reflect actions but cannot capture any other feedback “Appsumo reveals its A/B testing secret: only 1 out of 8 tests produce results” https://vwo.com/blog/a-b-testing-tips/
- And this result is more common than the first result we saw. One figure we saw indicated that as few as 1 out of 8 tests like these generate good enough feedback for a decision-maker to make the right decision. The company just spent a lot of money and came back with no good feedback for their client. And even in a case where there is a good result – they know they turned away customers that saw the inferior version. If they had been able to run this test in the lab, then at least they could have followed up with them and asked: hey, what do you think might have made you more likely to click on the ad more? Or ask them about what they want in a flower delivery service? And use the responses to improve the ads. A/B Testing is expensive, and often costs thousands of dollars for a single test Gathering sufficient results to interpret can take weeks A/B Tests often don’t produce interpretable results Because A/B Testing only works on ‘unwitting’ participants, results reflect actions but cannot capture any other feedback “Appsumo reveals its A/B testing secret: only 1 out of 8 tests produce results” https://vwo.com/blog/a-b-testing-tips/ A/B Testing is expensive, and often costs thousands of dollars for a single test Gathering sufficient results to interpret can take weeks A/B Tests often don’t produce interpretable results Because A/B Testing only works on ‘unwitting’ participants, results reflect actions but cannot capture any other feedback “Appsumo reveals its A/B testing secret: only 1 out of 8 tests produce results” https://vwo.com/blog/a-b-testing-tips/
- A / B tests can be run to help make a decision in all kinds of situations, but costs of A/B tests are too high for them to be useful in many of them (both in terms of time and cmonetary costs)
- The problems affect all of them. How to get quality feedback from a crowd of people you can trust is the problem Boost presented us with.
- Well, we had 1,000 professional ad writers at our disposal. So first asked ourselves: Could a group of people identify the Ad that would receive the most clicks – instead of running the A/B test? Because, if they could – our hope would be that they might be able to respond faster than a several-week A/B test. Not only that, but maybe they could provide feedback that might help shed light on the critical question: What made one version more successful?
- Write that it was boost media experts
- Write that it was boost media experts
- Write that it was boost media experts
- Were there some individuals more capable at making these decisions than others?
- Write that it was boost media experts
- Well, we had 1,000 professional ad writers at our disposal. So first asked ourselves: Could a group of people identify the Ad that would receive the most clicks – instead of running the A/B test? Because, if they could – our hope would be that they might be able to respond faster than a several-week A/B test. Not only that, but maybe they could provide feedback that might help shed light on the critical question: What made one version more successful?
- Well, we had 1,000 professional ad writers at our disposal. So first asked ourselves: Could a group of people identify the Ad that would receive the most clicks – instead of running the A/B test? Because, if they could – our hope would be that they might be able to respond faster than a several-week A/B test. Not only that, but maybe they could provide feedback that might help shed light on the critical question: What made one version more successful?
- Well, we had 1,000 professional ad writers at our disposal. So first asked ourselves: Could a group of people identify the Ad that would receive the most clicks – instead of running the A/B test? Because, if they could – our hope would be that they might be able to respond faster than a several-week A/B test. Not only that, but maybe they could provide feedback that might help shed light on the critical question: What made one version more successful?
- Based on the results we got from the survey we started working on our system. We realized that the problem was more generalizable. It is the problem of collection quality feeback. Our solution has two stakeholders. Joan is a marketing executive at a new fashion start-up and is two weeks away from the launch of a major ad campaign for their newest product, and there’s a problem. Two slogans have been put forward as the best candidates to go on their online ads, but no clear winner has emerged after two months of discussion. Paralysis has gripped the creative team and Joan is stuck. There isn’t enough time to enroll the services of a market research firm, and even if there were – they wouldn’t be able to afford it. Making smart decisions can be difficult.
- Alex is a stay at home parent who used to manage a chain of small retail stores that catered to the same clientele as Joan’s company. Alex has been looking for ways to earn a little extra income for his family, but his kids keep him busy and his schedule is often erratic. He’d love a way to earn money while he’s sitting in the school parking lot, or when he’s in line at the grocery store. In a perfect world, Alex would love to be paid for doing something with his mind and not his hands.
- Alex is a stay at home parent who used to manage a chain of small retail stores that catered to the same clientele as Joan’s company. Alex has been looking for ways to earn a little extra income for his family, but his kids keep him busy and his schedule is often erratic. He’d love a way to earn money while he’s sitting in the school parking lot, or when he’s in line at the grocery store. In a perfect world, Alex would love to be paid for doing something with his mind and not his hands.
- With these goals in mind we setout to design our system. We worked on the navigation flow to support all the interactions for both the Opinion Leaders and pollsters
- We worked on the wireframes
- And made an interactive low fidelity prototype to test out the system navigation.
- And made an interactive low fidelity prototype to test out the system navigation.
- H
- For us the most important page was the voting page: This is the page the Opinion Leaders use to express their feedback. It is also the page they are show during the training. We played around with different layouts: Positioning of the two advertisements being compared Position of the voting buttons We wanted to remove bias
- Teach how this works What did we learn from using this stack? Refreshing
- At this point in the project, we had I identified a partner with a pressing need. We had access to a large dataset past A/B tests the company run – along to knowing in each case which ad received the most clicks. And we also had access to another resource the company made available to us: 1,000 writers in the Boost Creative Network who Boost would let us send surveys if we felt it could help us in our research. And lastly a question: Can we build a solution that addresses the problems of A/B testing and helps Boost’s clients make smarter decisions? Next we’ll talk about the process we went through to evaluate this problem.
- For us the most important page was the voting page: This is the page the Opinion Leaders use to express their feedback. It is also the page they are show during the training. We played around with different layouts: Positioning of the two advertisements being compared Position of the voting buttons We wanted to remove bias
- For us the most important page was the voting page: This is the page the Opinion Leaders use to express their feedback. It is also the page they are show during the training. We played around with different layouts: Positioning of the two advertisements being compared Position of the voting buttons We wanted to remove bias
- For us the most important page was the voting page: This is the page the Opinion Leaders use to express their feedback. It is also the page they are show during the training. We played around with different layouts: Positioning of the two advertisements being compared Position of the voting buttons We wanted to remove bias
- The layout that we picked was one with the two ads listed next to each other. The ads are not marked by text as version A and B or 1 and 2 The user has four actions at any stage