Recommended
More Related Content
Similar to Getting Things Done with Crowdsourcing PWI May-2014
Similar to Getting Things Done with Crowdsourcing PWI May-2014 (20)
Recently uploaded
Recently uploaded (20)
Getting Things Done with Crowdsourcing PWI May-2014
- 1. http://bitsofknowledge.waterloohills.com • What is Crowdsourcing? • Crowd Motivations • What can it be used for? • Pro’s & Con’s • Quality Mgt & Workflows • Economical shift PWI – May 22, 2014 corina@waterloohills.com http://bitsofknowledge.waterloohills.com
- 2. http://bitsofknowledge.waterloohills.com Crowd or Community (online audience) Crowdsourcing 1 2 3 4
- 7. http://bitsofknowledge.waterloohills.com Ex: “Adult Websites” Classification • Large number of sites to label • Get people to look at sites and classify them as: – G (general audience) – PG (parental guidance) – R (restricted) – X (porn) [Panos Ipeirotis. WWW2011 tutorial]
- 8. http://bitsofknowledge.waterloohills.com Ex: “Adult Websites” Classification • Large number of hand labeled sites‐ • Get people to look at sites and classify them as: – G (general audience) – PG (parental guidance) – R (restricted) – X (porn) Cost/Speed Statistics: • Undergrad intern: 200 websites/hr, cost: $15/hr • MTurk: 2500 websites/hr, cost: $12/hr [Panos Ipeirotis. WWW2011 tutorial]
- 9. http://bitsofknowledge.waterloohills.com Crowd Demography (background defines motivation) • Amazon’s Mechanical Turk workers come mainly from 2 countries: a) USA b) India
- 11. http://bitsofknowledge.waterloohills.com • What is Crowdsourcing? • Crowd Motivations • What can it be used for? • Pro’s & Con’s • Quality Mgt & Workflows • Economical shift PWI – May 22, 2014 corina@waterloohills.com http://bitsofknowledge.waterloohills.com
- 12. http://bitsofknowledge.waterloohills.com Crowd Motivation • €,$ = Money! • Self-serving purpose (learning new skills, get recognition, avoid boredom, enjoyment, create a network with other profesionals) • Socializing, feeling of belonging to a community, friendship • Altruism (public good, help others)
- 14. http://bitsofknowledge.waterloohills.com • What is Crowdsourcing? • Crowd Motivations • What can it be used for? • Pro’s & Con’s • Quality Mgt & Workflows • Economical shift PWI – May 22, 2014 corina@waterloohills.com http://bitsofknowledge.waterloohills.com
- 15. http://bitsofknowledge.waterloohills.com What kind of activities can be done? • Find Suppliers for: Creative work (logos, websites, products..) Look for specific talent Software Testing Support (call centers, secretarial tasks, translations) To offload peak demands Tackle problems that need specific communities or human variety
- 16. http://bitsofknowledge.waterloohills.com Mass work, Distributed work, or just tedious work. 3 main goals: 1. Minimize Cost (cheap) 2. Minimize Completion Time (fast) 3. Maximize Quality (good) Remember Crowd Motivation! (ex.: Game-ify your task, Explain the final purpose) What else can be done?
- 19. http://bitsofknowledge.waterloohills.com Gamification and Collaboration • Game FoldIt: In 2011, a puzzle was presented to the participants. Gamers built on each other’s results, and in 10 days they found the molecular structure of a protein- cutting enzyme from an AIDS-like virus.
- 21. http://bitsofknowledge.waterloohills.com And still more… • Market research • Innovation • Create buzz, generate customers! • Find support, be backed up • Crowdsourcing is your business! • Be financed: Crowdfunding
- 22. http://bitsofknowledge.waterloohills.com Examples of Funding US: JOBS act legal to give equity to private investors. Europe: check if crowdfunding site is authorised by financial regulators (US/UK) (US/DE) (US) (LU) CrowdCube (UK) Symbid (NL) Angel.me (BE) Identitycoop (BE) LookandFin (BE) Patreon (US) Companisto (DE) Seedrs (UK) Siamosoci (IT)
- 23. http://bitsofknowledge.waterloohills.com • What is Crowdsourcing? • Crowd Motivations • What can it be used for? • Pro’s & Con’s • Quality Mgt & Workflows • Economical shift PWI – May 22, 2014 corina@waterloohills.com http://bitsofknowledge.waterloohills.com
- 24. http://bitsofknowledge.waterloohills.com Pro’s • Quicker: Parallellism reduces time • Cheap • Creativity, Innovation • Quality (*depends) • Access to scarce resources: The ‘long tail’ • Multiple feedback • Allows to create a community (followers) • Business Agility • Scales up! (*up to a level)
- 25. http://bitsofknowledge.waterloohills.com Con’s • Lack of professionalism: Unverified quality • Too many answers • No standards • Not always cheap: Added costs to bring a project to conclusion • Too few participants if task or pay is not attractive • If worker is not motivated, lower quality of work • Legal issues
- 26. http://bitsofknowledge.waterloohills.com • What is Crowdsourcing? • Crowd Motivations • What can it be used for? • Pro’s & Con’s • Quality Mgt & Workflows • Economical shift PWI – May 22, 2014 corina@waterloohills.com http://bitsofknowledge.waterloohills.com
- 27. http://bitsofknowledge.waterloohills.com Crowdsourcing websites Workflow Patterns • Generate / Create • Find • Improve / Edit / Fix Creation • Vote for accept reject‐ • Vote up, vote down, to generate rank • Vote for best / select top k‐ Quality Control, e-Reputation • Split task • Aggregate Flow Control • Iterate Flow Control
- 29. http://bitsofknowledge.waterloohills.com • What is Crowdsourcing? • Crowd Motivations • What can it be used for? • Pro’s & Con’s • Quality Mgt & Workflows • Economical shift PWI – May 22, 2014 corina@waterloohills.com http://bitsofknowledge.waterloohills.com
- 30. http://bitsofknowledge.waterloohills.com Economical Shift • From Social Networking to Social Production through Collaborative Innovation Mass-Collaboration changes how Products & Services are Designed, Manufactured, Marketed • Classical geo-political and economical organisations do not correspond to new economy Realignment of competitive advantages Move towards Collaborative Enterprises based on Open Infrastructure
- 31. http://bitsofknowledge.waterloohills.com Societal Shift Moral values Reinforcement Creates positive correlation between Ethical values and ROI Open Data access Transparent actions Accountable people Integrity E-Reputation Community Support
- 32. http://bitsofknowledge.waterloohills.com corina@waterloohills.com http://bitsofknowledge.waterloohills.com PWI – May 22, 2014 Questions?
- 33. http://bitsofknowledge.waterloohills.com References • Wikipedia,2011 • Dion Hinchcliffe Crowdsourcing: 5 Reasons Its Not Just For Start Ups Anymore,2009 • Tomoko A. Hosaka, MSNBC. "Facebook asks users to translate for free“ ,2008. • Daren C. Brabham. "Moving the Crowd at iStockphoto: The Composition of the Crowd and Motivatio First Monday, 13(6),2008. • Karim R. Lakhani, Lars Bo Jeppesen, Peter A. Lohse & Jill A. Panetta. The value of openness in scientific problem solving (Harvard Business School Working Paper No. 07-050),2007. • Klaus-Peter Speidel How to Do Intelligent Crowdsourcing,2011 • Panos Ipeirotis. Managing Crowdsourced Human Computation, WWW2011 tutorial,2011 • Omar Alonso & Matthew Lease. Crowdsourcing 101: Putting the WSDM of Crowds to Work for You, WSDM Hong Kong 2011. • Sanjoy Dasgupta, http://videolectures.net/icml09_dasgupta_langford_actl/,2009 •Don Tapscott, Anthony Williams. Macrowikinomics, 2010.
Editor's Notes
- The boum of Internet access allowed a rise of a new phenomenum that is the organisation of people with same interests: fans of .., customers of … users of … forums around a particular skill PRs, Marketers and other people thought of targeting that grouped audience with: - polls: what do you like the most or the least about the product? - requesting collaboration like: would you like to post a video of you using the product?,..who could do this or that to collaborate? Creating buzz: I am creating the next CD or film, it will be ready for … And then websites organising this need appeared, conglomerating people of a particular skill, providing an entry point to reach the crowd. This movement of reaching a crowd is what has been called crowdsourcing To understand how it works, we need to understand why is people participating. It will help us to see if an idea will succeed with crowdsourcing. Or how to present it in order to succeed. I will then mention some needs that can be satisfied with crowdsourcing nowadays. I will then discuss the pros and cons of this approach, so that you bare them in mind before deciding to use it. One of the issues is quality, so I talk specifically about what is being done to tackle this issue To conclude, I’ll mention how this business model shakes our economic geopolitical divisions
- June 2006: Jeff Howe created the term for his article in the Wired magazine "The Rise of Crowdsourcing".Depends on an online audience. It cannot survive without digital interactions, without the buzz in the social web. For that, cultivating a charismatic image is important in order to attract people. Companies can create open crowds or communities: People in the community don’t know each other, they are spread all over the world, so openness and share-ness at least within the group is essential in maintaining a sense of ‘belonging’. But a company can create also exclusive communities, based on human nature: spread the word about it, but reserve secrecy in details that only the ‘belonging-ones’ share. Nobody likes to be left apart, it is also a great motor to build communities, that’s how Facebook begun, or Gmail that is only by invitation. You set a challenge to the audience. The client posts a challenge (mental challenge like exposing a problem, or physical challenge like requesting a work to be done,..), and the members of the crowd submit their solutions. Answers are sent back to the client, and usually property of the solution goes to the client. The winner(s) usually get a reward for their participation Opensource is the same basically, the difference is on who’s the motivator : in crowdsourcing, a client posts a request, in opensource, the members of the community initiate it, even though we can find a cooperative task in both cases. If we follow this distinction, Wikipedia is not crowdsourcing but opensource. . Companies may: (a) pay for a task, (b) set a prize for it (in funds or moral recognition), (c) request their users to take action on ethical bases (for free if you want). This last one is powerful and a key in the 21th century! In times where goodies are available for (most) of the internauts, thus human basic needs covered (again we are not talking about the whole world unfortunately), society is engaging/turning more into long-term goals, with greater objectives like shaping a better future for all of us. This includes ecologic movements, human right movements, ethical questions as stem cells and more, depending on the vision of the followed greater purpose. Now, as it’s human nature to expect something in return for our actions, it would be wise that somehow the member sees the effect of their action, or at least that somebody noticed it :-)This profound reward (why we do the job) is very powerful and in the near future will be the most important factor to determine whether your business survives or not.
- . Companies may: (a) pay for a task, (b) set a prize for it (in funds or moral recognition), (c) request their users to take action on ethical bases (for free if you want). This last one is powerful and a key in the 21th century! In times where goodies are available for (most) of the internauts, thus human basic needs covered (again we are not talking about the whole world unfortunately), society is engaging/turning more into long-term goals, with greater objectives like shaping a better future for all of us. This includes ecologic movements, human right movements, ethical questions as stem cells and more, depending on the vision of the followed greater purpose. Now, as it’s human nature to expect something in return for our actions, it would be wise that somehow the member sees the effect of their action, or at least that somebody noticed it :-)This profound reward (why we do the job) is very powerful and in the near future will be the most important factor to determine whether your business survives or not.
- Bolder asks individuals and organizations to make a difference together. social initiative to do something ‘good’, creates a community (Facebook ‘likes’ needed) How Bolder Works Select & complete a challenge Pick a challenge to do something good, then go out and make it happen! Write your story, get your reward Write a short story about your action and get your reward * * Bolder emails your reward to you, so be sure to create an account Challenge yourself - make your action meaningful & post a picture The highest voted action (determined by number of Facebook "likes") for each challenge will be featured on the blog and will receive special rewards!
- The crowd is your supplier Creative work: It can be a graphical design, fashion, architecture, Websites to reach millions of designers: Crowdspring, 99designs but also problem-solving (Quora, Yahoo?Answers, Ask) or even bigger challenges as finding solutions to our society problems like traffic planning or even the climate change: Transit Planning in University Utah campus design bus-stops GE Ecomagination challenge look for “green technologies”, they got ideas Specific Talent: You are looking for the ‘pearl’ a) at a reasonable costb) the best one!c) or more than the best one, like a global answer, that will have more than the sum of the individual participations. Testing (the variety of an heteroclite group would be a plus) and getting feedback from it. (uTest) Translation, Support (can get Multilanguage support easily) Crowdsourced customer support allows businesses to rely on customers to solve other customers issues and questions. Specific communities: identifying needs of other cultures, or helping in catastrophes
- Mass work, Distributed work, or just tedious work with these characteristics: a) You have a lot of work to be done. Your task may or not be computerised, but it can be split in small and simple chunks of work. b) You cannot afford getting the resources to do it c) Your task needs to be done simultaneously (thus needing many resources for just one time) d) Your task needs collaboration of distributed resources (like measuring the temperature in different physical locations around the world at the same time) Example: Galaxy Zoo project: a crowd analyzing Hubble images to help them classifying galaxies as in the Simple tasks: Websites clasification, Annotations (image or music tags, recognition) Temporal ordering, Relevance, Annotation (tags, recognition) Strategy Collaboratition: collaborative effort to be successful, competition to be performant. A good example is the 2009 DARPA experiment: they paced 10 balloon markers across the United States and challenged teams to compete to be the first to report the location of all the balloons. Collaboration of efforts was required to complete the challenge quickly and in addition to the competitive motivation of the contest as a whole, the winning team (MIT, in less than seven hours) established its own "collaborapetitive" environment to generate participation in their team. Game-ify it??
- ESP: describe an image saying words, you have to match your partner’s word. Tag a Tune: describe it Verbosity:Give clues for somebody else to discover (common sense) Squirgl: draw the contour of one part of the image (a dog) Matchit: Say which of 2 images you prefer FlipIt: find another image of the same object ALL: have TOP SCORES of Day, Month, All
- The crowd is your market. What about if you have something and would like the crowd to buy it? Lays asked for their consumer’s best flavor and got a million answers from across India… Look how cost effective it is: on one hand the cost of the contest plus the prize versus identifying (usually tested by sampling) what the users want, plus the cost of the promotion campaign, plus the risk of failing finding what their consumers wanted? They can also be ambassadors of your ideas. Innovation: Crowdsourcing, in its most crude form distributes the power of contribution as widely as possible. No more privileges. Finally, innovation is being democratized. Virtues: by eliminating biases that lead to unjustified preference for certain contributors, it can allow you to find solutions you had not been looking for. If those who contribute ideas and vote on others’ ideas are also your customers – as in Dell’s Ideastorm – market research and innovation is totally integrated – and free. If there are 118,590 people (as of April 11th, 2011) who like the idea of having OpenOffice preinstalled on all Dells, you know for sure there’s a customer want. Now if Dell doesn’t want to look tyrannic to a web 2.0 public, they really have to take the idea seriously – which could be seen as a downturn of Dell’s form of Crude Crowdsourcing (this is linked to the difference between Overt vs. Covert Crowdsourcing below). And Dell’s status update, “Dell is constantly reviewing options to offer to customers. As this is available for free today, we do not currently plan on offering this as an option to factory install”, isn’t sure to satisfy the people who first suggested the Open Office pre-install. Buzz, customers: use the community as consumers, not suppliers! You may need to create awareness (demand) for your product. Let the community of consumers tell you what they need Better: let them design it! Use the community for market research to test an idea or a product before doing it. Cheetos used their users to ask them for commercials (” Got some great Cheetos® content that the world needs to see? Let us know about it! »), they not only got their consumers publicity, but also the buzz for the contest. Backed up: Propose something, and be followed. Spread an idea or work (like distribution of a film, evangelisation,..). Crowdsourcing is your business. You provide the crowd. provide tools or support for any of the parts: aggregators, private companies or public organisations that need to create a community. Need of ANONIMITY!
- Kickstarter democratizes the process of obtaining venture capital for artists and entrepreneurs. A person posts their project details and creates a funding goal. During a certain period of time, investors and individuals are able to contribute so that the project meets or exceeds that goal. The project will only get the funding if the goal is met. When the project creator reaches their goal, Kickstarter takes 5% of the monies for their service. Quirky is for designers. The most valuable part of Quirky is the feedback you can receive for your project idea. For a $10 fee, you can post your idea and then see what others think about it. This allows you to create a product or service that is going to meet your market needs, increasing your chances of success. Once you have received feedback, then Quirky will decide whether or not they want to make the product. If they do, then you get paid. With the community vibe, Quirky is more of a progressive site than a straightforward investment site. Fans Next Door is a crowdfunding site that accepts all types of creative projects: Arts, Fashion,Video games, Design, Music: There is a reward system in place, investors are compelled to invest more as they will get more for their investment when they do. European site, no additional fees outside of the PayPal processing fees. Funding: you have an idea and need to be sponsored (to create your startup, to help in the 3rd. world development,..) Crowdfunding occurs for any variety of purposes: disaster relief citizen journalism artists seeking support from fans, The Age of Stupid is the most successful case to-date; this film raised $1.2 million via crowdfunding, and also used crowdsourcing to distribute and exhibit it around the world political campaigns, funding a startup company or small business[1] or creating free software.
- Better quality in online work when work needs to be free of expectations, without having a direct judgement. In an interview with Wired, Andrea Grover, curator of the 2006 crowdsourcing art show Phantom Captain: Art and Crowdsourcing, states that individuals tend to be more open in crowdsourced projects because they are not being physically judged or scrutinized. This ultimately allows for well-designed artistic projects because individuals are less conscious, or maybe even less aware, of scrutiny towards their work. In an online atmosphere there is more attention being given to the project rather than communication with other individuals.Also people is less shy than when an opinion is requested in a meeting (as urban design…) Original ideas, Innovation emerge tapping on a crowd due to its great quantity of participants from different origins, with different experiences... Open and user-driven creative process that allows innovation: innovative solutions to difficult scientific problems can arise when knowledge from one scientific discipline is applied to another Globalisation is a major driver for open innovation processes, not only because it means more intense and global competition but also because it creates a more global landscape for innovation (OECD2008: 27). A 2006 study shows that there was a 10% increase in the probability of being a winning solver if the broadcast problem was assessed to be completely outside their field of expertise. Consistent with our finding about specialization, we found a marginally significant negative correlation between the number of scientific interests expressed and the probability of being a winning solver. Thus, being more specialized (expressing fewer scientific interests) resulted in a higher probability of creating a winning solution. Collective intelligence: a million heads are better than one (other litterature: 2 to 5 are better together, if more the just equal to its sum) ‘wisdom of crowds theory’ effect : aggregation of individual solutions is better that the parts (better than collaboration), reaches expert degree Available Talent not easy to find can be reached. generating better results, and undertaking problems that would have been too difficult to solve internally Feedback: by listening to the crowd, you have first-hand insight on their desires. See followers! Followers: Crowd loyalty. The community may feel a brand-building kinship with the crowdsourcing organization, which is the result of an earned sense of ownership through contribution and collaboration. Doritos "Crash the Super Bowl" campaign is one example of a fully integrated and successful program. Doritos fans created their own advertisements for the chance to win a trip to the game, $25,000 cash, and the fame of creating a Super Bowl advertisement. In 2011, four consumer-created ads for Doritos and Pepsi Max ranked among the top ten in the USA TODAY Ad Meter. Crowdsourcing for brands doesn’t always work. Levia, a medical device marketer, failed to generate crowdsourcing activity with a similar promotion. They lacked the prerequisites of a crowd, sufficient motivation, and a reasonable expectation of work effort. business agility (for ex. peack-demands) Prototyping, Pilots, low opportunity costs. What is clear now is that most companies have ready access to crowdsourcing across a wide set of functional areas, to the extent that it's often the easiest thing for them to try before going the more expensive outsourcing route. Businesses can use crowdsourcing to experiment and find out what they can do. This has implications for business agility as well that can't be ignored.
- Drawbacks Unverified quality! You don’t have CVs to check qualification of workers, nor know the person. Anyone can answer, even jokers or dishonests participants. Quality has to be checked, information verified. When Facebook did its localization program in 2008, it encountered both these criticisms. Too many answers! Often the challenge is that the contributions from the community can be large. This richness and variety is wonderful to have but sometimes requires considerable review to find the best one. Such a swamping of inputs led to early problems, such as when Amazon's Mechanical Turk was used to try to find Steve Fosset's plane using an army of 50,000 volunteers a few years ago. Newer more mature crowdsourcing services now have filters and controls, such as Kluster's ability to more readily tune the "relative influence" of various types of participants. No standards! Each contributor answer their own way. This problem led to a new set of tools to define formats for the answers. But still when it’s an open question, the style of communication varies from participant to participant. no organisation! When a bigger issue must be solved (a complex question is posed, or some steps have a predefined order in wich they have to be done…) , reconstruct from partial answers is a puzzle! It could be bettered by a moderator, but then you have the problems of confidence reported to the moderator, and possible bias (that may discard a brilliant idea)… Hierarchical task management solves the problem Added costs to bring a project to an acceptable conclusion Often the challenge is that the contributions from the community can be large. This richness and variety is wonderful to have but sometimes requires considerable review to find the best one. Some reports have focused on the negative effects of crowdsourcing on business owners, particularly in regard to how a crowdsourced project can sometimes end up costing a business more than a traditionally outsourced project. Project may fail due to lack of monetary motivation, Below-market wages, too few participants, lower quality of work, lack of personal interest in the project. see other More Drawbacks Global language barriers. Different laws in each country: adds complexity No written contracts, so no possibility of non-disclosure agreements. Hard to maintain a long term working relationship with workers. Difficulty managing a large-scale, crowdsourced project. Can be targeted by malicious work efforts. Lack of guaranteed investment, thus hard to convince stakeholders
- Intelligent Crowdsourcing There are several reasons why you’ll want to avoid that everyone contributes in this kind of case: First of all, quantity: if you get too many ideas, it will immobilize too many people for too long to evaluate them. Second, quality: if contributors are not targeted, you will get many ideas that are besides the point, which will reduce the overall credibility. On the contributors’ side, a lot of people will have contributed without being rewarded and you will make many people unhappy. If you reach out actively for solvers and solicit the wrong people too often, they’ll end up redflagging your organization. Other Point: if solutions to R&D and innovation challenges can be found, their value for an organization will be higher if the competitors don’t have access to the same solutions. In an overt intiative like Dell’s everyone can follow the ideas with growing customer support: Asus can decide to implement an idea before Dell does. So the solution is Intelligent Crowdsourcing: Some companies use netnography to identify relevant crowds. This approach works best for lead-user identification and approaches that want to integrate innovation and consumer research MercMob is a relatively new entrant to the crowdsourcing field and brings with it a unique spin on collaborative project management.
- AdSafe crowdsourcing experience: To prevent Brand Damage Online: Detect pages that discuss swine flu – Pharmaceutical firm had drug “treating” (off-label) swine flu – FDA prohibited pharmaceuticals to display drug ad in pages about swine flu Two days to comply! • Big fast-food chain does not want ad to appear: – In pages that discuss the brand (99% negative sentiment) – In pages discussing obesity
- Crowdsourcing is part of Globalization, you have to be ready for it. If a community exists that does the same as your company, you’ll better identify your strongest points, what differentiates you from it. If there’s no community yet, but your work could be done by one cheaper than by you, or better, again… There are still issues to be treated: How will taxes evolve? Global taxes as global digital rights? Aggregators with ‘employers’ without frontiers, no much formal contracts, what about workers rights? Will there be a middle class in the networked economy, or only heads (famous experts) & tails (the rest of us)? Reputation Systems (It is showing a need of trustability, and for now that need is being implemented through reputation systems) Intelligent Crowdsourcing Reputation all over…Good or Bad? No way of hiding something: under glass living effect stress added? Frontiers don't mean the same as before, are not so strong, and putting a community together transnational, means socio-economic interests don't stop on frontiers. So soon you will have to take into account this competition, at a very cheap price (free sometimes), with knowledge put there for everybody to access and profit on it. If your business model is based on propietary things or knowledge, or your competitivity based on price, you'll better look at this that will be global and is moving fast to a lot of domains. Evolution is not stoppable, and it may be great, as we see that big humanity problems are best tackled this way, openness facilitates innovation, lowering costs, and time of implementation...what we need to solve the climate change problem. So business will change, and the best thing to do is be prepare for it. Move towards: Openness of Data and KnowledgeCollaboration Innovation Objective: To create the infrastructure of the future economy. There will be business on creating value over the public energy grid, data and knowledge infrastructure.