Crowdsourcing is a process where an individual or an organization utilize the talent pool present over the Internet to accomplish their task(s). These platforms offer numerous advantages such as reduced cost, better quality, and lower task completion time. To execute tasks efficiently, with the worker pool available on the platform, task posters rely on the reputation managed and maintained by the platform. Usually, reputation management system works on ratings provided by the task posters. Such reputation systems are susceptible to several attacks as users or the platform owners, with malicious intents, can jeopardize the reputation system with fake reputations. A blockchain based approach for managing various crowdsourcing steps provides a promising direction to manage reputation system. We propose a crowdsourcing platform where each step of crowdsourcing process is managed as transactions in Blockchain. This helps in establishing better trust in the platform users and addresses various attacks which are possible on a centralized crowdsourcing platform. We have built the proposed platform on the Ethereum framework. Our system utilizes IOTA's consensus mechanism which reduces the cost for task evaluation to almost zero.
22. Research Aim
➢ Modeling crowdsourcing platform with
Blockchain based design principles..
➢ Propose submission evaluation strategy that is
insusceptible to malicious users so as to build
a robust and immutable reputation system for
our platform.
22
23. Outline
➢ Proposed Architecture
➢ Steps in Crowdsourcing Cycle
➢ Evaluating Submission and Updating
Reputation
➢ Experiments and Robustness Against Attacks
➢ Conclusion
23
24. Proposed Crowdsourcing Platform Overview
24
Post Task Task Search Select workerApply for task
Perform task
Submit solutionPerform review and send feedback
25. Application Layer
Storage Layer
Ethereum Layer
IPFS
Front End-Framework
Web3.js
User
Contract
Task
Contract
Submission
Contract
Agreement
Contract
Evaluation
Contract
Meta Mask
Proposed Architecture
Off
Blockchain
On
Blockchain
25
26. User Registration
1. Profile Info
2. Hash
3. Create Worker/
Task poster
INPUT : Hash , Public key
User
User
Contract
4. User Registration
transaction mined
5. Registration Successful
OUTPUT: Worker / Task Poster
26
27. Post Task
2. Hash
1. Task Info Task Poster
3. Post Task ( Hash,
Reward, Skills, Title)
4. Task transaction
mined
Task
Contract
Task
27
31. Outline
➢ Research Motivation
➢ Research Aim
➢ Proposed Architecture
➢ Steps in Crowdsourcing Cycle
➢ Evaluating Submission and Updating
Reputation
➢ Experiments and Robustness Against Attacks
➢ Conclusion
31
32. Terminology
➢ Given a Worker w who submitted the solution s for task t
➢ Let E = {e1
, e2
, … en
}be the set of evaluators for submission s
➢ {ci
, qi
} rating given by ei
➢ ri
reputation of ei
➢ weightq
and weightc
: weights assigned to quality score and
completeness score respectively.
➢ Let P denote the set of potential evaluators.
➢ evalSet (t1
,t2
,…, tf
) tasks to be evaluated by w corresponding to t.
32
33. Potential Evaluators
Denoted by P, Workers whose:
➢ Submission is not yet evaluated.
➢ Reputation > average reputation of workers on
platform.
33
34. Selecting Evaluators
34
Let |E| be number of evaluators required for
evaluating submission s
Divide P into |E| groups based on reputation
scores
Randomly choose one evaluator from each
interval
36. Consensus
➢ Outlier: if score given by evaluator not within 1.25 standard
deviation away from mean. outliers
set of outliers for
submission s.
➢ Consensus majority evaluators are not outliers.
➢ Reviews of only those evaluators who form consensus are
considered for computing reputation score of the worker.
After removing outlier let,
Cm
mean completeness score.
Qm
mean quality score.
36
37. Updating Reputation
Two ways through which the reputation score of
worker is updated:
➢ Submission and evaluation
➢ Only evaluation
37
42. Demo
42
➢ Post Task
➢ Task Application
➢ Select worker and create agreement
➢ Worker accepts the agreement and start working
on it
➢ Submits hash and evaluate task
➢ Get evaluators and send them encrypted solution
➢ Evaluators send review and reputation is
updated
43. Outline
➢ Research Motivation
➢ Research Aim
➢ Proposed Architecture
➢ Unboxing the Steps
➢ Evaluating Submission and Updating
Reputation
➢ Experiments and Robustness Against Attacks
➢ Conclusion
43
44. Various Attacks
➢ Unfair rating attack: is when the rater is biased
towards worker and does not give a truthful opinion
about him.
➢ Reciprocity: when worker reciprocates negatively for a
negative review that he receives.
➢ Ballot stuffing : when the worker tries to increase its
own reputation.
➢ Collusion: when a group of workers try to collude
together to improve their own reputation or decrease
the reputation of others.
➢ Sybil attack:where a malicious worker tries to create
multiple identities over the platform to gain influence
45
45. Robustness Against Various Attacks
➢ Unfair rating attack:
➢ Consensus among evaluators and final rating is being
normalized using median.
➢ Reputation of evaluator is decreased if he is an outlier.
➢ Reciprocity:
➢ Outliers removed.
➢ Ballot stuffing and Collusion
➢ Evaluators pseudo randomly chosen.
➢ A trail of evaluation transactions.
➢ Reputation value altered only by contract code.
➢ Sybil Attack
46
47. Conclusion
➢ How each step of crowdsourcing could be managed as
transaction on Blockchain.
➢ Address various attacks possible on centralized
Crowdsourcing platform.
➢ Establishing better trust on worker’s of the platform.
➢ Reduce cost for task poster as well as worker.
47
48. Challenges, Limitation and Future Work
➢ Time delay between creating a transaction and
its confirmation - Sharding.
➢ Computation cost on Ethereum network -
Computational oracles.
➢ Task poster having a say in the submission
evaluation and maintaining its reputation
score.
➢ Encrypting evaluation initially until all
evaluators have sent. 48
49. Acknowledgement
➢ Committee members
➢ Abhinav, Accenture Technology Labs
➢ Indira, Shubham, Simran, Shwetanshu
➢ Members of Precog family
➢ Family and friends
49
50. References
➢ Li, M., Weng, J., Yang, A., Lu, W., Zhang, Y., Hou, L., Liu, J.N., Xiang, Y. and
Deng, R.H., 2017. CrowdBC: A Blockchain-based Decentralized Framework
for Crowdsourcing. IACR Cryptol. ePrint Arch., Univ. California, Santa Barbara,
Santa Barbara, CA, USA, Tech. Rep, 444, p.2017.
➢ Dennis, R. and Owenson, G.H., 2016, February. Rep on the block: A next
generation reputation system based on the blockchain. In 10th
International Conference for Internet Technology and Secured Transactions
(ICITST). IEEE.
➢ Tavakolifard, M. and Almeroth, K.C., 2012. A taxonomy to express open
challenges in trust and reputation systems. Journal of
Communications, 7(7), pp.538-551.
➢ Jøsang, A. and Golbeck, J., 2009, September. Challenges for robust trust 50