SlideShare ist ein Scribd-Unternehmen logo
1 von 51
Downloaden Sie, um offline zu lesen
WorkerRep: Building Trust on Crowdsourcing
Platform Using Blockchain
Gurpriya Kaur Bhatia (MT16021)
Advisors:
Dr. Ponnurangam Kumaraguru(IIITD)
Dr. Alpana Dubey(Accenture Technology Labs)
LinkedIn/in/gurpriya-kaur
-562ab89a/
@gurpriya06 gurpriya.bhatia.9
Thesis Committee
➢ Dr. Arun Balaji Buduru, IIITD
➢ Mr. Vikrant Kaulgud, Accenture Technology Labs
➢ Dr. Alpana Dubey, Accenture Technology Labs (Co - Chair)
➢ Dr. Ponnurangam Kumaraguru, IIITD (Chair)
2
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
3
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Task
4
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Worker
Task
5
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Task Poster
Worker
Task
6
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Task Poster
Worker
Platform
Task
7
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task
8
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task Task Search
and Apply
9
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task Task Search
and Apply
Select worker
10
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task Task Search
and Apply
Select worker
Perform task
11
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task Task Search
and Apply
Select worker
Submit solution
12
Perform task
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post task Task search
and apply
Select worker
Submit solutionPerform review
13
Perform task
Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task Task Search
and Apply
Select worker
Submit solutionPerform review and send feedback
14
Perform task
Blockchain
15
BankUsers Transaction
Ledger Decentralized Distributed Miner Type
Ethereum
16
Blockchain - based distributed computing platform
Ether Smart Contract Ethereum
Ethereum
17
User Ethereum Wallet
Smart Contract
9f4195924cb79195ef9b07e
04e641a86e2eaa98340b40
15e8884a9f4ba4a6ec0
d59b04abcbeaf11aeebb6325e0
393bfdd1a016a5ae6b58c31a67
f1aa95030be6b02ae8577e8cbb
e17c4577272fa3055ea2d53917
c07e43d9105e4d7e5f65318e
0x0ca7B9c0E24c
Ac746F29BA984e
C8Ba79856c3a83
Private Key
Public Key
Public Address
DISTRIBUTED STORAGE
18
InterPlanetary File System
Storage cost
1. Data
2. Hash
QmfJgmCHkame9yxbNG7VwrzYJf8uZr
dbHHcor6oNPBYQou
User
Motivation
19
Motivation
20
Why Blockchain?
21
Research Aim
➢ Modeling crowdsourcing platform with
Blockchain based design principles..
➢ Propose submission evaluation strategy that is
insusceptible to malicious users so as to build
a robust and immutable reputation system for
our platform.
22
Outline
➢ Proposed Architecture
➢ Steps in Crowdsourcing Cycle
➢ Evaluating Submission and Updating
Reputation
➢ Experiments and Robustness Against Attacks
➢ Conclusion
23
Proposed Crowdsourcing Platform Overview
24
Post Task Task Search Select workerApply for task
Perform task
Submit solutionPerform review and send feedback
Application Layer
Storage Layer
Ethereum Layer
IPFS
Front End-Framework
Web3.js
User
Contract
Task
Contract
Submission
Contract
Agreement
Contract
Evaluation
Contract
Meta Mask
Proposed Architecture
Off
Blockchain
On
Blockchain
25
User Registration
1. Profile Info
2. Hash
3. Create Worker/
Task poster
INPUT : Hash , Public key
User
User
Contract
4. User Registration
transaction mined
5. Registration Successful
OUTPUT: Worker / Task Poster
26
Post Task
2. Hash
1. Task Info Task Poster
3. Post Task ( Hash,
Reward, Skills, Title)
4. Task transaction
mined
Task
Contract
Task
27
Task Search and Task Registration
28
Tasks ListWorker
1. Task
search
Task
Contract
2. Task register
(taskID)
Task Poster Agreement
1. Create
2. Informs worker
3. Accept
6. Submit hash
Worker
Task Assignment and Submission
Off - Blockchain
hashing
4. Submission
5. Hash
Evaluation
Contract
7. Perform evaluations
29
Agreement
4. Hash
8. Update Reputation
Worker
Evaluator N
Evaluator 1
Task Assignment, Evaluation and Updating Reputation
1. Submission
2. Encryption Encryption
algorithm
3. Store
encrypted
submission
5. Hash
6. Assign evaluators
7. Review
30
Outline
➢ Research Motivation
➢ Research Aim
➢ Proposed Architecture
➢ Steps in Crowdsourcing Cycle
➢ Evaluating Submission and Updating
Reputation
➢ Experiments and Robustness Against Attacks
➢ Conclusion
31
Terminology
➢ Given a Worker w who submitted the solution s for task t
➢ Let E = {e1
, e2
, … en
}be the set of evaluators for submission s
➢ {ci
, qi
} rating given by ei
➢ ri
reputation of ei
➢ weightq
and weightc
: weights assigned to quality score and
completeness score respectively.
➢ Let P denote the set of potential evaluators.
➢ evalSet (t1
,t2
,…, tf
) tasks to be evaluated by w corresponding to t.
32
Potential Evaluators
Denoted by P, Workers whose:
➢ Submission is not yet evaluated.
➢ Reputation > average reputation of workers on
platform.
33
Selecting Evaluators
34
Let |E| be number of evaluators required for
evaluating submission s
Divide P into |E| groups based on reputation
scores
Randomly choose one evaluator from each
interval
Review
Each evaluator gives 2 types of reviews:
➢ Textual
➢ Feedback, reason supporting rating.
➢ Rating (1-100)
➢ Quality(qi
)
➢ Completeness(ci
)
35
Consensus
➢ Outlier: if score given by evaluator not within 1.25 standard
deviation away from mean. outliers
set of outliers for
submission s.
➢ Consensus majority evaluators are not outliers.
➢ Reviews of only those evaluators who form consensus are
considered for computing reputation score of the worker.
After removing outlier let,
Cm
mean completeness score.
Qm
mean quality score.
36
Updating Reputation
Two ways through which the reputation score of
worker is updated:
➢ Submission and evaluation
➢ Only evaluation
37
Updating Reputation Through Submission and
Evaluation
◆
38
Updating Reputation Through Submission and
Evaluation
◆
39
Updating Reputation Through Evaluation
◆
40
Reward
◆
41
Demo
42
➢ Post Task
➢ Task Application
➢ Select worker and create agreement
➢ Worker accepts the agreement and start working
on it
➢ Submits hash and evaluate task
➢ Get evaluators and send them encrypted solution
➢ Evaluators send review and reputation is
updated
Outline
➢ Research Motivation
➢ Research Aim
➢ Proposed Architecture
➢ Unboxing the Steps
➢ Evaluating Submission and Updating
Reputation
➢ Experiments and Robustness Against Attacks
➢ Conclusion
43
Various Attacks
➢ Unfair rating attack: is when the rater is biased
towards worker and does not give a truthful opinion
about him.
➢ Reciprocity: when worker reciprocates negatively for a
negative review that he receives.
➢ Ballot stuffing : when the worker tries to increase its
own reputation.
➢ Collusion: when a group of workers try to collude
together to improve their own reputation or decrease
the reputation of others.
➢ Sybil attack:where a malicious worker tries to create
multiple identities over the platform to gain influence
45
Robustness Against Various Attacks
➢ Unfair rating attack:
➢ Consensus among evaluators and final rating is being
normalized using median.
➢ Reputation of evaluator is decreased if he is an outlier.
➢ Reciprocity:
➢ Outliers removed.
➢ Ballot stuffing and Collusion
➢ Evaluators pseudo randomly chosen.
➢ A trail of evaluation transactions.
➢ Reputation value altered only by contract code.
➢ Sybil Attack
46
Experimental Results
Reward
Amount($)
User type AMT($) Upwork($) WorkerRep($)
50 Worker - 10 0.436
Task Poster 10 1.375 0.195
100 Worker - 20 0.436
Task Poster 20 2.75 0.195
1000 Worker - 200 0.436
Task Poster 200 27.5 0.195
47
Conclusion
➢ How each step of crowdsourcing could be managed as
transaction on Blockchain.
➢ Address various attacks possible on centralized
Crowdsourcing platform.
➢ Establishing better trust on worker’s of the platform.
➢ Reduce cost for task poster as well as worker.
47
Challenges, Limitation and Future Work
➢ Time delay between creating a transaction and
its confirmation - Sharding.
➢ Computation cost on Ethereum network -
Computational oracles.
➢ Task poster having a say in the submission
evaluation and maintaining its reputation
score.
➢ Encrypting evaluation initially until all
evaluators have sent. 48
Acknowledgement
➢ Committee members
➢ Abhinav, Accenture Technology Labs
➢ Indira, Shubham, Simran, Shwetanshu
➢ Members of Precog family
➢ Family and friends
49
References
➢ Li, M., Weng, J., Yang, A., Lu, W., Zhang, Y., Hou, L., Liu, J.N., Xiang, Y. and
Deng, R.H., 2017. CrowdBC: A Blockchain-based Decentralized Framework
for Crowdsourcing. IACR Cryptol. ePrint Arch., Univ. California, Santa Barbara,
Santa Barbara, CA, USA, Tech. Rep, 444, p.2017.
➢ Dennis, R. and Owenson, G.H., 2016, February. Rep on the block: A next
generation reputation system based on the blockchain. In 10th
International Conference for Internet Technology and Secured Transactions
(ICITST). IEEE.
➢ Tavakolifard, M. and Almeroth, K.C., 2012. A taxonomy to express open
challenges in trust and reputation systems. Journal of
Communications, 7(7), pp.538-551.
➢ Jøsang, A. and Golbeck, J., 2009, September. Challenges for robust trust 50
Thanks!
gurpriya16021@iiitd.ac.in
@gurpriya06
51

Más contenido relacionado

Ähnlich wie WorkerRep: Building Trust on Crowdsourcing Platform Using Blockchain

The PMO Company journey to Agile & Scrum certification Session 1
The PMO Company journey to Agile & Scrum certification Session 1The PMO Company journey to Agile & Scrum certification Session 1
The PMO Company journey to Agile & Scrum certification Session 1Getjan Lammers
 
Pawan Kumar_Resume
Pawan Kumar_ResumePawan Kumar_Resume
Pawan Kumar_ResumePawan Kumar
 
Agile Project Management Workshop.pptx
Agile Project Management Workshop.pptxAgile Project Management Workshop.pptx
Agile Project Management Workshop.pptxAbhishek Prasoon
 
OWASP AppSec EU 2016 - Security Project Management - How to be Agile in Secu...
OWASP AppSec EU 2016 - Security Project Management -  How to be Agile in Secu...OWASP AppSec EU 2016 - Security Project Management -  How to be Agile in Secu...
OWASP AppSec EU 2016 - Security Project Management - How to be Agile in Secu...Simone Onofri
 
Agile Truths and Misconceptions
Agile Truths and MisconceptionsAgile Truths and Misconceptions
Agile Truths and MisconceptionsRichard Cheng
 
Manual testing good notes
Manual testing good notesManual testing good notes
Manual testing good notesdkns0906
 
Simran confidentiality protection in crowdsourcing
Simran confidentiality protection in crowdsourcingSimran confidentiality protection in crowdsourcing
Simran confidentiality protection in crowdsourcingIIIT Hyderabad
 
Rapid Solutions with Salesforce Flows
Rapid Solutions with Salesforce FlowsRapid Solutions with Salesforce Flows
Rapid Solutions with Salesforce FlowstheCodery
 
Der Test Manager ist tot - lang lebe der Test Master
Der Test Manager ist tot - lang lebe der Test MasterDer Test Manager ist tot - lang lebe der Test Master
Der Test Manager ist tot - lang lebe der Test MasterSwissQ Consulting AG
 
Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery M...
Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery M...Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery M...
Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery M...Simon Storm
 
Transferring Software Testing Tools to Practice
Transferring Software Testing Tools to PracticeTransferring Software Testing Tools to Practice
Transferring Software Testing Tools to PracticeTao Xie
 
Estimation - web software development estimation DrupalCon and DrupalCamp pre...
Estimation - web software development estimation DrupalCon and DrupalCamp pre...Estimation - web software development estimation DrupalCon and DrupalCamp pre...
Estimation - web software development estimation DrupalCon and DrupalCamp pre...Andy Kucharski
 
Michael Bolton - Two Futures of Software Testing
Michael Bolton - Two Futures of Software TestingMichael Bolton - Two Futures of Software Testing
Michael Bolton - Two Futures of Software TestingTEST Huddle
 
2015 drupalcampcebu estimation_jrf
2015 drupalcampcebu estimation_jrf2015 drupalcampcebu estimation_jrf
2015 drupalcampcebu estimation_jrfJohnnie Fox
 
Software Project Management Presentation Final
Software Project Management Presentation FinalSoftware Project Management Presentation Final
Software Project Management Presentation FinalMinhas Kamal
 
Lecture 5 -6(CSC205).pptx jsksnxbbxjxksnsnz
Lecture 5 -6(CSC205).pptx jsksnxbbxjxksnsnzLecture 5 -6(CSC205).pptx jsksnxbbxjxksnsnz
Lecture 5 -6(CSC205).pptx jsksnxbbxjxksnsnzAhmadSajjad34
 

Ähnlich wie WorkerRep: Building Trust on Crowdsourcing Platform Using Blockchain (20)

The PMO Company journey to Agile & Scrum certification Session 1
The PMO Company journey to Agile & Scrum certification Session 1The PMO Company journey to Agile & Scrum certification Session 1
The PMO Company journey to Agile & Scrum certification Session 1
 
Pawan Kumar_Resume
Pawan Kumar_ResumePawan Kumar_Resume
Pawan Kumar_Resume
 
Agile Project Management Workshop.pptx
Agile Project Management Workshop.pptxAgile Project Management Workshop.pptx
Agile Project Management Workshop.pptx
 
RanjanKumarPusty__Resume
RanjanKumarPusty__ResumeRanjanKumarPusty__Resume
RanjanKumarPusty__Resume
 
OWASP AppSec EU 2016 - Security Project Management - How to be Agile in Secu...
OWASP AppSec EU 2016 - Security Project Management -  How to be Agile in Secu...OWASP AppSec EU 2016 - Security Project Management -  How to be Agile in Secu...
OWASP AppSec EU 2016 - Security Project Management - How to be Agile in Secu...
 
Agile Truths and Misconceptions
Agile Truths and MisconceptionsAgile Truths and Misconceptions
Agile Truths and Misconceptions
 
Manual testing good notes
Manual testing good notesManual testing good notes
Manual testing good notes
 
Simran confidentiality protection in crowdsourcing
Simran confidentiality protection in crowdsourcingSimran confidentiality protection in crowdsourcing
Simran confidentiality protection in crowdsourcing
 
MagSlideDeck_17_External_v1.0
MagSlideDeck_17_External_v1.0MagSlideDeck_17_External_v1.0
MagSlideDeck_17_External_v1.0
 
Rapid Solutions with Salesforce Flows
Rapid Solutions with Salesforce FlowsRapid Solutions with Salesforce Flows
Rapid Solutions with Salesforce Flows
 
Der Test Manager ist tot - lang lebe der Test Master
Der Test Manager ist tot - lang lebe der Test MasterDer Test Manager ist tot - lang lebe der Test Master
Der Test Manager ist tot - lang lebe der Test Master
 
Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery M...
Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery M...Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery M...
Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery M...
 
Prasanth_Pendam_QA_9.5 Years
Prasanth_Pendam_QA_9.5 YearsPrasanth_Pendam_QA_9.5 Years
Prasanth_Pendam_QA_9.5 Years
 
Transferring Software Testing Tools to Practice
Transferring Software Testing Tools to PracticeTransferring Software Testing Tools to Practice
Transferring Software Testing Tools to Practice
 
Clustering-based Location Recommendation(Collaborative Filtering)
Clustering-based Location Recommendation(Collaborative Filtering)Clustering-based Location Recommendation(Collaborative Filtering)
Clustering-based Location Recommendation(Collaborative Filtering)
 
Estimation - web software development estimation DrupalCon and DrupalCamp pre...
Estimation - web software development estimation DrupalCon and DrupalCamp pre...Estimation - web software development estimation DrupalCon and DrupalCamp pre...
Estimation - web software development estimation DrupalCon and DrupalCamp pre...
 
Michael Bolton - Two Futures of Software Testing
Michael Bolton - Two Futures of Software TestingMichael Bolton - Two Futures of Software Testing
Michael Bolton - Two Futures of Software Testing
 
2015 drupalcampcebu estimation_jrf
2015 drupalcampcebu estimation_jrf2015 drupalcampcebu estimation_jrf
2015 drupalcampcebu estimation_jrf
 
Software Project Management Presentation Final
Software Project Management Presentation FinalSoftware Project Management Presentation Final
Software Project Management Presentation Final
 
Lecture 5 -6(CSC205).pptx jsksnxbbxjxksnsnz
Lecture 5 -6(CSC205).pptx jsksnxbbxjxksnsnzLecture 5 -6(CSC205).pptx jsksnxbbxjxksnsnz
Lecture 5 -6(CSC205).pptx jsksnxbbxjxksnsnz
 

Mehr von IIIT Hyderabad

Responsible & Safe AI Systems at ACM India ROCS at IIT Bombay
Responsible & Safe AI Systems at ACM India ROCS at IIT BombayResponsible & Safe AI Systems at ACM India ROCS at IIT Bombay
Responsible & Safe AI Systems at ACM India ROCS at IIT BombayIIIT Hyderabad
 
International Collaboration: Experiences, Challenges, Success stories
International Collaboration: Experiences, Challenges, Success storiesInternational Collaboration: Experiences, Challenges, Success stories
International Collaboration: Experiences, Challenges, Success storiesIIIT Hyderabad
 
Responsible & Safe AI: #LegalBias #Inconsistency #BiasinLLMs #MultiModalBias
Responsible & Safe AI: #LegalBias #Inconsistency #BiasinLLMs #MultiModalBiasResponsible & Safe AI: #LegalBias #Inconsistency #BiasinLLMs #MultiModalBias
Responsible & Safe AI: #LegalBias #Inconsistency #BiasinLLMs #MultiModalBiasIIIT Hyderabad
 
Identify, Inspect and Intervene Multimodal Fake News
Identify, Inspect and Intervene Multimodal Fake NewsIdentify, Inspect and Intervene Multimodal Fake News
Identify, Inspect and Intervene Multimodal Fake NewsIIIT Hyderabad
 
Data Science for Social Good: #MentalHealth #CodeMix #LegalNLP #AISafety
Data Science for Social Good: #MentalHealth #CodeMix #LegalNLP #AISafetyData Science for Social Good: #MentalHealth #CodeMix #LegalNLP #AISafety
Data Science for Social Good: #MentalHealth #CodeMix #LegalNLP #AISafetyIIIT Hyderabad
 
It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...IIIT Hyderabad
 
Beyond the Surface: A Computational Exploration of Linguistic Ambiguity
Beyond the Surface: A Computational Exploration of Linguistic AmbiguityBeyond the Surface: A Computational Exploration of Linguistic Ambiguity
Beyond the Surface: A Computational Exploration of Linguistic AmbiguityIIIT Hyderabad
 
Data Science for Social Good: #LegalNLP #AlgorithmicBias...
Data Science for Social Good:                      #LegalNLP #AlgorithmicBias...Data Science for Social Good:                      #LegalNLP #AlgorithmicBias...
Data Science for Social Good: #LegalNLP #AlgorithmicBias...IIIT Hyderabad
 
How to Write a (Good) Research Paper
How to Write a (Good) Research Paper How to Write a (Good) Research Paper
How to Write a (Good) Research Paper IIIT Hyderabad
 
Data Science for Social Good: #LegalNLP #AlgorithmicBias
Data Science for Social Good: #LegalNLP #AlgorithmicBiasData Science for Social Good: #LegalNLP #AlgorithmicBias
Data Science for Social Good: #LegalNLP #AlgorithmicBiasIIIT Hyderabad
 
Social Computing Research in India
Social Computing Research in IndiaSocial Computing Research in India
Social Computing Research in IndiaIIIT Hyderabad
 
Social Computing Research in India
Social Computing Research in IndiaSocial Computing Research in India
Social Computing Research in IndiaIIIT Hyderabad
 
Modeling Online User Interactions and their Offline effects on Socio-Technica...
Modeling Online User Interactions and their Offline effects on Socio-Technica...Modeling Online User Interactions and their Offline effects on Socio-Technica...
Modeling Online User Interactions and their Offline effects on Socio-Technica...IIIT Hyderabad
 
Privacy. Winter School on “Topics in Digital Trust”. IIT Bombay
Privacy. Winter School on “Topics in Digital Trust”. IIT BombayPrivacy. Winter School on “Topics in Digital Trust”. IIT Bombay
Privacy. Winter School on “Topics in Digital Trust”. IIT BombayIIIT Hyderabad
 
It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...IIIT Hyderabad
 
It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...IIIT Hyderabad
 
Leveraging Social Media for Financial Advice
Leveraging Social Media for Financial AdviceLeveraging Social Media for Financial Advice
Leveraging Social Media for Financial AdviceIIIT Hyderabad
 
Development of Stress Induction and Detection System to Study its Effect on B...
Development of Stress Induction and Detection System to Study its Effect on B...Development of Stress Induction and Detection System to Study its Effect on B...
Development of Stress Induction and Detection System to Study its Effect on B...IIIT Hyderabad
 
A Framework for Automatic Question Answering in Indian Languages
A Framework for Automatic Question Answering in Indian LanguagesA Framework for Automatic Question Answering in Indian Languages
A Framework for Automatic Question Answering in Indian LanguagesIIIT Hyderabad
 

Mehr von IIIT Hyderabad (20)

Responsible & Safe AI Systems at ACM India ROCS at IIT Bombay
Responsible & Safe AI Systems at ACM India ROCS at IIT BombayResponsible & Safe AI Systems at ACM India ROCS at IIT Bombay
Responsible & Safe AI Systems at ACM India ROCS at IIT Bombay
 
International Collaboration: Experiences, Challenges, Success stories
International Collaboration: Experiences, Challenges, Success storiesInternational Collaboration: Experiences, Challenges, Success stories
International Collaboration: Experiences, Challenges, Success stories
 
Responsible & Safe AI: #LegalBias #Inconsistency #BiasinLLMs #MultiModalBias
Responsible & Safe AI: #LegalBias #Inconsistency #BiasinLLMs #MultiModalBiasResponsible & Safe AI: #LegalBias #Inconsistency #BiasinLLMs #MultiModalBias
Responsible & Safe AI: #LegalBias #Inconsistency #BiasinLLMs #MultiModalBias
 
Identify, Inspect and Intervene Multimodal Fake News
Identify, Inspect and Intervene Multimodal Fake NewsIdentify, Inspect and Intervene Multimodal Fake News
Identify, Inspect and Intervene Multimodal Fake News
 
#ChatGPT #ResponsibleAI
#ChatGPT #ResponsibleAI#ChatGPT #ResponsibleAI
#ChatGPT #ResponsibleAI
 
Data Science for Social Good: #MentalHealth #CodeMix #LegalNLP #AISafety
Data Science for Social Good: #MentalHealth #CodeMix #LegalNLP #AISafetyData Science for Social Good: #MentalHealth #CodeMix #LegalNLP #AISafety
Data Science for Social Good: #MentalHealth #CodeMix #LegalNLP #AISafety
 
It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...
 
Beyond the Surface: A Computational Exploration of Linguistic Ambiguity
Beyond the Surface: A Computational Exploration of Linguistic AmbiguityBeyond the Surface: A Computational Exploration of Linguistic Ambiguity
Beyond the Surface: A Computational Exploration of Linguistic Ambiguity
 
Data Science for Social Good: #LegalNLP #AlgorithmicBias...
Data Science for Social Good:                      #LegalNLP #AlgorithmicBias...Data Science for Social Good:                      #LegalNLP #AlgorithmicBias...
Data Science for Social Good: #LegalNLP #AlgorithmicBias...
 
How to Write a (Good) Research Paper
How to Write a (Good) Research Paper How to Write a (Good) Research Paper
How to Write a (Good) Research Paper
 
Data Science for Social Good: #LegalNLP #AlgorithmicBias
Data Science for Social Good: #LegalNLP #AlgorithmicBiasData Science for Social Good: #LegalNLP #AlgorithmicBias
Data Science for Social Good: #LegalNLP #AlgorithmicBias
 
Social Computing Research in India
Social Computing Research in IndiaSocial Computing Research in India
Social Computing Research in India
 
Social Computing Research in India
Social Computing Research in IndiaSocial Computing Research in India
Social Computing Research in India
 
Modeling Online User Interactions and their Offline effects on Socio-Technica...
Modeling Online User Interactions and their Offline effects on Socio-Technica...Modeling Online User Interactions and their Offline effects on Socio-Technica...
Modeling Online User Interactions and their Offline effects on Socio-Technica...
 
Privacy. Winter School on “Topics in Digital Trust”. IIT Bombay
Privacy. Winter School on “Topics in Digital Trust”. IIT BombayPrivacy. Winter School on “Topics in Digital Trust”. IIT Bombay
Privacy. Winter School on “Topics in Digital Trust”. IIT Bombay
 
It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...
 
It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...It is our choices, Harry, that show what we truly are, far more than our abil...
It is our choices, Harry, that show what we truly are, far more than our abil...
 
Leveraging Social Media for Financial Advice
Leveraging Social Media for Financial AdviceLeveraging Social Media for Financial Advice
Leveraging Social Media for Financial Advice
 
Development of Stress Induction and Detection System to Study its Effect on B...
Development of Stress Induction and Detection System to Study its Effect on B...Development of Stress Induction and Detection System to Study its Effect on B...
Development of Stress Induction and Detection System to Study its Effect on B...
 
A Framework for Automatic Question Answering in Indian Languages
A Framework for Automatic Question Answering in Indian LanguagesA Framework for Automatic Question Answering in Indian Languages
A Framework for Automatic Question Answering in Indian Languages
 

Último

Searching and Sorting Algorithms
Searching and Sorting AlgorithmsSearching and Sorting Algorithms
Searching and Sorting AlgorithmsAshutosh Satapathy
 
The Art of Cloud Native Defense on Kubernetes
The Art of Cloud Native Defense on KubernetesThe Art of Cloud Native Defense on Kubernetes
The Art of Cloud Native Defense on KubernetesJacopo Nardiello
 
Introduction to Data Structures .
Introduction to Data Structures        .Introduction to Data Structures        .
Introduction to Data Structures .Ashutosh Satapathy
 
Flutter GDE session GDSC ZHCET AMU, aligarh
Flutter GDE session GDSC ZHCET AMU, aligarhFlutter GDE session GDSC ZHCET AMU, aligarh
Flutter GDE session GDSC ZHCET AMU, aligarhjamesbond00714
 
0950_Rodriguez_200520_Work_done-GEOGalicia_ELAB-converted.pptx
0950_Rodriguez_200520_Work_done-GEOGalicia_ELAB-converted.pptx0950_Rodriguez_200520_Work_done-GEOGalicia_ELAB-converted.pptx
0950_Rodriguez_200520_Work_done-GEOGalicia_ELAB-converted.pptxssuser886c55
 
Final PPT.ppt about human detection and counting
Final PPT.ppt  about human detection and countingFinal PPT.ppt  about human detection and counting
Final PPT.ppt about human detection and countingArbazAhmad25
 
Conventional vs Modern method (Philosophies) of Tunneling-re.pptx
Conventional vs Modern method (Philosophies) of Tunneling-re.pptxConventional vs Modern method (Philosophies) of Tunneling-re.pptx
Conventional vs Modern method (Philosophies) of Tunneling-re.pptxSAQIB KHURSHEED WANI
 
NIPORT Home Economics Questions Solution 2024.pdf
NIPORT Home Economics Questions Solution 2024.pdfNIPORT Home Economics Questions Solution 2024.pdf
NIPORT Home Economics Questions Solution 2024.pdfMohonDas
 
Field Report on present condition of Ward 1 and Ward 2 of Pabna Municipality
Field Report on present condition of Ward 1 and Ward 2 of Pabna MunicipalityField Report on present condition of Ward 1 and Ward 2 of Pabna Municipality
Field Report on present condition of Ward 1 and Ward 2 of Pabna MunicipalityMorshed Ahmed Rahath
 
introduction to python, fundamentals and basics
introduction to python, fundamentals and basicsintroduction to python, fundamentals and basics
introduction to python, fundamentals and basicsKNaveenKumarECE
 
12. Stairs by U Nyi Hla ngae from Myanmar.pdf
12. Stairs by U Nyi Hla ngae from Myanmar.pdf12. Stairs by U Nyi Hla ngae from Myanmar.pdf
12. Stairs by U Nyi Hla ngae from Myanmar.pdftpo482247
 
Navigating Process Safety through Automation and Digitalization in the Oil an...
Navigating Process Safety through Automation and Digitalization in the Oil an...Navigating Process Safety through Automation and Digitalization in the Oil an...
Navigating Process Safety through Automation and Digitalization in the Oil an...soginsider
 
Support nodes for large-span coal storage structures
Support nodes for large-span coal storage structuresSupport nodes for large-span coal storage structures
Support nodes for large-span coal storage structureswendy cai
 
PhD summary of Luuk Brederode, presented at 2023-10-17 to Veitch Lister Consu...
PhD summary of Luuk Brederode, presented at 2023-10-17 to Veitch Lister Consu...PhD summary of Luuk Brederode, presented at 2023-10-17 to Veitch Lister Consu...
PhD summary of Luuk Brederode, presented at 2023-10-17 to Veitch Lister Consu...Luuk Brederode
 
Artificial organ courses Hussein L1-C2.pptx
Artificial organ courses Hussein  L1-C2.pptxArtificial organ courses Hussein  L1-C2.pptx
Artificial organ courses Hussein L1-C2.pptxHusseinMishbak
 
Road to GDSC (Become the next GDSC lead)
Road to GDSC (Become the next GDSC lead)Road to GDSC (Become the next GDSC lead)
Road to GDSC (Become the next GDSC lead)GDSCNiT
 
A brief about Jeypore Sub-station Presentation
A brief about Jeypore Sub-station PresentationA brief about Jeypore Sub-station Presentation
A brief about Jeypore Sub-station PresentationJeyporess2021
 
The Journey of Process Safety Management: Past, Present, and Future Trends
The Journey of Process Safety Management: Past, Present, and Future TrendsThe Journey of Process Safety Management: Past, Present, and Future Trends
The Journey of Process Safety Management: Past, Present, and Future Trendssoginsider
 

Último (20)

Searching and Sorting Algorithms
Searching and Sorting AlgorithmsSearching and Sorting Algorithms
Searching and Sorting Algorithms
 
The Art of Cloud Native Defense on Kubernetes
The Art of Cloud Native Defense on KubernetesThe Art of Cloud Native Defense on Kubernetes
The Art of Cloud Native Defense on Kubernetes
 
Introduction to Data Structures .
Introduction to Data Structures        .Introduction to Data Structures        .
Introduction to Data Structures .
 
Flutter GDE session GDSC ZHCET AMU, aligarh
Flutter GDE session GDSC ZHCET AMU, aligarhFlutter GDE session GDSC ZHCET AMU, aligarh
Flutter GDE session GDSC ZHCET AMU, aligarh
 
FOREST FIRE USING IoT-A Visual to UG students
FOREST FIRE USING IoT-A Visual to UG studentsFOREST FIRE USING IoT-A Visual to UG students
FOREST FIRE USING IoT-A Visual to UG students
 
0950_Rodriguez_200520_Work_done-GEOGalicia_ELAB-converted.pptx
0950_Rodriguez_200520_Work_done-GEOGalicia_ELAB-converted.pptx0950_Rodriguez_200520_Work_done-GEOGalicia_ELAB-converted.pptx
0950_Rodriguez_200520_Work_done-GEOGalicia_ELAB-converted.pptx
 
Industry perspective on cold in-place recycling
Industry perspective on cold in-place recyclingIndustry perspective on cold in-place recycling
Industry perspective on cold in-place recycling
 
Final PPT.ppt about human detection and counting
Final PPT.ppt  about human detection and countingFinal PPT.ppt  about human detection and counting
Final PPT.ppt about human detection and counting
 
Conventional vs Modern method (Philosophies) of Tunneling-re.pptx
Conventional vs Modern method (Philosophies) of Tunneling-re.pptxConventional vs Modern method (Philosophies) of Tunneling-re.pptx
Conventional vs Modern method (Philosophies) of Tunneling-re.pptx
 
NIPORT Home Economics Questions Solution 2024.pdf
NIPORT Home Economics Questions Solution 2024.pdfNIPORT Home Economics Questions Solution 2024.pdf
NIPORT Home Economics Questions Solution 2024.pdf
 
Field Report on present condition of Ward 1 and Ward 2 of Pabna Municipality
Field Report on present condition of Ward 1 and Ward 2 of Pabna MunicipalityField Report on present condition of Ward 1 and Ward 2 of Pabna Municipality
Field Report on present condition of Ward 1 and Ward 2 of Pabna Municipality
 
introduction to python, fundamentals and basics
introduction to python, fundamentals and basicsintroduction to python, fundamentals and basics
introduction to python, fundamentals and basics
 
12. Stairs by U Nyi Hla ngae from Myanmar.pdf
12. Stairs by U Nyi Hla ngae from Myanmar.pdf12. Stairs by U Nyi Hla ngae from Myanmar.pdf
12. Stairs by U Nyi Hla ngae from Myanmar.pdf
 
Navigating Process Safety through Automation and Digitalization in the Oil an...
Navigating Process Safety through Automation and Digitalization in the Oil an...Navigating Process Safety through Automation and Digitalization in the Oil an...
Navigating Process Safety through Automation and Digitalization in the Oil an...
 
Support nodes for large-span coal storage structures
Support nodes for large-span coal storage structuresSupport nodes for large-span coal storage structures
Support nodes for large-span coal storage structures
 
PhD summary of Luuk Brederode, presented at 2023-10-17 to Veitch Lister Consu...
PhD summary of Luuk Brederode, presented at 2023-10-17 to Veitch Lister Consu...PhD summary of Luuk Brederode, presented at 2023-10-17 to Veitch Lister Consu...
PhD summary of Luuk Brederode, presented at 2023-10-17 to Veitch Lister Consu...
 
Artificial organ courses Hussein L1-C2.pptx
Artificial organ courses Hussein  L1-C2.pptxArtificial organ courses Hussein  L1-C2.pptx
Artificial organ courses Hussein L1-C2.pptx
 
Road to GDSC (Become the next GDSC lead)
Road to GDSC (Become the next GDSC lead)Road to GDSC (Become the next GDSC lead)
Road to GDSC (Become the next GDSC lead)
 
A brief about Jeypore Sub-station Presentation
A brief about Jeypore Sub-station PresentationA brief about Jeypore Sub-station Presentation
A brief about Jeypore Sub-station Presentation
 
The Journey of Process Safety Management: Past, Present, and Future Trends
The Journey of Process Safety Management: Past, Present, and Future TrendsThe Journey of Process Safety Management: Past, Present, and Future Trends
The Journey of Process Safety Management: Past, Present, and Future Trends
 

WorkerRep: Building Trust on Crowdsourcing Platform Using Blockchain

  • 1. WorkerRep: Building Trust on Crowdsourcing Platform Using Blockchain Gurpriya Kaur Bhatia (MT16021) Advisors: Dr. Ponnurangam Kumaraguru(IIITD) Dr. Alpana Dubey(Accenture Technology Labs) LinkedIn/in/gurpriya-kaur -562ab89a/ @gurpriya06 gurpriya.bhatia.9
  • 2. Thesis Committee ➢ Dr. Arun Balaji Buduru, IIITD ➢ Mr. Vikrant Kaulgud, Accenture Technology Labs ➢ Dr. Alpana Dubey, Accenture Technology Labs (Co - Chair) ➢ Dr. Ponnurangam Kumaraguru, IIITD (Chair) 2
  • 6. Crowdsourcing Platform Overview ENTITIES WORKFLOW Task Poster Worker Task 6
  • 7. Crowdsourcing Platform Overview ENTITIES WORKFLOW Task Poster Worker Platform Task 7
  • 8. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task 8
  • 9. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task Task Search and Apply 9
  • 10. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task Task Search and Apply Select worker 10
  • 11. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task Task Search and Apply Select worker Perform task 11
  • 12. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task Task Search and Apply Select worker Submit solution 12 Perform task
  • 13. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post task Task search and apply Select worker Submit solutionPerform review 13 Perform task
  • 14. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task Task Search and Apply Select worker Submit solutionPerform review and send feedback 14 Perform task
  • 16. Ethereum 16 Blockchain - based distributed computing platform Ether Smart Contract Ethereum
  • 17. Ethereum 17 User Ethereum Wallet Smart Contract 9f4195924cb79195ef9b07e 04e641a86e2eaa98340b40 15e8884a9f4ba4a6ec0 d59b04abcbeaf11aeebb6325e0 393bfdd1a016a5ae6b58c31a67 f1aa95030be6b02ae8577e8cbb e17c4577272fa3055ea2d53917 c07e43d9105e4d7e5f65318e 0x0ca7B9c0E24c Ac746F29BA984e C8Ba79856c3a83 Private Key Public Key Public Address
  • 18. DISTRIBUTED STORAGE 18 InterPlanetary File System Storage cost 1. Data 2. Hash QmfJgmCHkame9yxbNG7VwrzYJf8uZr dbHHcor6oNPBYQou User
  • 22. Research Aim ➢ Modeling crowdsourcing platform with Blockchain based design principles.. ➢ Propose submission evaluation strategy that is insusceptible to malicious users so as to build a robust and immutable reputation system for our platform. 22
  • 23. Outline ➢ Proposed Architecture ➢ Steps in Crowdsourcing Cycle ➢ Evaluating Submission and Updating Reputation ➢ Experiments and Robustness Against Attacks ➢ Conclusion 23
  • 24. Proposed Crowdsourcing Platform Overview 24 Post Task Task Search Select workerApply for task Perform task Submit solutionPerform review and send feedback
  • 25. Application Layer Storage Layer Ethereum Layer IPFS Front End-Framework Web3.js User Contract Task Contract Submission Contract Agreement Contract Evaluation Contract Meta Mask Proposed Architecture Off Blockchain On Blockchain 25
  • 26. User Registration 1. Profile Info 2. Hash 3. Create Worker/ Task poster INPUT : Hash , Public key User User Contract 4. User Registration transaction mined 5. Registration Successful OUTPUT: Worker / Task Poster 26
  • 27. Post Task 2. Hash 1. Task Info Task Poster 3. Post Task ( Hash, Reward, Skills, Title) 4. Task transaction mined Task Contract Task 27
  • 28. Task Search and Task Registration 28 Tasks ListWorker 1. Task search Task Contract 2. Task register (taskID)
  • 29. Task Poster Agreement 1. Create 2. Informs worker 3. Accept 6. Submit hash Worker Task Assignment and Submission Off - Blockchain hashing 4. Submission 5. Hash Evaluation Contract 7. Perform evaluations 29
  • 30. Agreement 4. Hash 8. Update Reputation Worker Evaluator N Evaluator 1 Task Assignment, Evaluation and Updating Reputation 1. Submission 2. Encryption Encryption algorithm 3. Store encrypted submission 5. Hash 6. Assign evaluators 7. Review 30
  • 31. Outline ➢ Research Motivation ➢ Research Aim ➢ Proposed Architecture ➢ Steps in Crowdsourcing Cycle ➢ Evaluating Submission and Updating Reputation ➢ Experiments and Robustness Against Attacks ➢ Conclusion 31
  • 32. Terminology ➢ Given a Worker w who submitted the solution s for task t ➢ Let E = {e1 , e2 , … en }be the set of evaluators for submission s ➢ {ci , qi } rating given by ei ➢ ri reputation of ei ➢ weightq and weightc : weights assigned to quality score and completeness score respectively. ➢ Let P denote the set of potential evaluators. ➢ evalSet (t1 ,t2 ,…, tf ) tasks to be evaluated by w corresponding to t. 32
  • 33. Potential Evaluators Denoted by P, Workers whose: ➢ Submission is not yet evaluated. ➢ Reputation > average reputation of workers on platform. 33
  • 34. Selecting Evaluators 34 Let |E| be number of evaluators required for evaluating submission s Divide P into |E| groups based on reputation scores Randomly choose one evaluator from each interval
  • 35. Review Each evaluator gives 2 types of reviews: ➢ Textual ➢ Feedback, reason supporting rating. ➢ Rating (1-100) ➢ Quality(qi ) ➢ Completeness(ci ) 35
  • 36. Consensus ➢ Outlier: if score given by evaluator not within 1.25 standard deviation away from mean. outliers set of outliers for submission s. ➢ Consensus majority evaluators are not outliers. ➢ Reviews of only those evaluators who form consensus are considered for computing reputation score of the worker. After removing outlier let, Cm mean completeness score. Qm mean quality score. 36
  • 37. Updating Reputation Two ways through which the reputation score of worker is updated: ➢ Submission and evaluation ➢ Only evaluation 37
  • 38. Updating Reputation Through Submission and Evaluation ◆ 38
  • 39. Updating Reputation Through Submission and Evaluation ◆ 39
  • 40. Updating Reputation Through Evaluation ◆ 40
  • 42. Demo 42 ➢ Post Task ➢ Task Application ➢ Select worker and create agreement ➢ Worker accepts the agreement and start working on it ➢ Submits hash and evaluate task ➢ Get evaluators and send them encrypted solution ➢ Evaluators send review and reputation is updated
  • 43. Outline ➢ Research Motivation ➢ Research Aim ➢ Proposed Architecture ➢ Unboxing the Steps ➢ Evaluating Submission and Updating Reputation ➢ Experiments and Robustness Against Attacks ➢ Conclusion 43
  • 44. Various Attacks ➢ Unfair rating attack: is when the rater is biased towards worker and does not give a truthful opinion about him. ➢ Reciprocity: when worker reciprocates negatively for a negative review that he receives. ➢ Ballot stuffing : when the worker tries to increase its own reputation. ➢ Collusion: when a group of workers try to collude together to improve their own reputation or decrease the reputation of others. ➢ Sybil attack:where a malicious worker tries to create multiple identities over the platform to gain influence 45
  • 45. Robustness Against Various Attacks ➢ Unfair rating attack: ➢ Consensus among evaluators and final rating is being normalized using median. ➢ Reputation of evaluator is decreased if he is an outlier. ➢ Reciprocity: ➢ Outliers removed. ➢ Ballot stuffing and Collusion ➢ Evaluators pseudo randomly chosen. ➢ A trail of evaluation transactions. ➢ Reputation value altered only by contract code. ➢ Sybil Attack 46
  • 46. Experimental Results Reward Amount($) User type AMT($) Upwork($) WorkerRep($) 50 Worker - 10 0.436 Task Poster 10 1.375 0.195 100 Worker - 20 0.436 Task Poster 20 2.75 0.195 1000 Worker - 200 0.436 Task Poster 200 27.5 0.195 47
  • 47. Conclusion ➢ How each step of crowdsourcing could be managed as transaction on Blockchain. ➢ Address various attacks possible on centralized Crowdsourcing platform. ➢ Establishing better trust on worker’s of the platform. ➢ Reduce cost for task poster as well as worker. 47
  • 48. Challenges, Limitation and Future Work ➢ Time delay between creating a transaction and its confirmation - Sharding. ➢ Computation cost on Ethereum network - Computational oracles. ➢ Task poster having a say in the submission evaluation and maintaining its reputation score. ➢ Encrypting evaluation initially until all evaluators have sent. 48
  • 49. Acknowledgement ➢ Committee members ➢ Abhinav, Accenture Technology Labs ➢ Indira, Shubham, Simran, Shwetanshu ➢ Members of Precog family ➢ Family and friends 49
  • 50. References ➢ Li, M., Weng, J., Yang, A., Lu, W., Zhang, Y., Hou, L., Liu, J.N., Xiang, Y. and Deng, R.H., 2017. CrowdBC: A Blockchain-based Decentralized Framework for Crowdsourcing. IACR Cryptol. ePrint Arch., Univ. California, Santa Barbara, Santa Barbara, CA, USA, Tech. Rep, 444, p.2017. ➢ Dennis, R. and Owenson, G.H., 2016, February. Rep on the block: A next generation reputation system based on the blockchain. In 10th International Conference for Internet Technology and Secured Transactions (ICITST). IEEE. ➢ Tavakolifard, M. and Almeroth, K.C., 2012. A taxonomy to express open challenges in trust and reputation systems. Journal of Communications, 7(7), pp.538-551. ➢ Jøsang, A. and Golbeck, J., 2009, September. Challenges for robust trust 50