Steve Hopkins, Principal of Market Cube, discusses the latest trends and data on routing at SampleCon 2013. He gives a brief look into some key findings from the ARF's Foundations of Quality 2 research initiative.
2. Key Findings from Three Papers
Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects.
Paper given at the 2010 CASRO Panel Conference.
Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with
Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled
Respondents. Presentation given at the 2011 CASRO Technology Conference.
Brigham, Nancy, Jason Fuller (2012). Survey Router Management: An Experimental
Examination of Impact on Survey Results. Presentation given at the 2012 CASRO
Online Conference.
3. Johnson & Fawson Reference the
”Tragedy of the Commons”
Wikipedia Says
In economics, the tragedy of the commons is the depletion of a shared resource by
individuals, acting independently and rationally according to each one's selfinterest, despite their understanding that depleting the common resource is contrary
to the group's long-term best interests. "Commons" can include the
atmosphere, oceans, rivers, fish stocks, national parks, advertising, and even parking
meters.
And, even online sample!
4. Source: Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.
5. Source: Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.
6. Source: Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.
7. Source: Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.
8. Johnson & Fawson Conclusions &
Suggested Controls
Source: Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.
9. Brigham et al: Reallocation
Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of
“Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.
10. Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of
“Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.
11. Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of
“Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.
12. Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of
“Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.
13. Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of
“Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.
14. Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of
“Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.
15. Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of
“Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.
16. Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of
“Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.
17. Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of
“Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.
18. Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of
“Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.
19. Brigham & Fuller Paper Sought to
Answer Three Questions
1. Does spending more time in the router screening process have
an impact on results?
2. Does correlation among screening questions matter? Should a
router be diverse in the types of studies it includes?
3. How small can a router be – are a minimum number of studies
needed?
Source: Brigham, Nancy, Jason Fuller (2012). Survey Router Management: An Experimental Examination of Impact on Survey Results.
Presentation given at the 2012 CASRO Online Conference.
20. “Spending a greater amount of time in the router
was not related to the screening process.”
Source: Brigham, Nancy, Jason Fuller (2012). Survey Router Management: An Experimental Examination of Impact on Survey Results.
Presentation given at the 2012 CASRO Online Conference.
21. “A highly correlated router produced skewed results; including
a mixture of correlations brought results back into line.”
Source: Brigham, Nancy, Jason Fuller (2012). Survey Router Management: An Experimental Examination of Impact on Survey Results.
Presentation given at the 2012 CASRO Online Conference.
22. “More studies are better – Test metrics are closer
to Control around 10-12 surveys in a router.”
Source: Brigham, Nancy, Jason Fuller (2012). Survey Router Management: An Experimental Examination of Impact on Survey Results.
Presentation given at the 2012 CASRO Online Conference.