1. # 7 Location In The Research Process Formulating Research Questions Appropriate Design Craft Knowledge Appropriate Sampling Decisions Previous Research Ensure Research Ethics Data Collection & Management Data Analysis Ongoing Research Interpretation Dissemination
2. Measurement II What did we catch? Was it the same as the others? How do we know we caught it?
3. Properties of Instruments Reliability The stability and consistency of a measure Validity How we know that our measure captures what we're interested in
5. Reliability The stability and consistency of an instrument Stability Similar results over time Consistency Test items will be consistent with each other Items will be consistent with the overall score on the test
6. Stability Measured using test-retest reliability checks A participant completes the test at one point in time After a period of time elapses they complete the test again Usually at least 6 weeks Assessed using correlation coefficients Pearson r Spearman
7. Consistency Internal consistency The items all measure the same core construct Measured using the Cronbach's Alpha Coefficient A form of multiple correlation Measures how much the individual items in a test or scale have in common statistically
8. First Question The Juvenile Court Administrator wants to evaluate a new assessment tool to find out if you can tell which kids, once released from Detention, will recidivate or not. What is the first thing that must be done to solve your groups research problem? How do you propose to do it?
9. Validity How we know that our instrument captures what we're interested in Concept translation How well our instrument translates a concept into a variable Validity is reported as correlations of varying complexity
11. Face Validity How well an instrument apparently accomplishes concept translation How completely an instrument samples all of the components of a concept Panels of experts In the domain being measured Who have experience with the target
12. Convergent / Concurrent Validity How well an instrument relates to other measures of the same or similar concepts Compare an instrument with: Other existing instruments Expert opinions Behavioral observations Assessed when the measure is given to subjects
13. Discriminant Validity How well an instrument discriminates between the presence or absence of concept components An instrument can discriminate between known groups, assessed when the measure is given to subjects
14. Predictive Validity How well an instrument predicts future events or behavior Useful concepts will predict phenomena Predictive validity assessment - examines how we know that our measure captures what we are interested in Compare the instrument with other predictive factors Test prediction of the measure alone Assessed longitudinally, after the instrument is given to subjects
15. Second Question The DCFS Area Regional Director wants to be able to tell how many future CPS referrals families are likely to have after their first CPS referral. He devises a questionnaire for the child’s mother to complete. What type of validity is involved in your research problem? What data might you use to address the validity question?
