1. Developmental Algebra Project Borough Of Manhattan Community College The Effects of Study Skills Trainingand Peer Coaching of At-Risk-Studentson Retention and Passing Rates in a Remedial Mathematics Course Leonid Khazanov Michael George Annette Gourgey Chris McCarthy 1

2. 2 What is Developmental Algebra? 1. First course in Algebra 2. Zero credit course 3. Placement 4. Student challenges poor math and study skills math anxiety lack of motivation

3. 3 Developmental Algebra ProjectGoals and Treatment Main Goals 1. Reduce attrition rate 2. Improve course pass rate

4. 4 Developmental Algebra ProjectGoals and Treatment Main Goals 1. Reduce attrition rate 2. Improve course pass rate Treatment Incorporate Study Skills 2. Identify at-risk-students and assign them peer coaches

5. 5 Developmental Algebra Project Rationale Q. Why teach study skills? A. Research shows that a large proportion of developmental students have poor study skills and high levels of math anxiety.

6. 6 Developmental Algebra Project Rationale Q. Why target at-risk students? A. At-risk students have the highest failure and drop out rates. Successful treatment of at-risk students could have the greatest impacts on overall pass and retention rates.

7. 7 Developmental Algebra ProjectImplementation Q. How were at-risk students identified? A. We developed an instrument to identify at risk students that had three components: (1) Arithmetic Diagnostic Test, (2) Student Survey, (3) Classroom Observation.

8. 8 Developmental Algebra Project Implementation Q. What are coaches? A. Coaches are successful math students at BMCC who assume the roles of both tutor and mentor.

9. 9 Developmental Algebra Project Implementation Coach Responsibilities: Meet with each of their students once or twice a week to: Discuss progress Review homework Prepare for tests Explain the material from previous classes if necessary Discuss counter productive behaviors such as absences and lateness (and how to avoid them in the future) Help student to improve math study habits and skills

10. 10 Developmental Algebra Project Implementation Coaches Submitted Weekly Reports: Reports contain information about what was accomplished at each meeting.

11. 11 Developmental Algebra ProjectCoach’s Reports Protégé’s name: T Date: Oct. 12 and Oct 14 2010 Time: 10 am to 11 am Summary of Meeting: In this week we checked the homework problem on the first-degree inequalities, and also on the first-degree equations. He has the first test and he gets 85% so I think this is a good score for the first time. We also review everything from the beginning of the school, because he is going to have test this week.

12. 12 Developmental Algebra ProjectCoach’s Reports Protégé’s name: Mrs. J Date: November 01, 2010(Monday) Time: 11 AM- 12 PM Summary of Meeting: Mrs. J was a little down on hope because she didn’t exceed well on her midterm. Her score was a 20%. Throughout the whole time of our meeting I gave her friendly inspirational advice. My advice to her was “Don’t let one test ruin your day and bring you down”. “You still have more chances to achieve in your class and I will work with you every step of the way to prepare you for your final exam”.

13. 13 Developmental Algebra ProjectCoach’s Reports Protégé’s name: C Date: November 01, 2010(Monday) Time: 5:30 PM- 6:30 PM Summary of Meeting: C is such a sweet and kind heart woman. She is serious about passing this course. Knowing she is intimated of the numbers I always told her the numbers are more afraid of us than we are to them… [She] did astounding showing her my quick techniques. Furthermore, she always comes prepare with her questions to be answer. I am always able to help her and give her hints on her homework handouts….

14. 14 Developmental Algebra ProjectCoach’s Reports Protégé’s name: R Date: Wednesday, Dec 1 Time: 12 – 2 pm Summary of Meeting: R has lot problems with the majority of exercises from the final review sheet. I been telling that he has to practice more and put more time and effort. He needs to memorize all this rules in order to do each problem and pass the final. Also, I let him do each problem twice so he doesn’t forget how to do them.

15. 15 Developmental Algebra Project Implementation Q. How were coaches recruited? Directly from pre-calc and calculus classes (classroom visit). Recommendations of colleagues. Word of mouth.

16. 16 Developmental Algebra Project Implementation Q. How were coaches trained? CITI Certification (IRB) 3 hour training session Ongoing training

17. 17 Developmental Algebra Project Implementation Coach Training Components Study skills training. How to tutor. How to mentor.

18. 18 Developmental Algebra Project Implementation Q. How were instructors trained? Three hour training session. Study skills expert. Math Study SkillsBook (Alan Bass)

19. 19 Developmental Algebra Project Experimental Design Five instructors were randomly selected from a list of BMCC math faculty who were scheduled to teach two sections of Elementary Algebra (Mat 051). One of section from each instructor was randomly designated as control and the other treatment.

20. 20 What was the treatment?1) Study Skills Training Instructors incorporated a series of short lessons developed by a study skills expert: effective note taking, managing time, overcoming math anxiety, etc.

21. 21 What was the treatment?2) Peer Coaching Peer coaches met with each of their students once or twice a week to: Discuss progress Review homework Prepare for tests Explain the material from previous classes if necessary Discuss counter productive behaviors such as absences and lateness (and how to avoid them in the future) Help student to improve math study habits and skills

22. 22 Developmental Algebra Project Outcomes: Treatment vs Control

23. Retention Results 23 Control Group Treatment Group Statistical Analysis: RESULTS ARE SIGNIFIGANT EVIDENCE THAT THE TREATMENT INCREASES RETENTION RATES p-value = 0.01299 (2x2x5 Exact Test, Combinatorial)

24. Aggregated Retention Results 24 Control Group Treatment Group Statistical Analysis: RESULTS ARE SIGNIFIGANT EVIDENCE THAT THE TREATMENT INCREASES RETENTION RATES p-value = 0.01212 (2-sample test for equality of proportions with continuity correction, Chi)

25. Passing Rate Results 25 Control Group Treatment Group Statistical Analysis: RESULTS DO NOT PROVIDE SIGNIFIGANT EVIDENCE THAT THE TREATMENT INCREASES PASS RATES p-value = 0.3013 (2x2x5 Exact Test, Combinatorial)

26. Aggregated Passing Rate Results 26 Control Group Treatment Group Statistical Analysis: TREATMENT GROUP HAD A HIGHER PASSING RATE, BUT THE RESULTS DO NOT PROVIDE SIGNIFIGANT EVIDENCE THAT THE TREATMENT INCREASES PASS RATES. p-value = 0.3160 (2-sample test for equality of proportions with continuity correction, Chi)

27. The Diagnostic Test we developed works Passing Rates vs. Total Risk Points. Aggregate data: 45% of students diagnosed as low-risk passed the class 24% diagnosed as high-risk passed class Statistical Analysis: High Risk Students (4+ total risk points) have a lower passing rate (p=.0021). Method: 2-sample test for equality of proportions with continuity correction 27

28. Passing Rates vs. Number of Coaching Sessions Attended Peer-coaching seems to have a positive effect on pass-rates after a threshold of about 4 sessions. 28

29. Peer coaching (mentoring) works. Statistical Analysis: Peer Coaching Increase Pass Rates (p= 0.03356) (2x2x4 Cochran-Mantel-Haenszel Test for Count Data, Exact Test, Combinatorial) 29

30. Peer coaching (mentoring) works - cost issues: 30/100 = 30% high-risk students passed without peer coaching 14/26 = 54% high-risk students passed with peer coaching Roughly speaking, for every 5 high-risk students treated we expect to get 1 extra student pass. If it costs $100 per treatment, that is $500 per additional pass. 30

31. 31 Borough of ManhattanCommunity CollegeWhat is next? CUNY It appears that peer coaching is an effective aspect of intervention for at-risk-students. Next step: finding cost effective ways to provide peer coaches to more students. Contact Information: Leonid Khazanov: lkhazanov@bmcc.cuny.edu Michael George: mgeorge@bmcc.cuny.edu Chris McCarthy: cmccarthy@bmcc.cuny.edu