1. Teacher Trajectories and Teacher Effectiveness: the role of policy and leadership Susanna Loeb CEPPE March 11, 2011

2. A Collaborative Effort Don Boyd, Hamp Lankford & Jim Wyckoff Teacher workforce Pam Grossman Teaching Jason Grissom, Eileen Horng, Demetra Kalogrides School Leadership

3. Problems and Contexts

4. Knowledge and Abilities of Teachers in the Classroom is a Function of… How can we improve each of these?

5. In the US, Differences and Similarities Low agreement / knowledge on support Preparation more similar to the 4+1 model for high schools – state policy Similarly, little direct recruitment or selection until recently More hours for PD, with recent focus on learning communities Low agreement / knowledge on how to select Similarly, substantially differences in teachers across schools (also within) Much bigger problem. (e.g. in NYC about 18% of teachers leave their school each year). Similarly, little differentiation of the teaching career

6. hypothesis: improvement rests on…

7. for example… what is best ?

8. In order to make good decisions, the decision makers need…

9. So considering our decision makers… Often better information (still imperfect) but little ability and often unaligned incentives Often little information to make good decisions May be missing different information and abilities Key is to improve incentives and abilities locally so they make better decisions But also to improve overall knowledge since the knowledge base in teaching is weak

11. Information

13. the starting place: year 2000, NY

14. Unequal Distributionteacher attributes by student test scores (New York State, in 2000)

15. Little Enforcement of State Policy

17. High achieving students

18. White students (at least for white teachers, majority)

19. Location - Close to home

20. Little State Enforcement

21. Little Direct Recruitment

22. Teacher education programs(little outreach, rarely selective)

23. Difficult-to-staff districts recruited certified teachers, little school-level control, late hiring (July and August), often relied on waivers

26. The New Teacher Project – 1997

28. Standardized tests used for accountability instead of just to inform parents or for student opportunities (e.g NY regents)

30. State flexibility

31. Weak research base for defining HQ – use professional consensus & State policy process

33. Middle and secondary teachers may complete an undergraduate or graduate degree in their field or advanced certification or credential.

35. Certification Exam Failure Rate of New Elementary Teachers, 2000-’05 High-poverty schools Low-poverty schools

36. Certification Exam Failure Rate of New Elementary Teachers, 2000-’05 High-poverty schools Low-poverty schools

37. Certification Exam Failure Rate of New Elementary Teachers, 2000-’05 High-poverty schools Low-poverty schools

38. Effect of changes in teacher characteristics on student math scores 25% decrease in the predicted gap

39. Conclusion 1 A small change in regulation can create a large change in incentives Incentives can inspire meaningful change in schools and for students

40. 2. Improvements to a Weak Knowledge Base linking teacher preparation to effectiveness to improve information

41. Current Knowledge Base Virtually no research linking preparation experiences to teachers’ classroom performance and, particularly to student outcomes Lack of information hinders simple decisions about certification policy at the state level as well as program design locally

42. Approach to Understanding Prep:first explore a first try

43. program features spring/summer 2004 18 institutions: 26 univ. programs + 4 TF + 1 TFA Document collection, interviews with director and director of field experiences, surveys of math and ELA methods faculty Many many measures but here focus mainly on link to practice. Program data are not ideal for this. whether or not the program required a capstone project a composite measure of the extent that the program maintains oversight over student teaching experiences requires a minimum number of years of teaching experience for its cooperating teachers (32%) program picks the cooperating teacher (42%) program supervisor observes their participants a minimum of five times during student teaching (27%) for comparison math and English content course requirements and the percent of tenure-line.

44. reported experiences: survey spring 2005 All first year teachers in NYC Practice Variables extent to which preparation included links to practice; opportunities to listen to an individual child read aloud for the purpose of assessing his/her reading achievement; …Plan a guided reading lesson, and …Study or analyze student math work (each 5-point scale). opportunities to study curriculum used in New York City whether or not the teacher had student teaching experiences, not as the teacher-of-record in the classroom; the congruence between student teaching placement and their current job assignment in terms of subject matter or grade level; Additional measures of preparation as controls and comparisons opportunities to learn about teaching math and ELA; learning; handling student misbehavior; and teaching English language learners.

45. opportunities to learn math methods alpha = 0.97 In your teacher preparation program, prior to September 2004, how much opportunity did you have to do the following (5-point scale)? learn typical difficulties students have with place value; learn typical difficulties students have with fractions; use representations (e.g., geometric representation, graphs, number lines) to show explicitly why a procedure works; prove that a solution is valid or that a method works for all similar cases; study, critique, or adapt math curriculum materials; study or analyze student math work; design math lessons; learn how to facilitate math learning for students in small groups; adapt math lessons for students with diverse needs and learning styles; And practice what you learned about teaching math in your teacher preparation program in your field experience.

46. first year results: program features * *

47. 1st year results: reported experiences

49. Well-supervised practice

50. Content course (effects in the second year)

51. Preparation for teaching English Learners

52. Surprisingly little research on preparation (little systematic identification of best practices)

54. 3. Understanding and Improving the Abilities of Local Decision Makers variation in the abilities and behaviors of school principals

55. 50.00% College Recommended (N=156) Teaching Fellows (N=36) Other (N=114) 40.00% 30.00% 20.00% 10.00% 0.00% Health Place SALARY Closer Home Prestige Benefits Pregnancy/Child Family/Personal Opportunity Job Security Dissatisfaction School Action Motivation: Why Teachers Leave

56. 50.00% College Recommended (N=150) 45.00% Teaching Fellows (N=35) Other (N=112) 40.00% 35.00% 30.00% 25.00% 20.00% 15.00% 10.00% 5.00% 0.00% Admin Safety Testing Respect Facilities Behavior Autonomy Colleagues Own Ability Assingment District Policy Teaching Philosophy when we look within dissatisfaction…

57. Motivation, more generally Principals linked to teacher satisfaction and career choices Principals central actors in most recent school reforms (accountability, school-based budgeting, charter schools) Better local information Improved incentives from accountability system Concern about abilities Increased policy attention on attracting and preparing effective school leaders, but Lack understanding of principal abilities to look for when hiring or to target development Lack of organized systems for recruiting and developing leaders (in most places)

58. our approach: again, exploratory

60. Research literature

61. Discussions with principals and

62. Piloting and shadowing in local California schools

63. Collected observational time use data

64. Observed each principal for one full day

65. Recorded time use on 47 (later more) tasks every five minutes

66. Sample

67. All high school principals in Miami-Dade County Public schools (plus 6 elementary and 6 middle school principals)

68. All schools serving 6th graders and above in Milwaukee Public Schools

69. All schools in San Francisco and Kansas City

71. Principals in M-DCPS (n = 314; 89% response)

72. Assistant principals (n = 585; 85%)

73. Teachers (n = 15,842; 83%) -- satisfaction

74. Asked principals how effective they felt at each of the tasks

75. Asked assistant principals how effective their principals were at each task

76. Identify groupings of self-assessed task effectiveness reflecting underlying skills

78. Across Districts: Most time on Administrative Tasks

80. Coaching teachers

81. Using data and assessments

82. PD for teachers

83. PD for themselves

85. Supervising students

86. Observing classrooms

87. Internal relationships

88. Compliance requirements

89. Managing budgetsMPS, M-DCPS

90. The Importance of Organization Management Time-Use Day-to-Day Instruction Internal Relationships Organization Management Instructional Program Teacher Satisfaction at Current School Teacher Assessments of School Parent Assessments of School Student Performance Gains positive relationship negative relationship

91. The Importance of Organization Management Efficacy Principals’ Organization Management Efficacy Other domains NOT related to outcomes. Student Performance Gains Parent Assessments TeacherSatisfaction

92. Corroboration Using Assistant Principal Assessments Principal self-assessments may not be the best way to measure task effectiveness Principals can’t be objective about themselves Administer the same inventory to their APs (multiple per school) and look for the same patterns Uncover 3 factors from AP responses: Instruction, Internal Relations, Operations

93. Assistant Principal Assessment of Principal Task Efficacy and Outcomes Assistant Principals’ Assessment of Principals’ Organization Management Efficacy Note: Other domains of efficacy NOT related to outcomes. Gains in Student Achievement ParentSatisfaction TeacherSatisfaction M-DCPS

95. Given current behaviors and abilities – improvements in Organization Management appear beneficial

96. Current focus on instruction likely useful but poorly focused

97. Classroom visits good for gaining information

98. But, only useful if information is used for improvement – rare.

99. Now looking at how to select leaders for organizational leadership skills as well as develop skills in principals

101. A range of factors affect and can improve the teaching workforce

102. solutions are complex and changing Needs vary For example, best professional development depends on the skills of the teachers and context of the teaching Best decisions vary and change over time Useful to consider who makes decisions and their abilities, information and incentives Our aim is to understand these dimensions and how policies can influence them

103. Some findings to date Simple regulations (such as higher test requirements) can incentivize local recruitment and improve the teaching workforce. Systematic knowledge generation in teacher development and teaching practice is both useful and doable Local variation in needs and options lead to benefits of local decision making but local decision makers (e.g. principals) vary in their abilities a similar policies and information are needed for the leadership labor markets.

104. Teacher Trajectories and Teacher Effectiveness: the role of policy and leadership Susanna Loeb CEPPE March 11, 2011