Table of Contents
Preface iii
Figures ix
Tables xi
Summary xiii
Acknowledgments xxv
Abbreviations xxvii
Chapter 1 Introduction 1
What We Know About Relationships Between School Leadership and Student Achievement 3
The Need for New Systems for Evaluating Principals 6
Evaluation Questions 8
Organization of This Report 9
Chapter 2 Data Sources and Analytic Approach 11
Data Sources 12
Principal Surveys 12
Coach Surveys 14
Teacher Survey 15
Interviews with Building-Level Staff 16
Interviews and Email Communications with District-Level Staff and External Consultants 18
Reviews of Documentation 18
Principals' Scores on the Administrators' Performance Standard Rubric 19
Student Data 19
Analytic Approach 19
Principal, Coach, and Teacher Survey Data 19
Factor Analysis 20
Interviews with Building-Level Staff 20
District Interviews and Document Review 21
Rubric, Achievement, and Bonus Data 21
Method for Linking and Jointly Analyzing Rubric and Achievement Data 21
Limitations 22
Chapter 3 District Context and Pittsburgh Principal Incentive Program Theory of Action 23
District Context and Related Reforms 23
Theory of Action Overview 27
Interventions for Building Capacity 29
Professional Development to Improve Leadership 29
Evaluation and Feedback 31
Mechanisms for Evaluation 31
Mechanisms for Feedback: Assistant Superintendent Coaching 33
Mechanisms for Feedback: Teaching and Learning Teams 33
Incentives 34
Salary Increment 34
Achievement Bonus 34
Determination of Bonus Award Amounts 39
Impact on Principals 41
Impact on Schools 42
Impact on Classrooms and Students 42
Chapter 4 Capacity-Building Interventions 43
Professional Development 43
Evaluation and Feedback 47
Incentives 50
Summary of Key Findings 55
Chapter 5 Principals' Leadership Practices, Principals' Skills, and School- and Classroom-Level Responses to the Pittsburgh Principal Incentive Program 57
Changes in Principals' Leadership Practices and Skills 57
Principals' Practices 57
Principals' Perceptions of Their Skills and Need for Support 60
Other Staff Perceptions of Principals' Leadership 60
Perceived Role of the Pittsburgh Principal Incentive Program 63
School- and Classroom-Level Changes 64
In-School Professional Development 64
Incorporating Data into Instructional Leadership 65
Teaching and Learning Teams 66
Changes in Classrooms 67
Summary 69
Chapter 6 Principals' Performance on Rubric and Achievement Measures 71
Principals' Performance on the Evaluation Rubric 71
Principals' Performance on the Achievement Bonus Measures 79
Bonus Payments and Principals' Mobility 85
Summary 87
Chapter 7 Student Achievement Trends 89
Achievement Trends 89
Addressing Racial/Ethnic and Socioeconomic Achievement Gaps 95
Summary 104
Chapter 8 Key Findings and Recommendations 105
Key Findings 105
What Is the District's Theory of Action Regarding How the Pittsburgh Principal Incentive Program Is Expected to Promote Improved Student Outcomes? 105
How Were the Pittsburgh Principal Incentive Program Capacity-Building Interventions Implemented, and How Have Principals Responded to Them? 106
In What Ways Did Principals' Skills and Practices Change over the Course of the Pittsburgh Principal Incentive Program? 108
What Conditions Changed at the School and Classroom Levels over the Course of the Pittsburgh Principal Incentive Program? 108
How Did Principals Perform on the Rubric and Bonus Measures, and How Was Performance Related to Principal Mobility? 109
How Did Student Achievement Change During the Course of the Pittsburgh Principal Incentive Program, and How Did Racial and Socioeconomic Gaps Change? 111
Recommendations 112
Recommendations for Evaluation System and Measure Development 112
Recommendations for Implementation 115
Conclusion 117
References 119
