Data Findings Point to Highly Effective School Year Partnerships in 2015-16

On August 30, Boston Beyond’s school year partners gathered for the semi-annual AQP and BoSTEM Summit to explore cohort-wide program quality trends for the 2015-16 school year, debrief about last year‘s data collection process, hear an update from Boston Public Schools, and discuss the plan for the year ahead.

FullSizeRender (1)

At the Summit, Jon Sproul and Miriam Rubin from BPS’s Office of School-Community Partnerships provided an overview of the district’s new five-year Strategic Implementation Plan, which includes a deliberate focus on increasing equitable access to high quality partner activities. In support of this goal, BPS recently launched PartnerBPS, the new collaborative platform for tracking and increasing access to these partnerships. Sproul and Rubin encouraged all school-year partners to create a program profile on the site. For more information, click the image below.

UntitledData Findings from SY2015-16

Slide04During the 2016-17 school year, the Advancing Quality Partnerships initiative is slated to involve 42 different programs, reaching students from more than 26 BPS schools. This marks a substantial increase from last year, when the network involved 31 programs and 19 schools.

Based on external observer data from last spring, AQP partners met or exceeded performance expectations in all 15 measured indicators of program quality. Furthermore, compared to baseline data from the fall, these programs improved or maintained high levels in 12 of these domains, most notably in Arrival Logistics & Greetings, Schedule and Offering, Youth Relations with Adults, and Youth Participation.


From the youth perspective, program performance mirrored these positive trends, with programs exceeding the quality benchmark in 5 out of the 8 measured domains: Supportive Social Environment, Supportive Adults Present, Youth Enjoy and Feel Engaged, Youth Feel Challenged, and Helps Youth Academically. Yet youth survey results pointed to room for improvement in the following domains: Opportunities for Leadership and Responsibility, Youth Have Choice and Autonomy, and Helps Youth Socially.


ACTFrameworkWHITEICONS_zps338fbde2Measures of program quality are closely connected to students’ social-emotional skill growth. Boston Beyond’s analysis of program quality is based on the Achieve-Connect-Thrive (ACT) Skills Framework, which outlines the foundational social-emotional skills that students need for success in school, college, career, and life.

To demonstrate our partners’ capacity to promote various social-emotional skills in their programming, we mapped these program quality data to the skill outcomes we’re working to develop in students. Looking at the data this way, we see that programs modestly improved practice between the fall and spring in the key areas that we’ve been focusing on as a group: critical thinking, teamwork, engaging youth, and leadership and choice.


When comparing observer and youth ratings, the youth results follow the same pattern as the adults. However, similar to the trends that we have seen over many years with both summer and school-year programs, when one the youth consistently rate programs more critically than their adults counterparts.

Recommendations for Improvement

Katie Tosh, Boston Beyond’s Director of Measurement offered the following recommendations to programs on how they can use these data to spur improvement in the upcoming school year:

  • Review program PRISM Report and/or DoS Write-Up
    • Are the results what you expected? Why or why not?
    • Did you improve in areas you focused on for the spring semester? Why or why not?
    • How will you share these results with different levels of staff in order to inform program content and delivery?
  • Considering the goals of your program, which skill areas can you intentionally prioritize for the upcoming school year?
    • Focus on maintaining areas of strengths while improving areas of challenge
    • Select a small number of practices/skills to address
    • What specific action steps will you take to incorporate practices that will help students develop those skills?
    • Will you attend a PEAR Action Planning session this year?
  • All programs should consider focusing on Critical Thinking, Teamwork, and Leadership & Choice
  • STEM programs should closely examine the areas of STEM knowledge and practice, which relate to critical thinking, as well as making real-world connections between STEM lessons and everyday life of students.

Resources from the Summit

Advancing Quality Partnerships Summer Summit DeckBPS School-Community Partnerships DeckAPT Observer Certification Pilot 2016-17AQP Calendar 2016-17BoSTEM Calendar 2016-17Partnership Effectiveness Self-Assessment