Boston Learns Together and the Achieve, Connect, Thrive (ACT) Skills Framework unite the city around a common youth agenda and allow flexibility in approach so that programs can meet the unique needs of their students. A common suite of assessment and observational tools provides detailed information on individual student-level outcomes and measures of program quality, so that stakeholders can identify what is working and what needs to be improved. Our central database, Cityspan Provider, provides the infrastructure for collecting, analyzing, and reporting on these data across out-of-school programs in Boston. This interactive platform allows us to document on a citywide level the number of programs providing expanded learning opportunities and participating in a coordinated system of measurement and evaluation. Similarly, this database allows us to track indicators of program quality, student enrollment, attendance, demographic data, and academic and social-emotional outcomes, in addition to supporting digital badges. From this work, BASB aims to extrapolate conclusions about the current reach and impact of the out-of-school time field in Boston as well as inform our continuous improvement strategies.
Closing the opportunity gap is a big–but achievable–goal. The suite of measurement tools adopted by all of Boston Beyond’s program partners provides a common, quantifiable language to validate progress toward meeting this goal. There are several ways for programs to get involved that are tailored to the readiness and capacity of the program to take on measurement work. See descriptions of the Membership Levels below for more details. Regardless of the Membership Level at which a program opts to participate, each program will receive reports with their summary data, and year-round professional development and peer-learning opportunities are available to all partners. It is expected that program partners will be active members in this community of practice.
Benefits to getting involved in this work include:
- Join a Learning Community of fellow program providers to share best practices, and collaborate to address collective areas of improvement.
- Gain access to experts in the fields of expanded learning, youth development, and social-emotional learning.
- Receive dashboard reports that benchmark your performance against the previous year and the community as a whole.
Basic Membership Level
Track program impact by consolidating program activities, youth enrollment, attendance, and demographic data in a citywide database. Programs collect parental consent forms for all participating youth to be involved in data collection.
After the Basic Membership Level, program partners may choose to enter into the Program Quality Membership Level, the Individualized Support Membership Level, or both.
Program Quality Membership Level
All responsibilities of Basic Membership apply. Additionally, programs measure and track their program performance using the perspectives of students and a third-party observer.
- The third-party observer rates the program on aspects of program organization and structure, supportive environment, engagement in activities and learning, and social-emotional skill building using the Assessment of Program Practices Tool (APT), developed by the National Institute on Out-of-School Time (NIOST) in partnership with the Massachusetts Department of Elementary and Secondary Education 21st Century Community Learning Center Program.
- The youth program participants in grades 4 and up take a common youth survey at the end of the program in which they reflect upon their program experiences and opportunities to learn certain social-emotional skills. This survey uses items from NIOST’s Survey of Academic and Youth Outcomes – Youth Version (SAYO Y) as well as from the Youth Experience Survey.
If you are a STEM-focused program, you can join BoSTEM and implement tools that are tailored to the nature of STEM programs.
- The third-party observer rates the program on features of the learning environment, youth development in STEM, activity engagement, and STEM knowledge and practices using the Dimensions of Success (DoS) tool developed by the PEAR Institute. This is instead of using the APT.
- The youth program participants in grades 4 and up take the SAYO Y to reflect upon their program experiences and opportunities to learn certain social-emotional skills.
- The youth program participants in grades 4 and up also reflect upon their interest and engagement in STEM at the end of the program by taking the Common Instrument survey, also developed by the PEAR Institute.
Individualized Support Membership Level
All responsibilities of Basic Membership apply. Additionally, partners administer the PEAR Institute’s Holistic Student Assessment (HSA) to students in grades 5 and up during the beginning of programming. With this survey students self-assess their social-emotional strengths. Programs then use this information to tailor their program content, deliver, and supports to the specific social-emotional needs of the students whom they serve.
Youth Outcomes Membership Level
All responsibilities of Basic Membership and Program Quality Membership apply. Additionally, the program partner assesses individual students’ skills using the Survey of Academic and Youth Outcomes – Teacher Version (SAYO T). Educators at the program complete a Pre SAYO T at the beginning of the program and a Post SAYO T at the end of the program for each student. Pre results may be used to tailor program content and delivery to the needs of students, and post results reveal whether or not students demonstrated growth in their social-emotional skills over the course of the program.
Program partners must participate at the Program Quality Membership Level for at least one program cycle in order to be eligible to participate at the Youth Outcomes Membership Level.
Badging Membership Level
All responsibilities of Basic, Program Quality, and Youth Outcomes Membership apply. Additionally, program educators award badges recognizing students’ social-emotional skill development based on attendance and SAYO T results.
Program partners must participate at the Program Quality Membership Level for at least one program cycle in order to be eligible to participate at the Badging Membership Level.