Measuring What Matters: How Data Can Drive Outcomes for Students, Programs, and Communities

It is well-known across industries that data can, and often must, drive business decisions to yield stronger outcomes. In the business of out-of-school time programming, it is no different. Using data for improvement spans all sectors, and if the right things are measured in the right way, after-school and summer programs can leverage data to drive stronger outcomes for the young people they serve.

A new report by the RAND Corporation and commissioned by Every Hour CountsPutting Data to Work for Young People, offers insights into gathering and using data to improve decision making in the out-of-school time sector. The report features Boston Beyond, the Providence After School Alliance, and Sprockets, three coordinating agencies that have embraced data and learned how it can inform their work with youth development organizations.

Boston Beyond mobilizes nearly 300 after-school and summer programs that use data to improve the quality of experiences they offer to Boston’s youth. All of these programs aspire for their young people to thrive in an increasingly complex world. When we understand what data to collect, how to collect it, and how to use it, it can be a powerful tool to help programs achieve this goal. A shared data agenda enables programs to track their progress and learn from one another.

So what do we measure — and why does it matter? In Boston, the development of skills is central to every program in our network, and every measurement activity we undertake. Our programs have embraced the Achieve, Connect, Thrive (ACT) Skills Framework, which codifies the skills necessary for success in school, college, work, and life. These are the transferable, “future proof” skills that programs seek to develop in their youth participants. One might read these in the graphic below and recognize them as the skills that hiring managers and college admissions officers seek in their applicants. The ability to persevere, think critically, and work on a team remain at the top of the list in the hunt for top talent. Hence, we rely on research-validated tools to measure students’ development of these skills.

But data on students’ skill development is only useful if we can understand the extent to which young people practice and apply these skills. And the youth development professionals leading the programs are responsible for creating the environments that call upon these skills. That is why we measure program performance — from program structure and organization, to the interactions between adults and youth, to engagement in actual learning. In doing so, we identify how program professionals can adjust their practices, routines, and program designs in order to create more engaging learning experiences for kids.

By measuring youth skills and program performance, we empower program leaders to use data to connect their staff practices with the skill development of their students, and create action plans for improvement. Boston Beyond takes an aggregate view of all programs’ performance, in addition to providing each program with a performance dashboard, and tailors professional and peer learning opportunities based on areas of improvement.

Data for the sake of data is not a good use of time, particularly in a sector where time and resources are scarce. But in Boston we’ve found that when you measure what matters, it propels a culture of continuous improvement that can lead to life-changing opportunities for our young people in the hours outside of school.