Program Impact Project

Executive Summary 

Background Information and Project Scope

The Program Impact Project is a research initiative aimed at better understanding the landscape at the University of Pittsburgh for support of first-generation, limited-income, and under-represented minority students. With the inauguration of the Pitt Success Pell Match Program and the Provost Academy, in addition to the many affordability initiatives currently in place, Pitt has made great efforts to support our students in gaining access to and better affording college at Pitt. 

At Pitt, we are committed to supporting every single one of our students and know there must also be supports in place beyond the access and transition to college phases for our students to be successful.  We recognize that some students stand to benefit from additional access to resources and programs that allow them to fully integrate and thrive at Pitt.  While we know there are many programs in place doing this thoughtful work, we sought to better understand if students have the level of support needed for the changing demographic of students coming to Pitt. 

Thus, with the help of program directors, we conducted a review and evaluation of Pitt’s existing programs so that we could (1) identify all programs that are already in place, (2) explore how intersections of this work might occur, and (3) identify gaps (if any) in services needed to support our students.  

Research Questions

  1. What programs on campus are doing the work of supporting at-promise students (e.g., first-generation, limited-income, and/or under-represented minority)?
  2. How many students can/does each program serve? 
  3. Can we map students (particularly those students awarded Pitt Success grants and those who attended the Provost Academy) back to individual programs; are students in multiple programs and receiving duplicity of services?  
  4. Is there an unmet need in student services?

Timeline

  • January 2020 – Began content analysis of current program websites
  • February 2020 – Drafted comprehensive research plan with project leads
  • March 2020 – Met with program directors:
    • Shared scope of project
    • Requested program data on participants with engagement score for last five years (AY 2015-2020) as appropriate and available
  • April-May 2020 – Received data back from program directors
  • June-October 2020 – Data analysis
  • July 2022 - Began second iteration of project
    • Requested and received project data from 2021-2022
    • Data analysis

Results

At-Promise Student Programs

Our survey revealed support activities at varying levels of the university, including efforts serving students across the university (e.g., Provost Academy), within colleges (e.g., PittEXCEL with in the Swanson School), as well as within departments (e.g., various initiatives in the Department of Biological Sciences). For the purposes of the present study, we identified four of largest, broadest organizations on campus: TRIO SSSRISEBRIDGES, and the Provost Academy.

2019 Student Program Participation

Among newly-enrolled students in fall 2019:

  • 38% of all Black students and 19% of all Hispanic students participated in at least one program. A large percentage of these students were Pell eligible.
  • 4% of all Asian students and 2% of all White students participated in at least one program evaluation. Most of these students were Pell eligible.
  • Overall, the evaluation identified approximately 900 students who had participated in at least one program over the previous two years, and we were able to match 853 students in the Pitt Data Warehouse. Below depicts the graphical breakdown by racial/ethnic group by each program.

The occurrence of students participating in more than one program was relatively low (10%), but it should be kept in mind that students may have participated in other organizations and outreach efforts that were not targeted in this evaluation.

Unmet Need

The question of unmet need is difficult to diagnose from these data. Approximately 1 in 3 new Black students and 1 in 5 new Hispanic students participated in a program. Although White and Asian students participated at lower rates, 88% White and Asian participants were Pell eligible vs. 37% of Black and Hispanic participants were Pell eligible (data from 2019).

2022 First-Year Retention and Program Participation

In 2022, the Programs provided updated student participant data. The results from this analysis support the argument that participation in a structured program to support at-promise students improves first-year retention rates. This is particularly true among historically underrepresented student populations and those from limited-income households, who, for social, political, and historical reasons, tend to shoulder disproportionate levels of belonging uncertainty. In other words, participation in one of Pitt’s programs designed to support our at-promise student populations has a positive effect on their first-year retention at Pitt.

2022 Program Impact and Pitt Transition Study Combo Effect

In 2022, we also analyzed the Program Impact Project student data in combination with the Pitt Transition Study data.  The results from this analysis show that either participation in a program or the belonging intervention has a positive effect on URM student first-year retention.

Implications

With the Pitt Success Program increasing the presence of Pell eligible students on campus, these data suggest there is opportunity to increase participation among Pell eligible students. However, future evaluation efforts would benefit from additional sources of data, such as student surveys to gauge perceptions of unmet need and a more comprehensive group of campus support organizations.

Analyses of the Pitt Transition Study and the Pitt Belongingness Intervention suggest also that programs and supporting first-year students during the transition to college are beneficial to transitional space.

Project Leads:

  • Joseph J. McCarthy, Interim Provost and Senior Vice Chancellor
  • Kenyon Bonner, Vice Provost and Dean of Students
  • Julia Spears, Associate Vice Provost for Academic Innovation
  • April Belback, Associate Vice Provost of Student Success and Advising
  • Kevin Binning, Assistant Professor, Research Scientist, Department of Psychology, Learning Research and Development Center
  • Omid Fotuhi, Research Associate, Learning Research and Development Center

Program Directors and Program Points of Contact: 

  • Provost Academy:
    • Breanne Donohue, Interim Director, New Student Programs
  • TRIO SSS (Student Support Services):
    • Michele Lagnese, Director
  • RISE:
    • Linda Williams Moore, Associate Dean of Students and Director of Student Life
    • Deena Kelly, Associate Director, Office of Student Life
  • BRIDGES:
    • Patrick Mullen, Director of the Office of Undergraduate Research, Scholarship and Creative Activity
    • Kayla Banner, Program Coordinator