Lunch & Learn series

Though we cannot host our Student Success Conference in person in 2021, we hope to provide the UNC System community with some of the benefits of the annual conference and share the great research coming out of the Student Success Innovation Lab and Math Pathways Initiative. Throughout this spring and summer, the UNC System Office will host 60-minute Zoom events, offering our practitioners and researchers an opportunity to present their projects and findings.

Presentations will allow ample time for discussion, and cover the following topics:

  • Problem Statement: Summarize the challenges the student success initiative addresses.
  • Initiative Description: Describe the intervention and the specific populations on which the initiative focuses.
  • Research Questions: Identify the research questions the research design addresses.
  • Research Design: What experimental or quasi-experimental design are being used?
  • Outcomes: Explain the measurement outcomes and data collection methods. Explain the proposed impact the initiative will have on student outcomes and share any preliminary outcomes.
  • Implications: Share what (1) practitioners, (2) researchers, and (3) policymakers should take away from the project and research.

All events will occur on Zoom, from 12:00pm – 1:00pm ET, be recorded, and feature live transcription (download instructions). Attendees will be off-camera and muted. Presentation slides will be posted on this page a day prior to the event.

Advanced registration is required and capped at 500 participants. To join the Zoom meeting through the unique ID in your registration confirmation email, you will need to download the Zoom client for computers or the smartphone Zoom application.

Registration button for Lunch and Learn webinars.

Advising 2.0: Measuring the Impact of an Academic Case Manager at UNC Asheville

In fall 2019, UNC Asheville addressed a declining first-to-second-year retention rate by assigning an academic case manager (ACM) to higher risk students. This two-year SSIL-funded project evaluates the impact of the ACM on students’ earned credit hours, term GPA, and year-over-year retention using a randomized control trial.

First-year, first-time students with standardized tests below the class mean and not served by other support programs were randomly assigned to a control or treatment group by an external evaluator. The ACM meets with students in the treatment group multiple times a semester, helping these students navigate college expectations, solve problems as they arise, and stay on their path to degree completion.

While all professional advisors assist students experiencing difficulties during the academic term, the ACM offers a more holistic, proactive, and sustained approach than academic advisors. Using a standard intake form, an iterative approach, and methods based in person-centered theory, the ACM supports students for up to two years with a range of assistance, from exploring strategies for academic success to locating resources such as food and housing assistance or connecting with appropriate mental health services. Because their connection is sustained over multiple meetings and semesters, the ACM functions as a student’s on-campus mentor and advocate in ways that exceed the capacity of even the most dedicated professional advisors.

Presenters
Initiative Team Lead: Mr. Deaver Traywick, Interim Senior Director of Academic Success, UNC Asheville
Academic Case Manager: Ms. Azure Samuels, Academic Case Manager, UNC Asheville
Research Affiliate: Dr. Angela Dills, Professor of Economics, Western Carolina University

Event Details
Wednesday, February 24, 2021
12:00 – 1:00 pm

Download Presentation Slides

View Recorded Presentation

Additional Questions

What did Asheville do?
In fall of 2019, the University of North Carolina Asheville hired an academic case manager to advise a randomly-assigned group of first-year students who had SAT or ACT scores below the entering class average and were not served by other support services. UNCA randomly assigned 100 first-time students from the entering classes of fall 2019 and fall 2020.

How was the study designed, and what did it measure?
Academic outcomes for the treated students, including credits earned, GPA, and retention—were compared to those of a control group. For the first cohort (fall 2019), outcomes were measured over the course of ~16 months (beginning of Fall 2019 through the end of Fall 2020), while outcomes for the 2020 cohort are only available through the end of their first semester.

Did the intervention increase retention?
Results for the 2019 cohort indicated significant, positive increase of 7 percent in the Fall to Spring retention rate. When treated students were tracked into fall 2020, the study found an 8 percent increase in Fall-to-Fall retention. Both of those retention gains were statistically different from zero. The study estimates that the tuition, fees, and room and board revenues generated by the increase in retention between fall and spring was sufficient to cover the yearly salary and benefits for the academic case manager. By the end of Fall 2020, the study measured a 5% increase in retention among treated students, though that difference was not statistically different from zero.  

For the Fall 2020 cohort, the effect on retention was not available at the time the presentation was made but will be reported when those data are analyzed.

Did the Academic Case Manager increase credits earned and GPA’s?
Among the Fall 2019 cohort, the study found significant, positive impacts on GPA and credits earned in Fall 2019. However, the study also found that some of those positive effects seen early in the first cohort faded over time. In one case (credits earned), the effect seemed to reverse by Fall 2020, when treated students who had returned for their second year earned one fewer credit than control students who had returned for their second year, a statistically significant difference. However, it is important to note that this finding is contingent on whether students who dropped out prior to Fall 2020 are included in the analysis or not. If stop-outs are included and considered to have earned zero credits in Fall 2020, there is no statistical difference between treatment and control groups on credits earned. After 3 semesters, the treated students earned 2.5 more credit hours than the control students, on average. But this difference isn’t statistically significant.

The benefits to the Fall 2020 cohort appear to be weaker on GPA and credits earned, with no statistically significant differences across treatment and control on either metric. The academic case manager hypothesizes that the changes to campus brought on by the pandemic, including limits on meeting in-person, made it more difficult to build rapport and relationship with treated students from the start of the semester.

Why report mixed results?
A bedrock principle of the Student Success Innovation Lab (SSIL) is that rigorous evaluation studies produce valuable evidence that can inform decision-making whether they find a positive, null, or negative effect. In cases where evaluations find positive impacts, we can use that information to understand how and why it worked and for which students and identify opportunities to take that idea to scale. Where evaluations find no impact (or negative impact) on a particular outcome, we can use that information to better understand the obstacles to student success and, in some cases, move on from interventions that are not effective or cost-effective.

Evaluating Classroom Supports: Drop-In Tutoring Lab Program at Appalachian State University & the Learning Community Program at East Carolina University

A subset of SSIL grant awards enabled institutions to evaluate the effectiveness of existing initiatives to inform institutional and System-level decision-making about “what works” to improve student success. This session will highlight the evaluation results from two studies:

Appalachian State University – Drop-In Tutoring Labs Program
This study examined the correlation between time spent in one or more of the Drop-In Learning Labs and student course GPA. Additional qualitative research explored students’ reasons for participation, perceived and actual benefits received and potential barriers for student learning.

East Carolina University – Pirate Academic Success Center’s Learning Community Program
This study analyzed whether enrollment in PASC’s learning community program results in greater retention at sophomore year, academic performance throughout students’ college career, and years taken to graduate.

Presenters
ASU Evaluator: Dr. Alison LaGarry-Cahoon, Clinical Assistant Professor, School of Education, UNC-Chapel Hill
ECU Evaluator: Dr. Aubrianne Rote, Associate Professor of Health and Wellness, UNC Asheville

Event Details
Friday, March 12, 2021
12:00 – 1:00 pm

Download Presentation Slides

View Recorded Presentation

Living & Learning Communities at North Carolina A&T State University

N.C. A&T has been undertaking a variety of efforts to improve outcomes for their students. One such effort is the Aggie Success Academy (ASA), a comprehensive summer bridge program that allows students to earn course credits and participate in activities designed to lay the groundwork for college success. Another is the Living and Learning Communities in which groups of N.C. A&T students are housed together and receive supplemental programming that support their academic and career interests. Both of these efforts have been formally evaluated.  

The first year of ASA was evaluated using a quasi-experimental design in which ASA participants were compared to similar students who did not enroll in ASA. Results showed a large positive impact on the percentage of students who finished the year with more than 30 credits, putting them on track for graduating in four years. 

Data on the Living and Learning Communities were analyzed on 2,398 students over three academic years (2016-2019) to determine if enrollment in these communities resulted in greater retention and academic performance compared to a similar sample of students. Qualitative data were also collected to examine trends in experiences among LLC advisors. Results demonstrate higher academic performance and retention among students in an LLC compared to a similar group of students not in an LLC. In addition, this assessment provides strong evidence for increased support for LLCs at N.C. A&T in the form of funding, personnel and leadership support and training.

This presentation will describe the interventions, present results from the evaluations, and discuss how these interventions are being re-envisioned for a COVID-impacted world. 

Presenters
Aggie Success Academy
Initiative Team Lead: Dr. Regina Williams-Davis, Assistant Provost for Student Success and Academic Support, North Carolina A&T State University
Research Affiliate: Dr. Julie Edmunds, Director, Secondary School Reform Program at SERVE Center, UNC Greensboro

Living and Learning Community Assessment
Evaluator: Dr. Aubrianne Rote, Associate Professor of Health and Wellness, UNC Asheville

Event Details
Friday, March 26, 2021
12:00 – 1:00 pm

Download Presentation Slides

View Recorded Presentation

Removing Barriers to Access: Completion Grants Across Four UNC System Institutions

The UNC System Student Success and Innovation Lab (SSIL) created the Completion Grant Pilot Program (CGPP) to improve undergraduate degree completion by students from families with low incomes. CGPP was structured to increase the likelihood of graduation by reducing financial stressors that can lead to stop-out and better preparing students for graduation and their future careers. Eligible students nearing graduation were awarded a small, just-in-time completion grant of $1,500 to fill financial gaps in the cost of attendance. As a condition of the grant, these students were required to complete two future-building activities such as completing online financial literacy modules or performing a mock interview. SSIL contracted with Insight Policy Research to conduct a randomized controlled trial (RCT) designed to estimate the effect of CGPP on graduation status relative to the status quo financial aid packages.

Presenters
Completion Grant Coach: Tina McEntire, Vice Chancellor for Enrollment Management, UNC Greensboro
Completion Grant Coach: Katie Ramstack, Project Manager for Enrollment Management, UNC Charlotte
Evaluator: Albert Liu, Director of Education, Insight Policy Research
Evaluator: Jackson Miller, Senior Research Analyst, Insight Policy Research

Event Details
Tuesday, April 27, 2021
12:00 – 1:00 pm

Download Presentation Slides

View Recorded Presentation

UnC System Student success conference

The annual Student Success Conference focused on three areas aimed at improving student success: financial aid; teaching and learning; and student advising and support. The topics highlighted innovative work happening at our institutions in a variety of areas, including summer financial aid, food insecurity, resources for veterans, academic advising and coaching, large lecture classroom models, and using data to drive student success.

2020 Student Success Conference
Date:             February 12, 2020
Theme:         The Student-Ready University: Removing Obstacles & Building Support
Location:       Friday Conference Center, Chapel Hill
Learn More:  Website | Event Program  |  Presentations  |  Article: Student Success Today, Even More Tomorrow

2019 Student Success Conference
Date:             February 7, 2019
Location:       Friday Conference Center, Chapel Hill
Learn More:  Event Program  |  Presentations  |  Article: Closing the Gap