Interim Findings from the Detroit Promise Path Evaluation 04/2019 | Alyssa Ratledge, Rebekah O’Donoghue, Dan Cullinan, Jasmina Camo-Biogradlija
Postsecondary education is widely seen as a necessity in the modern economy, yet among low- and middle-income families, college enrollment rates are dismayingly low — and graduation rates are even lower. College Promise programs, which cover local students’ college tuition and fees, are one strategy states and municipalities use to help. College Promise programs aim to put college in students’ reach financially and to institute a college-going culture for all K-12 students. But traditionally, these programs look only to expand college access, not to address college success.
Detroit’s Promise program was designed to encourage college attendance among some of the nation’s most underserved students, those in Detroit, Michigan. The next step was to help students succeed once they enrolled in college. To do so, MDRC and the Detroit Promise partnered to create the Detroit Promise Path, an evidence-based student services program. Detroit Promise Path students begin meeting with college coaches in the late summer before their first semester of college. They are given an incentive to attend coaching meetings in the form of a monthly gift card refilled with $50 each month that they meet with coaches as directed. The program lasts all year, including summer semesters, when students are encouraged to enroll in summer classes or engage in a local summer jobs program. The entire operation is supported by a management information system.
This report presents findings from MDRC’s randomized controlled trial evaluation of the Detroit Promise Path. About two-thirds of eligible students were randomly assigned to be offered the new program, while the rest were assigned to a control group who receives the Promise scholarship alone, and thus does not meet with coaches or receive incentives. Comparing the two groups’ outcomes over time provides a reliable estimate of the effects of the Detroit Promise Path. The findings in this report include the following:
A future report will examine the program’s effect on graduation rates. It is clear, though, that Detroit Promise Path is having a positive effect on students in the first two years. This evaluation shows that building student support services into Promise scholarships can have a meaningful effect on students’ academic progress.
By Ellie Ashford April 24, 2019 Print
Promise programs are more effective when community college students are given targeted support services, according to two new studies by MDRC.
One study evaluated the Detroit Promise Path program, which provides extra support to students participating in the Detroit Promise, and the other one reviewed results from the MDRC’s College Promise Success Initiative.
The Detroit Promise, one of more than 300 promise programs nationwide, was launched by the Detroit Regional Chamber in 2013 to provide college scholarships to high school graduates for up to three years. It’s a last-dollar scholarship, meaning it covers the difference between a student’s financial aid award and the cost of tuition.
While the program resulted in an increase in students who enrolled in college, there was a concern that large numbers of Detroit Promise recipients were dropping out before their second year. To improve college retention, the chamber partnered with MDRC to create the Detroit Promise Path, which adds student services and benefits to the program, including:
The MDRC study compared two randomly selected groups: students who participated in the Detroit Promise Path and students who received Detroit Promise scholarships but didn’t receive the extra supports in the Path program.
The study covered students who attended five Detroit-area community colleges: Henry Ford, Macomb, Oakland, Schoolcraft and Wayne County Community College District.
Early results from that study suggest that “well-designed, well-implemented student support services in College Promise programs can enhance students’ experience, improve their semester-to-semester persistence in college and potentially increase the percentage of them who graduate,” MDRC found. For example, the program’s estimated impact on full-time enrollment for the full study sample increases from about 6 percentage points in the first semester to about 10 percentage points in the second semester.
This shows “there is a sizable group of students who currently enroll part-time but would enroll full-time with direction and support,” the report states.
In another example, students in the Detroit Promise Path earned an average 1.7 more credits than students in the control group, a 25 percent increase that is statistically significant.
In 2018, MDRC launched the College Promise Success Initiative to expand its efforts to help Promise programs in five other locations – Flint, Michigan; Richmond, California; Los Angeles; Portland, Oregon; and Rhode Island – design and implement student support efforts. Those efforts include coaching and advising students and plans for targeted communications with students.
One of the lessons learned from those programs is that Promise initiatives should be mapped out from start to finish – articulating the “messy middle” as well as the end goal – and should clearly spell out what kinds of information students should receive and how they should receive it.
MDRC is working with the Richmond Promise Program, for example, to create visual tools that clearly outline program components and the desired long-term and short-term outcomes related to each component, the report states.
MDRC advises Promise Program designers to carefully plan how to support consistent implementation of the model. Messages to students should be “personalized, encouraging and succinct.”
Because funds are often limited, the report urges program administrators to carefully consider where to deploy scarce resources. As an example, MDRC notes that the Oregon Promise team at Portland Community College only has one advisor, who is responsible for supporting more than 2,000 students. That advisor handles the large caseload by sending automated text messages and a newsletter with embedded videos that share information about financial aid and how to navigate college.
In another lesson from the report, “programs that build in the collection and use of data from the beginning can be in a better position to assess their progress and adjust their approaches as necessary.”
Success coaches with the Promise Program at the Community College of Rhode Island use academic data to intervene when students are at risk. For example, first-year students not on track to earn 30 credits by the end of the spring semester receive messages encouraging them to take courses in the summer.
Finally, because the perspectives of students are important, MDRC gained useful information through focus groups. Students in one focus group said they were worried about losing the motivation to return to school after winter break. As a result, the Promise staff developed engagement strategies, such as inviting students to social events during winter break, scheduling in-person check-in meetings and sending motivational text messages.