BEI Survey: Measuring Success and the Impact of Innovation: More than Butts in Seats?

This month we continue our series on innovation in part-time MBA programs with a look at how programs track success. Recall that some 80 percent of respondents in our survey indicate that innovation is critical to their success and, despite several clear barriers to innovation, there is an expressed value regarding the need to continually innovate. Given this strong value, one salient question arises centering on the importance or value that business school policy makers place on collecting information about the effectiveness or success of their programs. Without evaluation it is simply impossible to meaningfully understand the impact of a program’s efforts to innovate.

In this regard, we asked part-time MBA policymakers to indicate the importance they placed on various factors or information sources used to evaluate overall program success. Figure 1 shows the percentage of policymakers endorsing a particular factor as important for evaluating success.

Measures Figure 1

As can be seen, three factors were endorsed by more than 60 percent of the respondents.  Most notably, close to 100 percent of the respondents indicated that maintaining enrollments was a critical factor in evaluating program success. The data also show that policymakers believe that alumni feedback (81 percent) and student completion rates (75 percent) were also central to evaluating success. Not surprisingly, placement rates, media rankings and salary increases are seen as less important success indicators as these factors are often not within the direct control of a program.

Although it’s clear that policymakers do consider a handful of factors important in evaluating success, maintaining enrollments appears to be seen as the only one that really matters. In Figure 2 below, we limited the responses to only those who endorsed each factor as “extremely important.” Here, the centrality of maintaining enrollments becomes abundantly clear.

Measures Figure 2

A dominant emphasis on enrollments may seem innocuous to many; after all, the existence of a program is explicitly tied to student enrollment. However, maintaining enrollments represents a difficult paradox to manage when it comes to program innovation. For instance, it’s likely difficult to ascertain how closely linked program innovations are to observed changes in enrollments. Innovations that increase alumni satisfaction, improve career progress and increase graduate salaries are likely to create a virtuous cycle that drives enrollment changes. Further still, the hyper-focus on enrollments as the main success metric makes one wonder if the need to innovate becomes tangible only when enrollments are “down”—exemplifying a program improvement strategy that is reactive rather than proactive in nature.

Beyond Butts in Seats

Taken with results presented in earlier installments of this research series, a distinct value-gap is beginning to reveal itself. More specifically, this gap resides between the clear desire to innovate and the perceived value of measuring indicators that could help drive areas in need of innovation. Moreover, many of these areas, such as placement rates and career progress, benefit a school’s primary stakeholders—students.

This value-gap is further complicated when one considers how vital particular personal, professional and financial outcomes are to prospective students and alumni. GMAC’s surveys consistently corroborate the value alumni and organizations derive from part-time MBA programs. For instance, large majorities (over 90 percent) of MBA alumni from 2000 through 2012 report obtaining excellent career outcomes in skills (e.g., quantitative analysis), professional network development, faster career advancement and preparation for their chosen career. Put bluntly, policymakers are undervaluing the very career-related success factors that are leading MBA alumni to attribute strong value to their programs.

It is often said that “what gets measured is what gets done.” If we limit our view of success to enrollments numbers, we end up managing only to enrollments. Ironically, the consequence is a diversion of resources necessary to sustaining enrollments in the long-term toward quick fixes to maintain them in short run. To create sustainable value, part-time MBA programs must expand their definitions of success and track a broader portfolio of indicators to help diagnose and prescribe innovative interventions that keep programs attractive for the long-haul.

Click below to read this survey report as a PDF.

Business Education Insider Survey 4 – Success Measures

By Erich C. Dierdorff and Robert S. Rubin, DePaul University Driehaus College of Business

%d bloggers like this: