Body
UC Davis has long been a leader in harnessing institutional data to identify, measure, and research factors that promote or inhibit the effectiveness of undergraduate education. Many of these efforts are currently centralized through the university’s Center for Educational Effectiveness (CEE).
Part of this leadership has come through devising novel ways to get traditionally separate types of university data to work together to provide a fuller picture of the student experience. Another part comes from the development of powerful and intuitive data visualizations to display this information. Ribbon, for example, allows departments and campus administrators to look at migration in and out of STEM departments and between departments. More than 100 universities worldwide are currently using some form of this tool to visualize student migrations. CEE has also developed a Heat Map tool to identify courses with large DFW (drop, fail, withdraw) rates or courses that lead to disproportionately poor performance in subsequent courses.
CEE continues to develop tools for faculty and administrators. Many of the visualizations will soon be incorporated into the new campus Tableau server to facilitate secure delivery to department users. Know Your Students (KYS), for example, is intended to help faculty with their large courses by: 1) providing information about students’ backgrounds and tips for engaging different types of students prior to the first day of class, 2) collecting and presenting targeted information on student outcomes during the course, and 3) aggregating, analyzing, and cataloging course outcomes in a teaching “portfolio” for reflection and use in the merit and promotion process. The first stage of pre-course information has been tested and iterated while the other two stages (MIDAS - Multidimensional Instructional Development for Achievement and Success) have received HHMI funding to develop.
The Diagnostic Department Dashboard (DDD), is meant for department- and campus-level administration and displays and helps interpret information about enrollment, diversity, retention, course “hot spots”, student performance gaps, instructional staffing and more. This tool has been used formatively to assist various academic units in developing plans to improve student outcomes.
Beyond developing visualizations and tools, CEE has taken a strategic approach to using this information to bring about action. For example, DDD has been used in consultation, rather than being made independently available, to avoid misinterpretation and provide an opportunity to suggest relevant support services.
Underlying this approach is the understanding that while data by themselves may not change behavior, they can serve as the basis for conversations that will. Visualizations can help focus these conversations. Data must be understood in institutional context, and often more local context as well. While larger studies can highlight overall problems, issues, or trends, localized data (e.g., data about a single department or set of courses) is more effective and in influential in terms of bringing about specific action.