Data Insights: Usability

Mixed-Methods Case Study


Second part of a design research project to improve the usability of Data Insights based on tasks to be done, usability interviews, and web analytics

The outcome of this work was a redesigned user interface and experience. Design improvements led to a sustained >50% y/y growth in usage.


Dane focusing on enhancing the usability of Data Insights

This presented me with another business challenge

How could we ensure the continued growth in use of Data Insights by instructors?


Project Outline

This was a multifaceted project that transpired between the Spring of 2022 and late Winter 2023. We hit an unanticipated snag around our intended release in Dec. 2022 and decided to invest significant resources in redesigning the data model that underpins all course analytics.

📏 Scope

📦 Deliverables

👥 Roles

Mixed-Methods UX Researcher
Software Engineer


How can we improve the usability of Data Insights to ensure that it empowers instructors to run their courses in a data-driven way?

Research Objectives

Research Methods & Findings

1. Usability Interviews

Show data over time

Some issues in course administration only become obvious over time. Things like falling behind on grading or disparities in student scores stand out more clearly when shown with time as dimension.

Provide tools to explore data

Instructors often wanted to what data they are looking at to isolate a few key groups like specific graders or sections. These tools were available in the interface, but not placed where users expected.

Enhance visibility of outliers

Outliers are important to know because they map to groups of students at a disadvantage. Making outliers easy to isolate, especially in large courses, is vital for instructor efficiency.

UX Research Data Insights

Complete presentation on the usability findings from user interviews

2. Design & Development

Usability interviews highlighted a number of changes and new views that instructors wanted of their data. I channeled these insights into a number of enhanced visual and functional designs.

Evolution of the Grading Status tab, with initial prototype (left) and production-ready (right) iterations

3. Data Model Redesign

Sample of the process to develop a new data model for Data Insights

Dane recognizing that meeting our users needs required a completely new data model

One of the key insights from usability interviews was that critical data needed to be visible over time.

The data model we had, though, was only transactional—records would be updated and any change history lost.

I worked closely with engineering and QA to completely revamp how our data collection worked to support our users' current needs and serve as the basis for future analytics.

4. Establish & Define Web Analytics 

Collaborative activity to define common instructor goals for using Data Insights to check course health

Aligning user goals to the HEART framework

As a complement to my qualitative research, I worked with the product team to articulate what core goals an instructor would want to accomplish in using Data Insights. I then connected each of these goals to a web analytics metric in the HEART framework.

These measurement targets were sorted into ordinal tiers Level 1, 2, or 3 based on the order I would build up the appropriate measurement & analysis infrastructure. 

Data Insights Quant UX metrics (Summer 2023)

Detailed overview of Level 1 metrics captured with Google Analytics and interpreted within the HEART framework

Research Impact

Annotated Score Distribution component highlighting unified interface controls design

Steady uptick in Data Insights adoption, accelerating noticeable since the release of UX improvements

I also developed training materials to support our instructor users to interpret and effectively leverage their course data.

Example of the training materials ecosystem developed around Data Insights

Dane looking to the future of measuring Level 2 & 3 analytics

Level 2 (data interaction) & Level 3 (data synthesis) move beyond basic engagement metrics to ask whether Data Insights has everyday utility for instructors. 

This means the next phase of quant analytics will need a new strategy. 

Stay tuned...