Data Insights
Qualitative Case Study
First phase of a design research project to create on-demand data visualizations & analytics for university instructors to track the health of their course during peak disruption from COVID-19.
The outcome of this work was a set of data visualizations that process and visualize course data for instructors. Data Insights instantly became a market differentiator for Labflow.
"It makes my job a million times easier"
— Dr. Megan Rudock, UNCG
Challenge
Laboratory instructors face challenges keeping tabs on the quality of implementation of their courses
COVID-19 exacerbated these problems, with laboratory courses being especially hard hit due to difficult remote facilitation
Labflow was already collecting a wide range of data for the core application that could serve as the basis for broader course analytics
Data Insights would give instructors high visibility into their courses and address a need in the market
This also would provide the company with a significant competitive advantage, since course analytics in off the shelf LMS/courseware solutions are very rudimentary
Dane setting out to solve a big business challenge
There was a sense of urgency to build a solution quickly to help alleviate the pressure that university science courses were experiencing during the pandemic.
I set out to figure out to understand what "course health" meant in practical terms and how data could be foregrounded and presented to address the needs of lab instructors.
Objective
As a Mixed-Methods UX Researcher & Software Engineer, I worked across product and engineering to define the secondary use goals of our platform data and what analytics they could support.
Built relationships with medium and large universities as we iterated and honed Data Insights.
Designed a research & development plan for building, validating, and iterating on Data Insights.
The company had a goal to launch an MVP of course analytics in the platform in a short timeframe to address the pressing need in the market due to COVID-19 disruption.
Project Outline
This project took place between Dec. 2020 – Apr. 2021. At the end of the design research cycle, we released a beta version of Data Insights to early adopters behind a feature flag.
📏 Scope
Identify the gaps in administration of university science lab courses
Map gaps in course administration to data in Labflow's transactional databases
Report what core tasks analytics could be used to measure course health
Design, iterate, and validate a beta release for Data Insights
📦 Deliverables
Task analysis of lab course administration
Concept & code prototypes of data visualization of course analytics
Beta release of Data Insights
👥 Roles
Mixed-Methods UX Researcher
Software Engineer
What “data windows” would make grader behavior and student performance visible to instructors?
Research Objectives
Uncover the key facets of "course health" for university science labs and how they are quantified
Identify the tasks that lab instructors want to perform with course data to monitor course health
Transform insights into production ready interfaces for a beta release of Data Insights
Research Methods & Findings
Method: Expert task analysis
Findings: Tracking good course health requires data that let you assess
Effective course administration (e.g., monitoring grading progress)
Good student engagement (e.g., regular engagement with course materials)
Quality instructional experience (e.g., minimal group differences by grader or section)
Excerpt of the task analysis of "course health." Colleagues who formally ran labs were asked to comment and validate.
My task analysis identified three foundational pillars of a course that need to be in good shape to support good course health:
Course administration
Student engagement
Quality of instructional experience
Through an iterative process, I produced a number of designs to address our customers' needs. Here are some of the designs that emerged from the task analysis:
Dane feeling tension between simplicity and comprehensiveness
Should course health be reduced to "a single number" or be more multifaceted?
By focusing on tasks to be done, I kept coming back to a consistent need for data to only be minimally rolled up along specific dimensions, with more complexity available on drill down.
Data Insights debuted at the American Chemical Society (ACS) conference.
Demo video of the first Data Insights dashboard. Presented at American Chemical Society (ACS) Spring 2021.
Research Impact
Data Insights became an instant cornerstone of sales pitches and demos
Faculty reception overwhelmingly positive
Determined with product team that followup usability research would be necessary
Sales associate shares how Data Insights made it easier to open new opportunities
"Data Insights is probably our biggest, most unique differentiator. It opens up so many conversations with the big [universities] we want to work with."
— Elizabeth Crowe, Account Development Scientist
University lab coordinator, adopter of Data Insights
"It’s nice to see all the grading complete! I appreciate the Data Insights tool [it] helps me find those errors easier. I feel like I am a little ahead this term. 🙂"
— Mary Jo Smith, Gran Valley State University
Dane thinking about the next steps for the evolution of Data Insights
The launch was a success. But we now had a golden opportunity to slow down and reassess:
How could we improve the usability of Data Insights?
To read more, please visit my mixed-methods case study on the usability of Data Insights: