How Brigham Young University-Idaho is using analytics to paint a clearer picture of student outcomes and instructor engagement
Learning leaders know analytics are important for measuring outcomes and trends, but they often don’t have the processes and tools in place to conduct effective data analysis. For many institutions of higher learning, the number crunching process is still manual and cumbersome.
The way D2L makes learner data available to you is through standardized data sets. You can access them by downloading comma-separated values (CSVs) from a user interface in the Brightspace learning management system (LMS), or you can access them via an application program interface (APIs). The data that we provide consists of more than 50 raw data sets across various learner metrics, such as user enrollments and grades, as well as event data such as course access and tool usage.
The Two Main Reasons Behind this Approach
First, by separating the data itself from the underlying distributed cloud architecture, our approach allows our platform to be more scalable and more stable than would be possible with direct access to a database mirror. This means that Brightspace can see frequent updates, changes, and improvements to deliver you the latest innovations, without impacting any integrations you may have built.
Second, we recognize that data from your learning platform is often only part of the picture. Data from Brightspace is available for local warehousing and analysis, meaning you can easily combine it with data from other institutional sources that live outside of Brightspace, such as a Student Information System (SIS), Enterprise Resource Planning (ERP) or Customer Relationship Management (CRM) platform. This gives you more flexibility in how you slice and dice your data and allows you to build reports and analyses using your own business intelligence (BI) tools to develop insights truly relevant to your institution, and not be restricted to data found in the LMS.
So how can you use this data? To answer that question, we turned to one of our clients, Brigham Young University-Idaho (BYU-Idaho). BYU-Idaho is analyzing data from Brightspace using several BI tools including Microsoft® SQL Server® Reporting Services (SSRS) Report Builder and Microsoft Power BI®.
Department- or Program-Level Reporting on Course Outcome Achievement and Learning Activity Performance
Brightspace comes with built-in tools for analysis and reporting at the course offering level, but BYU-Idaho wanted to see interactive administrative reports for outcome achievement and learning activity performance aggregated across all offerings of a course, and for all courses.
Using Course Learning Objectives Evaluation data, they can aggregate course outcome achievement data across all offerings of a given course and for all courses. This enables BYU-Idaho to show what assessments contribute to–and how they are mapped–to course outcomes. It also enables BYU-Idaho’s administrative team and academic departments to quickly identify whether students are meeting course outcomes, and where they may need extra help for mastery of certain outcomes.
Using grades data sets, they can aggregate learning activity performance data across courses–for example, the average grade, pass rate, and grade distribution. By layering in instructor feedback data, they can also see what percentage of students within a course received feedback from the instructor and the depth of that feedback. This allows BYU-Idaho’s administrative team to get a quick, at-a-glance view of how students are performing in aggregate, across multiple courses.
Monitoring Instructor Engagement Metrics
BYU-Idaho places great emphasis on ensuring its instructors are engaging with students in their courses and providing sufficient feedback.
By combining SIS enrollment data with instructor feedback data, BYU-Idaho can report on its own custom instructor engagement metrics, such as feedback-per-enrollment or percentage of enrollments receiving feedback by an instructor.
This allows BYU-Idaho’s administrative and academic teams to get a quick view of how instructors are engaging with their students. They can also identify and reach out to instructors that are demonstrating high levels of student engagement to learn what they are doing differently and then share best practices or tips with other instructors.
Visualizing Instructor Achievement Against Institutional Objectives
BYU-Idaho has some basic standards it requires its online instructors to meet with respect to interacting with a course. For example, instructors should ideally log into their courses at least five times over any given seven-day period.
By building a dashboard that pulls in various data sets from Brightspace, BYU-Idaho’s administrative team can assess how well instructors are meeting those standards.
For example, events data indicates when an individual instructor last logged into a course and can be visualized to show the distribution of instructors who have logged in once, twice, or more during the past seven days.
This gives BYU-Idaho’s administrative and academic leaders a one-stop report where they can easily see how well their instructors are meeting objectives and standards defined by the university.
Unlocking Additional Institutional Insights
These are just some of the ways our client BYU-Idaho is using BI tools to visualize the data available in Brightspace. But with a little bit of data science and creativity, there are all sorts of additional insights you can derive about your students, your instructors, and your institution.
Improving Course Outcomes
Using statistical software to combine tool usage data for instructors and grade data for students, you may be able to uncover correlations between which tools are used and how students perform. If positive correlations exist, a Department Chair could encourage other instructors to start using the same tools in their courses, to see whether that change positively impacts results across the program.
Anticipating Program Dropouts
Statistical software can help you aggregate and cross-reference course engagement metrics with student performance data such as grades. This is helpful in building an understanding of what behaviors exist within and across courses. It may also indicate a student’s likelihood of dropping out of a program. Institutions could use this data to study student retention and possibly even build predictive models to anticipate which students are likely to drop out.
Understanding Student Satisfaction
By combining data from Brightspace with data from an external course evaluation system, you might uncover patterns of student behavior and engagement that are associated with more positive course evaluations. Where such insights exist, a program coordinator could determine how to improve student satisfaction by encouraging these behaviors.
Analytics can be a key component in higher level decision-making. But what if data science and deep analytics aren’t your strength?
Our standardized approach to data access makes it easy for clients like BYU-Idaho to share their own use cases, challenges, and solutions in the Brightspace Community. And, when experts in the community aren’t able to address a particular challenge, D2L’s in-house analytics consulting experts can partner with you to help define your overall data and reporting strategy and execute on the answers.
Interested in learning more? Let’s talk!