Using Analytics to Improve Course Engagement
As instructional designers, how can you keep a pulse on how engaged your learners are in interacting with the course content and with each other? In a classroom, you can see wandering eyes or a bored expression. In an online course, it may seem challenging to see how learners are engaging behind their computer screens, but you can use data and analytics to gain a comprehensive view of how learners are engaging with the course. In this article, we’ll discuss some indicators of low engagement in an online course and interventions tactics you can use to support learners and help them become active again.
How to View Approach Your Course Analytics
Before diving into the analytics of an online course, it is important to first understand the learners’ context so that you can understand the metrics appropriately. Factors such as the topic, course type (external or internal training), and learner motivation can lead to different expectations and benchmarks for learner engagement in a course. For example, some learners who are busy professionals may join a course without anticipating completion. Others may join a course with the goal to receive a certificate of completion. You can take proactive steps to to understand your learners’ motivations and expectations for taking the course through surveys or profile questions. Possible survey questions include:
- What are your motivations for taking the course?
- How important is achieving the course certification?
- How much time do you expect to allocate to the course each week?
By understanding your learners’ motivations, you can be better informed about which metrics and thresholds in the analytics you will want to pay attention to. This will also lead to the types of intervention tactics you may want to use. Below are three scenarios where analytics can be used to help improve learner engagement in a course. Since every course and community is unique, we recommend that you regard these as suggestions and find ones that work best for your course.
Scenario #1: In the first week, some learners are not active
We’ve seen that the first week of a course is the most important stage for intervention as learners who drop off here may never come back. What could it mean if a sizeable group of your learners are not active in the first week? First, It might mean that learners forgot that the course is starting or can no longer commit to it. Second, learners might be feeling confused about how or when to start the course.
- Learners didn’t realize the course is starting
- Learners realize they can’t commit to participate
- Learners don’t know where to start
- Send an announcement or email with action items to students, including completing their profile.
- Reach out to individuals to ask if they need any assistance
- Email targeting to reach out to learners who haven’t logged in
- Post an interesting question in the Discussion forums and respond to some discussion threads
- Send Pre-course emails leading up to the course to remind learners that the course is starting soon.
- Include an onboarding message on the homepage having a list of action items
- Have learners introduce themselves in discussions to kick-off social activity
Scenario #2: Learners are not submitting assignments on time
If your course using project deliverables towards completion, you can use NovoEd Analytics to see what percent of learners submitted, read the instructions but didn’t submit, or didn’t reach the instructions at all. By using analytics and understanding why your learners haven’t submitted assignments, you can engage in targeted interventions to help them complete the course deliverables.
What could it mean if your assignment submission targets are missed? There may be a variety of reasons, such as the level of difficulty of assignments, or a perceived lack of connection between the assignment and participants’ learning objectives.
- Assignment level is too difficult or too easy for learners
- Learners don’t see the value of assignment (not connected to learning objective)
- Learners feel that there are too many assignments or they require too much time
- Learners are not feeling engaged with the course community (social)
- Learners haven’t received enough feedback on their previous submissions to feel it is worthwhile
- Conduct a mid-course survey, ask about assignment difficulty level
- Extend the deadline/change to soft deadline or provide a catch-up week
- Assign a “course champion” to mentor/motivate a learner
- Follow-up with individuals through targeted messaging
- Highlight exemplary submissions to the community
- Create varying levels of assignments (easy/hard)
- Create a team assignment for the next course (if not already)
- Add peer evaluations for the assignment (if not already)
- Allot more time for TAs/Instructors to provide feedback
- Implement course catalyst program
Scenario #3: Teams are not forming or are not very active
If you are using teams in a NovoEd course, you can use NovoEd Analytics to see how many teams have formed and how many have joined a team. You can also see a quick snapshot of team activity within their workspaces by looking at the number of messages exchanged, meetings scheduled, and files uploaded in the Team Activities section.
What does it mean if teams aren’t forming? It could mean that participants don’t know how to create teams or don’t know how to join one. If course-long teams aren’t active, it could mean that participants don’t know how to get started.
- Learners don’t see the value of working in teams
- Learners don’t realize they can join a team or how to find them
- Learners don’t know who to ask to join their team
- Learners haven’t received response from team requests or doesn’t realize they can form their own team
- Learners don’t know how to start (a lack of defined team roles)
- Reach out to encourage individuals who haven’t formed teams
- Describe the value of forming teams (able to share knowledge, expand network etc.)
- Create a step-by-step tutorial on how to form or join teams
- Create a team formation discussion topic for people to look for teammates
- Offer discussion starters for teams to use to get to know each other
- Re-assess the way you form teams (self-formed, manually assigned, algorithmically assigned) based on the course objectives and your audience
- Determine the best time in the course to describe how teams work and when you ask participants to start forming them
- Create good profile questions for participants and teams that will be help participants choose teams and teammates
From measuring activity in the first week of a course to identifying the level of team activity, you can utilize NovoEd’s Analytics to help you identify where you can intervene to support or encourage learners to become more engaged in the course. We encourage you to try these intervention ideas, and others, as you sharpen and target the tools you use to increase learner engagement in a course