The Complete LMS Evaluation Guide

For higher education institutions (HEIs) in the Philippines that are evaluating learning management systems (LMSs), It’s critical to align with both Commission on Higher Education (CHED) mandates and the unique needs of the local academic landscape. Deciding on an LMS is a long-term commitment that impacts the daily lives of faculty, students and staff across the campus.  

By taking the time to thoroughly understand your institution’s strategic needs and goals, you’ll be better able to select the most suitable vendor and LMS. 

Following the steps outlined in this guide will help your institution ensure that no rock is left unturned when it comes to evaluating which LMS is best for your institution. Putting in the time to complete a thorough, fair evaluation will ultimately make the adoption of the new LMS easier. 

While motivations and specific goals vary, most colleges and universities keep in mind some common overarching strategic objectives when choosing an LMS:  

1. Improve the quality of the learning experience to align with CHED standards. An LMS should promote accessibility and inclusivity for as many students as possible and enable personalised learning through flexible modalities—including blended learning—and multi-device access. It should also support outcomes-based education with individualised feedback, clear progress indicators and defined skill pathways.

2. Reduce the cost of delivery. Faculty want to teach by leveraging tools that are easy to use. Institutions can accomplish this by choosing an LMS with centralised features that reduce the number of add-ons needed to hit your goals. Faculty are also looking to work efficiently, which you can accomplish with a provider who uses something like automation to save time. Often, adoption of a new LMS includes cost savings thanks to hosting efficiencies too. 

3. Retain and attract learners. An LMS should be innovative and versatile enough to cater to both current and future students. It should support lifelong learning initiatives, while also offering robust data to improve student retention and enable proactive interventions that help students complete their programs and graduate with confidence. 

4. Implement worry-free technology with an innovative partner you can trust. Institutions need an innovative partner that provides seamless implementation, maintains reliability, offers robust support, provides context-specific training and satisfies faculty and students—helping institutions meet their goals and comply with Philippine data privacy laws and CHED requirements.

To help prepare you for choosing your next LMS, we’ve divided the evaluation process into four phases, each with examples and takeaways so you can make the best choice for your institution.  

Where to Start?

In our experience working with thousands of higher ed institutions—from being evaluated as an LMS, chosen as a provider, and supporting users throughout the adoption and its associated changes—the most common place to begin is with planning the evaluation. This is the first of four phases that we’d recommend tackling before launching a new LMS. 

The Four Phases of an Evaluation Process  

An effective evaluation process typically consists of four phases:  

1. Evaluation planning: Identify key requirements and what differentiators are important to you.  

2. Request for proposal (RFP): Release the RFP (if required) to the public.  

3. Evaluation: Review all submitted materials and calculate scoring based on your evaluation criteria.  

4. Vendor selection: Select the vendor and begin contract discussions.  

The rest of this guide takes a deeper dive into these four phases and provides guidance and tips to help you choose the best LMS for your needs. 

Phase One: Plan Your Evaluation 

Success Checklist

☑ Define your strategic goals. 

☑ Name your committee and executive sponsor. 

☑ Conduct your stakeholder survey. 

☑ Consider advice from external consultants. 

☑ Define your timelines for evaluation, decision and implementation. 

☑ Map strategic goals to your RFP requirements and a minimum threshold for shortlisted vendors.  

☑ Design the rubric for demonstrations.  

☑ Establish your overall scoring criteria.  

☑ Arrange for town hall–style demonstrations with vendors. 

☑ Keep your community informed. 

Define Your Strategic Goals

Determining your strategic goals is one of the most important parts of keeping your entire LMS evaluation on track and making it a success. Further, your goals should be linked to CHED’s quality assurance frameworks, national educational directives and desired program outcomes. Some examples of common strategic goals at a high level are: 

• improving the quality of the learning experience 
• reducing the cost of delivery 
• retaining and attracting new learners 
• working with an innovative partner 

If stakeholders get caught up in small details or distracted by shiny features, make sure to refer to your strategic goals. This way you can clearly tie your decision back to what’s most important to your institution and ensure you’re making the right choice. 

Name Your Committee and Executive Sponsor

When establishing your LMS evaluation committee, it’s critical that you consider the needs of all stakeholders and users at every level. Make sure to gather representation from all parties and incorporate use cases across campus, including the following:  

• executive sponsor  
• early adopter faculty 
• institutional success administrator  
• IT support administrator  
• director of online learning  
• continuing education personnel  
• instructional designers  
• opinion leaders  
• academic support services staff  
• student services employees  
• accessibility coordinator  

Choose team members who will work well together. Include a mix of people in technical roles and faculty from each separate college, if applicable.

Conduct Your Stakeholder Survey

Go straight to the source to understand what matters most to those using the LMS. Use a survey to zero in on specific teaching and learning needs, instead of features or functions.

For example, instead of focusing on a feature request like “a better gradebook,” get to the foot of the feedback. Ask about challenges with calculating and submitting grades in adherence with CHED grading policies. Distribute surveys to a variety of stakeholders, including: 

• students 
• faculty
• staff
• IT leadership
• accessibility coordinators
• instructional designers
• continuing education personnel 

Balancing perspectives from all your user groups gives you clues about the highest-priority needs across your campus. Including between two and 10 participants from each user group is ample and ensures appropriate representation for everyone. 

Get feedback from all kinds of students—full-time, part-time, mature, undergraduate and graduate—and from those in different departments. You can consider making the survey responses anonymous if you don’t require the participant’s use case and department. If you want the option to follow up, you can make respondents’ identity disclosure optional. If you need more clarity or information, supplement the survey with select interviews and focus groups.  

Consider Advice From External Consultants

While you know your institution and its needs best, considering advice from a third-party consulting company that specialises in the field can help you better understand the LMS market. Seeking the guidance of an external ed tech consultant can provide an unbiased perspective on market reviews and professional, strategic consulting on the different tech solutions available. 

Define Your Timelines for Evaluation, Decision and Implementation  

Starting with the date you’re aiming to be fully live, create a workback schedule that factors in specific compliance deadlines, academic calendars and other logistical challenges that could impact implementation. 

You also want to allow ample buffer time for course migration, design testing and user training, especially given potential internet or power fluctuations in some regions. 

Refer to this timeline regularly to ensure you stay on track with your goals and start date. 

Map Strategic Goals to RFP Requirements and Minimum Standards for Shortlisted Vendors

You’ll want to keep your strategic goals top of mind when it comes to your RFP. Ensure requirements explicitly mention compliance with CHED memoranda, particularly those related to online learning, blended learning and data privacy.

Ask procurement managers about specific government procurement policies or consortia relevant to Philippine HEIs. Make sure to ask your procurement manager these questions:  

1. Are there accessibility requirements or technical integrations that need to be considered?  

2. Do we require a formal RFP?  

3. Are we a member of a purchasing consortium that enables us to receive best-value pricing or skip the RFP process? 

4. Do our jurisdiction’s laws and our own institution’s rules allow “piggybacking” off another institution’s or government entity’s contract?  

You’ll also want to determine your non-negotiable elements for vendors, including data security, compliance with local regulations, and the ability to integrate with existing systems, such as the student information systems that are common in the Philippines.

Design a Rubric for Demonstrations

Develop a scripted rubric for demonstrations to ensure vendors focus on your specific needs and requirements.

Create five to 10 use cases for several user groups, and tailor the template to your unique needs. These should reflect common scenarios in Philippine HEIs, such as handling large class sizes, supporting varied student digital literacy levels, and facilitating CHED-mandated reporting. 

Set out the high-level goals of the LMS evaluation project so you have a good idea of what you want your project team to accomplish. Then, review these goals with the team so everyone is clear on their mission and purpose. Some sample goals might include:  

• Achieve the goals in our strategic plan using our LMS. 
• Increase ROI with an LMS that delivers measurable results. 
• Meet our future needs with a scalable LMS that grows with us. 
• Increase learning platform adoption rates and effective campuswide usage. 
• Attain higher student retention and completion rates with an LMS that can measure these key metrics. 
• Implement a new LMS by the required deadline. 
• Support student learning and faculty professional development through the LMS. 
• Provide an LMS that adjusts to diverse user needs and offers deep analytics for administrative insights. 
• Find an LMS that has a fully responsive design. 
• Develop a partnership with the LMS vendor to maximise our results. 

Establish Your Overall Scoring Criteria 

Your campus will have unique priorities, depending on your strategic goals as well as your stakeholders and their needs. It’ll be important to adjust and implement your scoring criteria to appropriately reflect these priorities. Weighing the importance of all the different areas of an RFP process will help during contract negotiations with your stakeholders. Knowing what’s most important to you helps determine which LMS is the best fit for your institution. 

Check out this example of how to weigh different parts of your RFP process. You can adjust these values depending on your institution’s strategic objectives, as outlined in a previous section. 

Arrange for Town Hall–Style Demonstrations With Vendors

Invite the top three contenders to visit your campus or meet virtually to demonstrate their product to your evaluation team. Prior to the meeting, send them your completed demonstration rubric and ask them to show how their product accomplishes each of your use cases.  

Revisit the list of external apps and tools requested by stakeholders in the surveys. Ask your vendors to either demonstrate these integrations or prove they can combine those tools into their platform.  

Arrange a separate presentation from each vendor’s leadership to better understand how the company will support your needs and goals—not just through their LMS software but also through their account management and other consulting services. 

In advance, share your institution’s strategic plan and goals with each vendor so the presenting teams know what’s important to your institution and can deliver a presentation that will resonate with you.  

Create a scorecard based on your evaluation criteria—such as the example previously given—and ask each member of your evaluation team to take notes during the demo and presentation. Afterwards, make sure to leave time for your team to discuss their notes.  

Keep Your Community Informed

The selection of the right LMS for your institution begins and ends with achieving your strategic goals and meeting the needs of your community, from administration to faculty to students. Throughout the evaluation process, remember to keep your community in the loop. Leverage all available communications channels, including your website and social media. 

Phase Two: Conduct the Request for Proposal

Success Checklist

☑ Write your RFP. 

☑ Arrange for deep-dive demonstrations with your top vendor candidates. 

☑ Check your references. 

☑ Conduct sandbox testing. 

Write Your RFP

Your RFP will let prospective vendors know that you’re on the hunt for a new LMS and what you’re looking for. It’ll also act as an offering to those who think they’re qualified to meet your needs. Here are some tips on how to prepare your RFP: 

• Keep your strategic goals top of mind when crafting your RFP so you don’t lose sight of what you’ve set out to accomplish. 
• Don’t focus on specific features and functions, but rather on your required teaching and learning needs. 
• Consider the different elements of your scoring criteria to help inform what you’re looking for. 
• Present recommendations in order of importance, from what you must have to what you would like to have. 
• Aim for 30 to 60 requirements. Remember, the clearer you are with your teaching and learning needs, the easier it will be for your team to evaluate options. 
• Out of these requirements, identify 10 that your LMS absolutely must have—for example, it must have responsive design.  
• Identify what you do not need and clearly determine what’s out of scope. 

Arrange for Deep-Dive Demonstrations With Each Vendor Candidate

All LMS solutions, no matter the vendor, have a broad set of features and tools to support the needs of a range of roles, from administration to course designers to instructors and students.  

Breakout or deep-dive demonstrations can go beyond the basics to address the specific needs of stakeholders and do so on a granular level. These demos are a great way to increase buy-in from the parties involved.

Consider dedicating sessions to: 

• faculty-specific needs, such as gradebook functionalities, content creation tools and assessment types relevant to Philippine academic practices 
• student-specific needs, such as mobile app experiences, accessibility features for diverse learners and ease of accessing course materials given varying internet conditions 
• administrator/IT-specific needs, such as integration with existing systems, security protocols and data analytics for reporting 
• instructional designer needs, such as course development tools, template availability and multimedia integration 

Check Your References

Ask for customer references from each vendor. Make sure the ones they provide are current users of the LMS, as they’ll have fresher insights. When you speak with the references, ask whether they can share their experiences with the vendor’s responsiveness, commitment to their road map, problems encountered and resolved, and lessons learned.  

You should also research the following for each vendor:  

• financial health and stability
• history and longevity  
• customer retention/churn rate 
• executive stability and capability  
• thought leadership  
• innovation, R&D and road map
• relevance in new trends and technology  
• expertise with implementation, course conversion and support  
• commitment to long-term partnership  

The strength of your vendor’s position in these areas will help you fully assess your partnership to ensure long-term support and reliability. This will help build your confidence that the vendor will be a good support for the product and your community. Solidifying a reliable partnership will also allow the selected vendor to grow with you toward future product development and innovation. 

Conduct Sandbox Testing

Ask each vendor to provide you with access to a sandbox—software used for demos and testing—where you can test tasks and functions inside its product. Send each vendor two of your courses: one you aspire all of your courses to be like and one that needs work. Ask the vendor to import each course into the sandbox so you can see what your content will look like and how it will perform.  

Once your vendor has created your sandbox, ask one or two people from each user group to try out the activities outlined in a sandbox evaluation document. This tool uses the identical use cases each vendor demonstrated to you but has spaces where you can document your experience. 

Another perk of sandbox testing is that it can increase stakeholder buy-in. Hearing about what a provider can do versus actually seeing it work makes a difference. This hands-on experience is invaluable for faculty and students who might be transitioning from diverse learning experiences and are looking for intuitive and reliable platforms.

Phase Three: Evaluate

Success Checklist

☑ Consider and compare offers. 

☑ Compare cumulative scoring from RFP responses and demonstrations.  

☑ Write your project report and recommendations.  

☑ Present your recommendations to the project’s executive sponsor. 

Discuss your findings and establish consensus in your final meeting with the project team. While each member of your team may have their own preferences, it is very important at this stage to stay focused on your objectives.  

Each member of your team and the project team should align their evaluation of each candidate LMS to the following, in this order:  

1. The goals you have established—don’t lose sight of what you’ve set out to do and the expectations you’ve set for potential vendors, including adherence to relevant standards  

2. Your recommendations based on the survey  

3. The vendors’ corporate stability, thought leadership and long-term partnership potential  

4. The vendor’s demonstrations  

5. The sandbox experience 

Consider and Compare Offers

At this point, the committee should do a full review and comparison of the offers from each vendor that is being considered.  

Think not only about which offers hit all your must-haves, but also the ones that go beyond your expectations. Does an offer include everything your institution requires to be successful? 

Be wary of upfront and surprise add-ons. These can range from messy integrations to added costs to achieve your non-negotiable goals. Make sure the offers you’re given are clear on what’s included and ask questions to make sure you’re getting what you need. 

Don’t forget about support. Make sure your offers are clear on the level of support your vendor will be providing and their response times. Be sure to understand whether you’ll have an assigned employee (or more) from the vendor who will get to know your business. Working directly with the vendor’s staff will start building the strong partnership needed for successful migration, adoption and growth. 

Compare Cumulative Scoring From RFP Responses and Demonstrations

Compare the scoring for each vendor’s RFP response and demonstrations according to the scripted rubric. This rigorous comparison will help ensure your evaluation is objective and justify your final decision, which is particularly important for accountability and transparency in Philippine HEIs. 

Write Your Project Report and Recommendation  

After all the time and careful consideration you’ve put into your LMS evaluation, you now have enough information to select the one vendor offering the LMS that best aligns to deliver on your goals. You can feel confident in choosing the vendor that is most qualified to support your success and meet your strategic goals. Congratulations on a job well done!  

Many institutions find that writing a report outlining how you reached your decision and publishing the document both internally and externally generates enthusiasm for the path forward chosen by your evaluation team. Your students, faculty, staff and external stakeholders will appreciate hearing about your decision and the reasons behind it. You can invite the vendor onto campus or do a virtual session to help you generate buzz surrounding your adoption of its LMS, so you get maximum exposure and buy-in from all your user groups.  

Present the Recommendation to Your Executive Sponsor  

With the evaluation complete, now it’s time to present your selection to your executive sponsor and secure their seal of approval. Remember, you’re not just presenting technical findings but also a business case. Show that your evaluation and selection process are aligned with the key objectives and strategic goals of the institution and how the vendor you have chosen meets your partnership requirements. 

Phase Four: Select a Vendor

Success Checklist

☑ Contract negotiations including IT, legal and other stakeholders.  

☑ Share and celebrate with your community. 

Contract Negotiations Including IT, Legal and Other Stakeholders  

Make sure you understand your internal contracting process. Who needs to review and sign off on this contract? What is their time frame and availability to do so? Who will be the point person with the vendor to ensure that this contract is executed in a streamlined way? 

Share With Your Community

Your community is what has driven your LMS evaluation. Show them that you’ve listened to their needs and made a decision focused on addressing the goals they helped identify throughout the evaluation process. Keep them in the loop by sharing your decision on your website and social media. Celebrating the decision will also help generate adoption and buy-in of the new platform. 

Next Steps

While choosing the best LMS is a big step, it’s not the end of your journey. Next, you’ll need to work with your new provider to develop a change management plan including your implementation and migration strategy. 

You’ll also want to have a plan of action to get your stakeholders up to speed on the LMS and manage the change to a new system. 

Just because your current LMS isn’t broken doesn’t mean it’s the best fit for your institution. Make sure to establish and reflect on the goals of your institution to ensure your LMS aligns with the standards you’re striving to achieve. 

Written by: