OER Pros/Cons and Evaluation Methods

Advantages of using OERs include:

  • Expanded access to learning.Students anywhere in the world can access OERs at any time, and they can access the material repeatedly.
  • Scalability. OERs are easy to distribute widely with little or no cost.
  • Augmentation of class materials. OERs can supplement textbooks and lectures where deficiencies in information are evident.
  • Enhancement of regular course content For example, multimedia material such as videos can accompany text. Presenting information in multiple formats may help students to more easily learn the material being taught.
  • Quick circulation Information may be disseminated rapidly (especially when compared to information published in textbooks or journals, which may take months or even years to become available). Quick availability of material may increase the timeliness and/or relevance of the material being presented.
  • Less expense for students The use of OERs instead of traditional textbooks or course packs, etc. can substantially reduce the cost of course materials for students.
  • Showcasing of innovation and talent. A wide audience may learn of faculty research interests and expertise. Potential students and donors may be impressed, and student and faculty recruitment efforts may be enhanced.
  • Ties for alumni. OERs provide an excellent way for alumni to stay connected to the institution and continue with a program of lifelong learning.
  • Continually improved resources. Unlike textbooks and other static sources of information, OERs can be improved quickly through direct editing by users or through solicitation and incorporation of user feedback. Instructors can take an existing OER, adapt it for a class, and make the modified OER available for others to use.

Disadvantages of OERs include:

  • Quality issues.Since many OER repositories allow any user to create an account and post material, some resources may not be relevant and/or accurate.
  • Lack of human interaction between teachers and students.OER material is created to stand alone, and since self-learning users may access the material outside of a classroom environment, they will miss out on the discussion and instructor feedback that characterize for-credit classes and that make such classes useful and valuable.
  • Language and/or cultural barriers.Although efforts are being made to make OERs available in multiple languages, many are only available in English, limiting their usefulness to non-English speakers. Additionally, not all resources are culturally appropriate for all audiences.
  • Technological issues.Some students may have trouble using some OERs if they have a slow or erratic internet connection. Other OERs may require software that students don’t have and that they may not be able to afford.
  • Intellectual property/copyright concerns.Since OERs are meant to be shared openly, the “fair use" exemption from the U.S. Copyright Act ceases to apply; all content put online must be checked to ensure that it doesn’t violate copyright law.
  • Sustainability issues.Since OER creators generally do not receive any type of payment for their OER, there may be little incentive for them to update their OER or to ensure that it will continue to be available online.

OER Evaluation Criteria Recommendations

Because OERs may vary in quality, it is important for instructors to carefully evaluate them before using them in their classroom. Criteria to consider may include the following:

  • Authority: Is it clear who developed and wrote the material? Are his or her qualifications for creating the material clearly stated?
  • Accuracy: Are there errors or omissions visible?
  • Objectivity: Is any type of bias present?
  • Currency: Is the resource up-to-date and/or is a creation or update date visible?
  • Coverage: Does it address the topic at hand sufficiently to add value to the class? Does only a portion of it apply? Do you need to combine it with other resources? Can you align each resource with the learning objectives and weekly lessons on your syllabus in order to identify gaps?
  • Accessibility: Is it ADA compliant? (Instructors planning to use OERs in their classrooms should also keep in mind that the OERs should comply with federal and state accessibility requirements. A checklist for compliance with Section 508 of the Americans with Disabilities Act can be found here.)
  • License: Has a Creative Commons License be applied? Can you remix or reuse the item? Who do you have to attribute copyright to, if anyone?
  • Persistance: Prior to using an OER in another class, you'll need to check that the URL is still valid and whether the OER was updated since you last access it.

Evaluation Rubrics, Checklist and Tool

Achieve's OER rubrics

  • Achieve.org has developed eight OER rubrics as well as an evaluation tool to help users determine the degree of alignment of OER to the Common Core State Standards, and aspects of quality of OER. More OER Rubrics training materials can be found through Archieve.org website.

[Summarized] Rubrics for Evaluating Open Education Resources Objects

  • This 2-page rubric is a synthesis version of the eight (8) separate rubrics for the evaluation of OERs created by ACHIEVE.org. It is meant as a ready reference for quick evaluation of an OER.

Achieve Open Educational Resources Evaluation Tool Handbook

  • This handbook will guide a user through the process of evaluating an online resources using Achieve OER Evaluation Tool, which is hosted on OERCommons.org.

iRubric: Evaluating OER rubric

OER Evaluation Checklist

  • A short checklist to complete when evaluating an OER.

How to Evaluate Use of OER In Your Course

There is no single method of evaluating OER quality or its effectiveness in the learning activities involving it. For many educators, the most important thing to measure is the learning outcomes. This part of evaluation is routine, since you are already evaluating learners on what they have learned. Although learners failing to acquire the knowledge and information does not mean the OER is faulty, it does raise questions about its effectiveness.

Another metric for evaluation is learner reaction. In addition to finding out whether or not learners liked the OER, find out the "whys" behind their preferences. Although the composition of classrooms change over time, you should start to see patterns in the preferences of students. This evaluation can take the form of a paper survey, in-class discussion or focus groups. Which method you chose will depend on the time you are able to devote to evaluation.

The third metric is a difficult one to measure, but it is what is often called "return on investment (ROI)." The concept of return on investment essentially asks "Was it worth the investment?" In order for measurement to be fully accurate, you need take into consideration the time taken at each part of the OER life cycle. This metric is largely subjective, as only you can measure how much your time is worth. You'll probably find that your first OER will take more time than you originally thought. It is not uncommon to have technological issues during the first implementation. This should not discourage you from future OER production and use; as you develop new skills and refine others the amount of time needed will be reduced. You should also consider how much time it would have taken you to build the OER from scratch in relation to the other costs of proprietary solutions.