Mea Culpa – Accessibility Concerns of Using External Tools in the LMS

Over the past 10 years or so, my most popular conference workshops have been those involving the use of Web 2.0 tools inside the LMS. I’ve always enjoyed those workshops immensely, and it’s fun to see the light bulbs come on as they discover ways that they can engage their students in new and different ways.

Gravity is a lot like accessibility - it's the lawI gave another one of those workshops in November 2014, and it is probably the first time I’ve done so in the last year and a half. The topic has become a bit dated and I usually had different topics to explore instead. As I was making the recent presentation, it struck me how totally different it was now compared to how it used to be. My focus was completely different, and with very good reason.

There were two things that I focused on that never used to be part of the presentation:

1) Due to browser changes, I now focus on the web sites that that don’t create a mixed content issue when embedding content inside the secure (https) learning platform (learn more here), and

2) Due to a change in priorities, I now focus on the web accessibility issues of using external content inside the learning platform. It is this second point that is the focus of this post and a few additional posts to come.

When giving this presentation 5-10 years ago, questions about accessibility would occasionally come up, but not often. Although I don’t specifically remember what my responses were at the time, I do know that they were not adequate. There have always been serious issues related to accessibility for online learning content/activities, but years ago it was easier to blow them off as not being a high priority. These days I consider it to be one of the highest priorities. Yes, I guess I had a revelation about accessibility somewhere along the way.

So here’s the deal. If you are creating course content using an external tool, or if you are having students do required work using an external tool, then it is incumbent upon you to make sure that you are using creative output and/or external tools that are fully accessible to students who have disabilities.

I’m currently wrapping up facilitation of the first offering of the Web Accessibility MOOC for Online Educators (WAMOE). Throughout this MOOC, we’ve focused on creating accessible course content using HTML pages, Word Docs, PPT slideshows, and PDF docs. That’s all well and good – but if you then introduce external tools, you need to make sure that they are accessible too; and many of them are not.

In a series of undetermined length, I’ll examine the accessibility features (or lack thereof) of many of the popular external tools used in eLearning. I’ll probably say it repeatedly, but it’s okay to use a non-accessible tool as long as you make appropriate accommodations for students who are not able to use the tools. For the tools that reduce accessibility, I’ll also give one or more suggestions for a work-around that should be put in place if you really want to use the inaccessible tool.

Advertisements

Two More Things that Bug me About Quality in e-Learning

Last week I published the first post in this series of undetermined length. Here is post number two. As a recap, bug Number One was the lack of a definition for the term e-Learning Quality. Bug Number Two was the excessive focus on course design and the lack of focus on more important pieces (IMO) of e-Learning Quality. Here are the next two bugs.

Three.

This one goes hand-in-hand with the issue about defining quality. We need to look at quality as a continuum, something along the lines of the graphic below.

Online Learning Quality Continuum

When people use the term “eLearning Quality,” we usually assume they mean high quality or excellent quality, but of course there can also be low quality. In other words, the term “quality” is really not descriptive enough. Just adding the word assurance, such as in a quality assurance program, still doesn’t cut it for me. “I assure you that your quality is really crappy.” How’s that for quality assurance?

We would be better off to talk about Excellence in e-Learning, or my preferred term of Excellence in e-Education. The other thing that the chart illustrates is that most quality standards tend to aim at the large middle section – which is indicative of being good enough, rather than excellent. It’s really not that high a bar. My suggestion is that you stop talking about Quality, and start talking about Excellence.

Four.

Very often a discussion about e-Learning Excellence comes around to a comparison of course completion rates between online courses and F2F courses. That focus on completion rates is seriously misplaced. It’s a classic case of comparing peaches to pears.

Completion rates expectations.

Of course it makes perfect sense to most people when that comparison is made. Those people believe that if online learning results in fewer successful course completions, it must be due to the lower level of quality for those online courses. What other reason could there be? Well, there could be several other reasons. Here are some of those reasons:

1. Demographics differences (Learning House Report pages 27-31)

We know that there is a fair number of students who mix F2F classes with online classes. But we also know that those students who are exclusively engaged in online learning, taken as a group, have some different demographics than those who engage exclusively in F2F learning. In other words, they are not two homogenous groups. Online students work more hours (60% of them work full-time), are more likely to be parents (and especially single parents), and are more likely to be part-time students rather than full-time. In fact, those demographics differences are some of the reasons why students choose an online education path in the first place – because their plates are already pretty full. See also the demographics differences from my former college.

2. Differences in student support

All students need many support services, and the on-campus services are typically better delivered than the comparable online services. Sure, some schools are doing an exemplary job of providing online access to a full range of student services, but many others are not.

Online students expect 24/7 access to these services (L/H page 20), but see a significant performance gap (Noel/Levitz pg. 6) in those services provided by their colleges and universities:

  • Online library and research
  • Technical support
  • Academic advising
  • Career placement

3. Differences in instructor preparation/familiarity

Every term there are lots of faculty members who teach  their  very  first  online class. Sure, there are also some faculty who are teaching their very first F2F course, but not to the same level as the number of noobs for online teaching. And besides, chances are good that the first-time F2F teacher has been sitting in F2F classrooms as a student for 15-20 years before their first attempt at teaching in the same setting. It’s not unfamiliar territory.

At my former college, I tracked the student final course grades in classes that were taught by new online faculty over their first few semesters. This was not part of their evaluation, nor was it used in any significant way with, for, or against the instructor. I was simply curious. What happened to student performance as the instructors became more experienced with online teaching? I doubt that you’ll be surprised that student achievements (as measured by final grades) generally  got better as the instructor gathered experience with the modality. The chart below is based on real data collected about dozens of faculty members who started teaching online courses during my tenure at the college. These three anonymized instructors are representative of the typical data – although there were a few exceptions to this general upward trend. Loosely speaking, I think the data show that instructors see better results from their students as the instructor gains more experience in online teaching. I don’t think that’s surprising, but I do think it’s something that we need to consider about the maturity of online education. We get better at it the longer we do it. At least most of us do.

Student grades with new online instructors

NOTE:  this chart depicts the percentage of students who successfully completed (passed) an online class during the first five terms that an instructor taught that class online. In each case, this was the instructor’s first foray into online teaching.

4. Differences in student familiarity with online learning

Much like newbie faculty members, there are also a new batch of first-time online students each and every term. Every fall term at my former college, about 40% of the online students were first-time online learners. No matter how many years of schooling they had endured, this was their first taste of online learning. That 40% figure remained fairly constant from about 2005 to 2010, when I left the college. Nationally, the rate is probably below that number, and it is probably slowly going down over the years as online learning is less and less of a novelty.  I’d say that a conservative estimate of the overall rate is 25%. In other words, every time a new term starts, at least 25% of the online students are taking their first-ever online class. It’s probably a higher rate, but 25% is high enough to make the following point.

At the beginning of each term, what percentage of students sitting in a traditional F2F classroom are doing so for the very first time?

Effectively zero percent. Let’s allow for the occasional home-schooled student who has never been in a traditional classroom – not ever. It could happen. So, let’s conservatively estimate that about 0.001% of the F2F students are sitting in a F2F classroom for the first time ever.

And yet we expect those two groups to achieve the same level of success in those courses? Doesn’t that sound just a little bit insane?

Comparing completion rates between online and face-to-face

Click to enlarge

Online course completions continue to get better and better, as we figure out how to be more effective online educators and as students learn how to be better online learners. If anything, the e-Learning Atheists (naysayers) should be concerned that the difference in rates is so small, and getting smaller. More about those improving completion rates next time.

Two Things That Bug Me (Today) About Quality in e-Learning

eQuality iconOver the past ten years or so, I’ve been involved in one way or the other in the arena of assessing quality in Online Learning. It was 2004 that I first learned about the work being done by QualityMatters during the first year of their FIPSE grant. As the Dean of Distance Learning at a Minnesota College at the time, I attended the ITC eLearning conference and brought back version one of the QM rubric. At a faculty retreat that spring, the online faculty at my college decided that they wanted to revise the rubric for local considerations, create their own faculty-driven review process, and pursue a voluntary quality review process for online courses at the school.

That process at the college is still going strong now 10 years later, even though I’m long gone. During my time as an independent e-Learning consultant, I received several contracts to work with colleges on their e-Quality initiatives and for accreditation for online programs. I’ve had first-hand experience with many different schools and have examined what they are doing (and what they are NOT doing) with regard to assuring and/or increasing the quality of their online education offerings; including courses, programs, and student services. In other words, I’m neither a noob nor a rube when it comes to e-Learning quality.

As I continue to find new developments in the e-Quality world for my Scoop.it page on such, I continue to be amazed at how little there is that is new in this field. Mostly the same old conversations about the same old issues. Does that mean that we’re done here? I doubt it, but not quite sure what else to make of it. There are several things that continue to bug me about the conversations around e-Quality. In this post I’ll mention the first two things that are bugging me. More posts will follow shortly until I’ve sprayed all the bugs (or just run out of things that bug me).

One.

To the best of my knowledge, we still don’t have a widely accepted definition of what we mean when we use the term e-Quality (my term for e-Learning Quality, Online Course Quality, Distance Ed Quality, etc. etc.) Heck, we can’t even agree on the term that we use when referring to it. The word ‘quality’ itself is not that muddy: “a high level of value or excellence” says the online Merriam-Webster dictionary. But specifically, what do we mean when we refer to quality in an online education setting?

For comparison, let’s consider how quality is defined in a different arena where the concept is applied on a regular basis: manufacturing/business. What does the International Organization for Standardization (ISO 8402:1986) have to say about this? The standard defines quality as “the totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs.”

Another definition from Business Dictionary dot com: “In manufacturing, a measure of excellence or a state of being free from defects, deficiencies, and significant variations, brought about by the strict and consistent adherence to measurable and verifiable standards to achieve uniformity of output that satisfies specific customer or user requirements.”

I’m pretty sure that almost all educators would be uncomfortable  using that sort of language when talking about educational quality. Free from defects, uniformity of output – yikes. So let’s try something like this:

Excellence in e-Education includes high expectations for all parties involved, clearly communicating standards upon which evaluations will be based, gathering and reporting qualitative measurements of student learning achievements, while operating efficiently and providing good value for the costs incurred by all stakeholders.

Two.

The almost-singular focus on course design is seriously misplaced. Some of you are probably familiar with this diagram I developed about 6 or 7 years ago. The three triangles indicate three potential areas of eLearning Quality Trianglesassessing “quality” in online education, specifically at the course level. My mantra during the past has been that there is an unbalanced focus on the lower right triangle  with little, if any, attention being paid to the other two triangles. I have noticed a slight uptick in the attention paid to the quality of instruction and the quality of student learning achievements, but those areas still pale in comparison to the focus on course design. Most people believe that if they have a QM (or QM-like) rubric and process, then they’re answering their critics’ questions about online course quality.

Those of you who’ve heard me speak on this matter know that I consider the course design to be several magnitudes of lesser importance than the other two areas. In fact, if you can provide solid, repeatable evidence that the online students are achieving the intended learning outcomes at a very high level – then I’d be willing to throw out the other two altogether. That’s rarely the case. Let me restate that: if your online students are learning what they are supposed to learn, then I’d rather not dictate certain teaching techniques or course design standards, because what they are doing must be working. It’s nice to have evidence of all three – as long as your use of a design rubric doesn’t become a one-size fits-all approach.

My suggestion is that you focus on gathering evidence for the blue triangle. If that evidence is weak, then look at the other two triangles to identify potential causes (there are other potential causes, of course). If the evidence of success in the blue triangle is strong, don’t worry so much about the other two.

I’m just getting started. Check back soon.

I Say Bull ____ !!

I haven’t been posting here much lately, but there are so many things that I want to say about what I’m seeing in this increasingly insane world, that I feel compelled to start writing some of it here – if nothing else for my own I say bull, and I know it when I see itsanity (what’s left of it).

Have you noticed the explosion of infographics on the web lately? Are you still breathing? If so, then you’ve noticed. You can find lots of very positive (glowing, in fact) web articles about how to use infographics as link bait. That is the number one purpose for an infographic, to drive traffic to somebody’s website. I get several emails each week asking me to embed/share some infographic on my site. The emails are slowing down now since I don’t respond (positively) very often, and also because if I choose to respond it is usually to tell them how wrong they are.

Are you familiar with the Telephone game? Also called Grapevine, the Operator Game, or this culturally insensitive name. You’ve probably played this game from time-to-time, either as a planned activity or just accidentally. This is where you whisper something in the ear of the person next to you, they whisper it to the next person, etc. etc. until you get to the end of the line and a very different phrase (often, anyway) comes out of the mouth of the last person. That’s analogous to what I see happening more and more with the crazy explosion of infographics on the Internet. They take information (often, anyway) from a source and twist it around until you cannot recognize it any more.

Here’s a recent, classic example from one of the several SEO trolls (see Infographics as link bait) that are trying to make money out of being a middleman in the higher education (usually online learning) market. I won’t link to them because that could be a good thing for them if I drove eyeballs to their site. Instead, I’ll share a few snippets from their recent infographic on the Evolution of Online Schooling.

Infographic Henry Ford in 1950

1950, really? Seems odd since Henry Ford (THE Henry Ford) died in 1947, at least if you can believe Wikipedia. Could it be that they were talking about his grandson Henry Ford II (not HF III, which is weird)? Actually, it turns out that they are talking about the Ford Foundation. But when you invoke the ghost of “Henry Ford,” you should be talking about the real deal.

Infographic confusing CAPA with LON-CAPA

Seriously? This has something to do with international education? Someone please enlighten me. One of their sources mentions CAPA, but doesn’t say anything about “ushering in international online learning.” Another source mentions CAPA, but also says nothing about what the infographic claims. There’s a reason why this doesn’t make any sense. Turns out that there is a CAPA International Education organization, founded in 1972. The only problem is that it has nothing to do with the CAPA online learning program (now called LON-CAPA) developed in 1992. And yet thousands (just guessing) of people will look at this and think they’ve learned something.

Infographic using MOOC term in 1004

This one leaves me speechless. The first MOOC was in 1994? That’s pretty good considering that the term was coined in 2008 (see page 12). And if you’re going to proclaim something as the first thing that “sort of sounds like what we today call a MOOC,” then you can find lots of things that came earlier than this. This just shows an incredible lack of knowledge.

Infographic claims online degrees earned exceeds traditional

The little snippet above (Today) is where the Telephone Game comes into play. Into the first ear, USA Today whispers that “four big universities, operating mostly online, have quickly become the largest education schools in the USA.” Then into the second ear, Techcrunch whisper/shouts that “online education degrees now dwarf traditional universities.” At which time the owner of the third ear (that of the infographic people) ends the game by saying “Twice as many students earn online degrees as traditional degrees.” My first reaction to which is shown below:

Tweet about Infographic

To recap:

  1. USA Today uses the headline “Online Education Degrees Skyrocket” in an article about how the four institutions awarding the most degrees in the field of education are online schools. They further state that those four online institutions granted 6% of the total bachelor’s degrees in education (“one in 16”) and 9% of the total graduate degrees in education (“one in 9”). They fail to say (because they probably can’t find the data) what percentage of the total education degrees were awarded by all the online schools, or even what percentage of graduates earned their degrees online.
  2. Techcrunch cites the USA Today article in an article with the headline of “Online Education Degrees Now Dwarf Traditional Universities.” First of all, I’m not even sure that I can make sense of that statement: Degrees dwarf universities? Whatever. They quickly veer off on some mental gymnastics regarding the quality of online degrees.
  3. The infographic people cite the Techcrunch article (and all other citations have nothing to do with the number of graduates) in coming up with their incredibly inaccurate claim about how many students earn their degrees online.
  4. I smh and call Bull!!

Not all infographics are this poor, but more and more of them are. Guess there’s a lot of pressure when you have to churn one of these out each and every day, lest your page views go down. Oy.

Regional Accreditation for Online Programs

This is a repost from my old business site: Excellence in e-Education (xlents.com). That site is being shut down and I am preserving those items that I don’t want to have disappear. Originally posted April 16, 2012.


Yay! We're accredited!Recently I saw a discussion thread suggesting that there should be national standards for the delivery of online programs, rather than each of the regional accrediting bodies dealing with the issue. On one hand this may make some sense since online learning doesn’t conform to arbitrary regional boundaries, such as those drawn around the six regions.

However, higher ed has a long history of regional accreditation agencies being the authoritative bodies for higher ed accreditation without a single national body (I’m sure you already knew that). There are national accrediting bodies, but they are not as highly respected as the regional bodies.  In fact, any school that touts its national accreditation instead of regional accreditation is typically considered second-tier (not by themselves, just by others) because they haven’t been able to attain regional accreditation. Of course the national accrediting bodies will argue strenuously against that point of view. You can easily find those arguments on their websites.

I do find it interesting that online learning is one area where there has been a purposeful confluence of thinking by the regional bodies. Through C-RAC (Council of Regional Accrediting Commissions), they developed the Interregional Guidelines for the Evaluation of Distance Education (Online Learning). The new Guidelines have been endorsed by all regional accrediting organizations in the U.S.

Here is that doc: Guidelines for the Evaluation of Distance Education Programs (PDF)

E-Learning Quality – Building the Framework

This is a repost from my old business site: Excellence in e-Education (xlents.com). That site is being shut down and I am preserving those items that I don’t want to have disappear. Originally posted on January 12, 2011.


Barry Dahl presenting an e-Quality workshopWhat does the word “quality” mean in the context of online learning?

How do you recognize quality when you see it?

What steps can we take to improve our quality of online teaching and learning?

The main focus of this workshop deals with the questions related to “quality” in online learning. Quality means very different things to different people when it comes to online learning. In this workshop we focus on three different aspects of e-Learning quality:

  1. Quality of online learning
  2. Quality of online teaching
  3. Quality of online course design

We start this workshop by looking at models of good practice on online learning course design. This helps set the stage for the later pieces that focus on the quality of the learning and the quality of the teaching.

A wiki containing links to a large number of online resources is shared with attendees for their use during and after the workshop.

For a more active workshop, we also encourage the formation of teams to work on specific projects during the workshop. Some examples of group projects include the following:

  1. Peer review group of faculty for improving course design (create process for assessing quality of design)
  2. Course design rubric creation (create tool for assessing quality of course design)
  3. Learning assessment project for online students  (quality of online learning assessment)
  4. Develop formal expectations for faculty teaching in online courses (quality of teaching baseline)
  5. Online student end-of-course evaluation instrument and process
  6. Faculty evaluation process and instrument creation (quality of online teaching)

Brainstorming a Learner Flexibility Rubric

I’m hoping I can get all the smart people in my network to add to this little (asynchronous) brainstorm. For some time now I’ve been wanting to develop a rubric for measuring the amount of flexibility in online courses. I’m particularly interested in how the course design and student requirements relate to providing students with a reasonable amount of time flexibility for their online course work.

Flexibility quote by Everett Dirksen

All the data that I have collected over the years from online students indicate that time flexibility is the number one reason that they enroll in online courses. Not cost, not distance, not even preferred learning methodology; but clearly they place a premium on having time flexibility that allows them to fit higher education into their otherwise busy lives.

UPDATE (4/23): This will probably not result in an actual rubric. I’m having a hard time figuring out the lousy-good-better-best type of hierarchy that you typically need for a rubric. Instead, I’m thinking that the output will be a list of effective (or maybe “promising”) practices for ensuring time flexibility in an online course. Please keep the comments coming.

The purpose of the rubric is to come up with a way of measuring the flexibility in various online courses, and to encourage more flexibility rather than less by those who are using the rubric. I have no doubt that some online students are frustrated when they enroll in an online course with certain personal expectations about flexibility, only to find that the course offers them very little of it. These are some of the factors that come to mind for me.

Time flexibility, measured by the size of the “window of opportunity” for:

  • Quizzes and exams (how long are they open to students?)
  • Dropbox assignments (how long between start and end of assignment availability)
  • Making the required number of discussion postings per module/topic (how many days to do it, etc.)

Other factors affecting time flexibility:

  • Timeliness of quiz/exam grading (is quiz feedback received before next one is required?)
  • Stated expectations (syllabus/outline) for instructor responses to student inquiries
  • Required synchronous activities – how many, how long, and when (& are there alternatives?)
  • Required group work projects/assignments
  • Days of the week when due dates are scheduled (e.g. weekends and/or holidays)
  • Online office hours: timing of availability
  • Online office hours: modes of availability (might affect timeliness for students)
  • Added 5/3/12 from comment received: Online courses should have a rhythm or consistency to them as to when things open, when they close, how large the windows of opportunity are, etc.
  • Added 5/3/12: Appropriate amount of content
    • Too little content provided means students have to spend time hunting down relevant content that allows them to achieve the required outcomes.
    • Too much content provided typically includes content that might be on-topic, but not relevant or useful in achieving the required outcomes,; causing students to spend time filtering through the information (a case where the instructor throws everything but the kitchen sink at them).

One of the keys to this type of rubric will be to walk the fine line between enough flexibility and too much flexibility. An extreme example of “too much” flexibility would be those cases where there are no deadlines at all in the course except for the course ending date. This might work for an electronic independent study course, but not for online courses where learner interaction is expected and/or required. Based on my past online teaching experience, I find that I am in agreement with many faculty members I know who state that you must keep students engaged with the course every week (and probably 2 or more times per week) in order to not have them fall off the face of the earth (fall behind and drop out or just quit).

I’ve thought about including other types of flexibility rather than those focused on the students’ time availability. I’ve decided not to tackle those right now. For example, it might be interesting to include things related to the nature of assignments – do they allow students to choose different assignments or different methods of completing the assignment (write a paper, make a movie, make an oral presentation, etc.). I think that’s interesting, but I don’t think it captures the reason that students take online courses – which is clearly time flexibility.

Determining the specific parameters for each rubric item is yet to come. Right now I would love to have your input on other factors that could be included in a rubric to measure time flexibility for online learners. Please leave your thoughts in the comments or email me at barrydahl at gmail.

Thanks in advance, and yes, I realize this could be a colossal failure. I’m okay with that.