Barry Dahl dot com

Last week I published the first post in this series of undetermined length. Here is post number two. As a recap, bug Number One was the lack of a definition for the term e-Learning Quality. Bug Number Two was the excessive focus on course design and the lack of focus on more important pieces (IMO) of e-Learning Quality. Here are the next two bugs.

Three.

This one goes hand-in-hand with the issue about defining quality. We need to look at quality as a continuum, something along the lines of the graphic below.

When people use the term “eLearning Quality,” we usually assume they mean high quality or excellent quality, but of course there can also be low quality. In other words, the term “quality” is really not descriptive enough. Just adding the word assurance, such as in a quality assurance program, still doesn’t cut it for me. “I assure you that your quality is really crappy.” How’s that for quality assurance?

We would be better off to talk about Excellence in e-Learning, or my preferred term of Excellence in e-Education. The other thing that the chart illustrates is that most quality standards tend to aim at the large middle section – which is indicative of being good enough, rather than excellent. It’s really not that high a bar. My suggestion is that you stop talking about Quality, and start talking about Excellence.

Four.

Very often a discussion about e-Learning Excellence comes around to a comparison of course completion rates between online courses and F2F courses. That focus on completion rates is seriously misplaced. It’s a classic case of comparing peaches to pears.

Of course it makes perfect sense to most people when that comparison is made. Those people believe that if online learning results in fewer successful course completions, it must be due to the lower level of quality for those online courses. What other reason could there be? Well, there could be several other reasons. Here are some of those reasons:

1. Demographics differences (Learning House Report pages 27-31)

We know that there is a fair number of students who mix F2F classes with online classes. But we also know that those students who are exclusively engaged in online learning, taken as a group, have some different demographics than those who engage exclusively in F2F learning. In other words, they are not two homogenous groups. Online students work more hours (60% of them work full-time), are more likely to be parents (and especially single parents), and are more likely to be part-time students rather than full-time. In fact, those demographics differences are some of the reasons why students choose an online education path in the first place – because their plates are already pretty full. See also the demographics differences from my former college.

2. Differences in student support

All students need many support services, and the on-campus services are typically better delivered than the comparable online services. Sure, some schools are doing an exemplary job of providing online access to a full range of student services, but many others are not.

Online students expect 24/7 access to these services (L/H page 20), but see a significant performance gap (Noel/Levitz pg. 6) in those services provided by their colleges and universities:

• Online library and research
• Technical support
• Academic advising
• Career placement

3. Differences in instructor preparation/familiarity

Every term there are lots of faculty members who teach  their  very  first  online class. Sure, there are also some faculty who are teaching their very first F2F course, but not to the same level as the number of noobs for online teaching. And besides, chances are good that the first-time F2F teacher has been sitting in F2F classrooms as a student for 15-20 years before their first attempt at teaching in the same setting. It’s not unfamiliar territory.

At my former college, I tracked the student final course grades in classes that were taught by new online faculty over their first few semesters. This was not part of their evaluation, nor was it used in any significant way with, for, or against the instructor. I was simply curious. What happened to student performance as the instructors became more experienced with online teaching? I doubt that you’ll be surprised that student achievements (as measured by final grades) generally  got better as the instructor gathered experience with the modality. The chart below is based on real data collected about dozens of faculty members who started teaching online courses during my tenure at the college. These three anonymized instructors are representative of the typical data – although there were a few exceptions to this general upward trend. Loosely speaking, I think the data show that instructors see better results from their students as the instructor gains more experience in online teaching. I don’t think that’s surprising, but I do think it’s something that we need to consider about the maturity of online education. We get better at it the longer we do it. At least most of us do.

NOTE:  this chart depicts the percentage of students who successfully completed (passed) an online class during the first five terms that an instructor taught that class online. In each case, this was the instructor’s first foray into online teaching.

4. Differences in student familiarity with online learning

Much like newbie faculty members, there are also a new batch of first-time online students each and every term. Every fall term at my former college, about 40% of the online students were first-time online learners. No matter how many years of schooling they had endured, this was their first taste of online learning. That 40% figure remained fairly constant from about 2005 to 2010, when I left the college. Nationally, the rate is probably below that number, and it is probably slowly going down over the years as online learning is less and less of a novelty.  I’d say that a conservative estimate of the overall rate is 25%. In other words, every time a new term starts, at least 25% of the online students are taking their first-ever online class. It’s probably a higher rate, but 25% is high enough to make the following point.

At the beginning of each term, what percentage of students sitting in a traditional F2F classroom are doing so for the very first time?

Effectively zero percent. Let’s allow for the occasional home-schooled student who has never been in a traditional classroom – not ever. It could happen. So, let’s conservatively estimate that about 0.001% of the F2F students are sitting in a F2F classroom for the first time ever.

And yet we expect those two groups to achieve the same level of success in those courses? Doesn’t that sound just a little bit insane?

Click to enlarge

Online course completions continue to get better and better, as we figure out how to be more effective online educators and as students learn how to be better online learners. If anything, the e-Learning Atheists (naysayers) should be concerned that the difference in rates is so small, and getting smaller. More about those improving completion rates next time.

Filed under: e-Quality, Online learning | 1 Comment »

Over the past ten years or so, I’ve been involved in one way or the other in the arena of assessing quality in Online Learning. It was 2004 that I first learned about the work being done by QualityMatters during the first year of their FIPSE grant. As the Dean of Distance Learning at a Minnesota College at the time, I attended the ITC eLearning conference and brought back version one of the QM rubric. At a faculty retreat that spring, the online faculty at my college decided that they wanted to revise the rubric for local considerations, create their own faculty-driven review process, and pursue a voluntary quality review process for online courses at the school.

That process at the college is still going strong now 10 years later, even though I’m long gone. During my time as an independent e-Learning consultant, I received several contracts to work with colleges on their e-Quality initiatives and for accreditation for online programs. I’ve had first-hand experience with many different schools and have examined what they are doing (and what they are NOT doing) with regard to assuring and/or increasing the quality of their online education offerings; including courses, programs, and student services. In other words, I’m neither a noob nor a rube when it comes to e-Learning quality.

As I continue to find new developments in the e-Quality world for my Scoop.it page on such, I continue to be amazed at how little there is that is new in this field. Mostly the same old conversations about the same old issues. Does that mean that we’re done here? I doubt it, but not quite sure what else to make of it. There are several things that continue to bug me about the conversations around e-Quality. In this post I’ll mention the first two things that are bugging me. More posts will follow shortly until I’ve sprayed all the bugs (or just run out of things that bug me).

One.

To the best of my knowledge, we still don’t have a widely accepted definition of what we mean when we use the term e-Quality (my term for e-Learning Quality, Online Course Quality, Distance Ed Quality, etc. etc.) Heck, we can’t even agree on the term that we use when referring to it. The word ‘quality’ itself is not that muddy: “a high level of value or excellence” says the online Merriam-Webster dictionary. But specifically, what do we mean when we refer to quality in an online education setting?

For comparison, let’s consider how quality is defined in a different arena where the concept is applied on a regular basis: manufacturing/business. What does the International Organization for Standardization (ISO 8402:1986) have to say about this? The standard defines quality as “the totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs.”

Another definition from Business Dictionary dot com: “In manufacturing, a measure of excellence or a state of being free from defects, deficiencies, and significant variations, brought about by the strict and consistent adherence to measurable and verifiable standards to achieve uniformity of output that satisfies specific customer or user requirements.”

I’m pretty sure that almost all educators would be uncomfortable  using that sort of language when talking about educational quality. Free from defects, uniformity of output – yikes. So let’s try something like this:

Excellence in e-Education includes high expectations for all parties involved, clearly communicating standards upon which evaluations will be based, gathering and reporting qualitative measurements of student learning achievements, while operating efficiently and providing good value for the costs incurred by all stakeholders.

Two.

The almost-singular focus on course design is seriously misplaced. Some of you are probably familiar with this diagram I developed about 6 or 7 years ago. The three triangles indicate three potential areas of assessing “quality” in online education, specifically at the course level. My mantra during the past has been that there is an unbalanced focus on the lower right triangle  with little, if any, attention being paid to the other two triangles. I have noticed a slight uptick in the attention paid to the quality of instruction and the quality of student learning achievements, but those areas still pale in comparison to the focus on course design. Most people believe that if they have a QM (or QM-like) rubric and process, then they’re answering their critics’ questions about online course quality.

Those of you who’ve heard me speak on this matter know that I consider the course design to be several magnitudes of lesser importance than the other two areas. In fact, if you can provide solid, repeatable evidence that the online students are achieving the intended learning outcomes at a very high level – then I’d be willing to throw out the other two altogether. That’s rarely the case. Let me restate that: if your online students are learning what they are supposed to learn, then I’d rather not dictate certain teaching techniques or course design standards, because what they are doing must be working. It’s nice to have evidence of all three – as long as your use of a design rubric doesn’t become a one-size fits-all approach.

My suggestion is that you focus on gathering evidence for the blue triangle. If that evidence is weak, then look at the other two triangles to identify potential causes (there are other potential causes, of course). If the evidence of success in the blue triangle is strong, don’t worry so much about the other two.

I’m just getting started. Check back soon.

Filed under: e-Quality, Online learning | 3 Comments »

This is a repost from my old business site: Excellence in e-Education (xlents.com). That site is being shut down and I am preserving those items that I don’t want to have disappear. Originally posted on January 12, 2011.

What does the word “quality” mean in the context of online learning?

How do you recognize quality when you see it?

What steps can we take to improve our quality of online teaching and learning?

The main focus of this workshop deals with the questions related to “quality” in online learning. Quality means very different things to different people when it comes to online learning. In this workshop we focus on three different aspects of e-Learning quality:

1. Quality of online learning
2. Quality of online teaching
3. Quality of online course design

We start this workshop by looking at models of good practice on online learning course design. This helps set the stage for the later pieces that focus on the quality of the learning and the quality of the teaching.

A wiki containing links to a large number of online resources is shared with attendees for their use during and after the workshop.

For a more active workshop, we also encourage the formation of teams to work on specific projects during the workshop. Some examples of group projects include the following:

1. Peer review group of faculty for improving course design (create process for assessing quality of design)
2. Course design rubric creation (create tool for assessing quality of course design)
3. Learning assessment project for online students  (quality of online learning assessment)
4. Develop formal expectations for faculty teaching in online courses (quality of teaching baseline)
5. Online student end-of-course evaluation instrument and process
6. Faculty evaluation process and instrument creation (quality of online teaching)

Filed under: e-Quality, Online learning | 1 Comment »