There was a time when words were all that mattered. The teacher would talk, the children would copy down what they said precisely, and that was considered learning.

Allan Paivio, a researcher at the University of Western Ontario in Canada (now Western University), played a key role in shifting education away from such approaches. In 1971, he proposed the theory of dual coding, also known as dual channels, which posits that humans have two separate systems for processing verbal and pictorial information.

This finding has become one of the fundamental concepts in cognitive science. It was with this in mind that American educational psychologist Richard Mayer, a distinguished professor at the Department of Psychological and Brain Sciences at the University of California, Santa Barbara, developed his theory of multimedia learning: the idea that people learn more deeply from words and graphics than from words alone.

“Traditionally, in education, we presented material in words,” explains Mayer. “So, either through speech, giving a lecture, let’s say, or through books that have a lot of text students are simply supposed to read. Sometimes there are graphics, but they are usually supplementary - not really considered part of the main story - and my idea with multimedia learning is that adding graphics to text can greatly improve people’s comprehension of the material.”

Ditch the cat pics

This does not mean that teachers should add a couple of semi-relevant images into their lesson plans and assume that their job is done. As Mayer notes, there is an art to combining text and graphics - and not just the art involved in drawing the pictures.

“If you look at textbooks, at least in California, about half the space in science textbooks is allocated to graphics and the other half to text. But if you look at the quality of the graphics, oftentimes they are not that instructionally impactful. Sometimes they are just decoration - so a story about rocket ships might have a picture of a rocket ship and underneath it would say ‘this is a rocket ship’. That doesn’t really tell you a lot.”

Since he came up with his theory about 25 years ago, Mayer has been developing principles on how to design effective graphics for learning.

“The picture has to support the text,” he says. “A lot of illustrations [I see teachers using] are, like I said, decorations. They are put in there to be cute and to be amusing or motivational - you might have little pictures of animals that the kids like. That, generally speaking, is not helpful.”

He has a number of “rules” for images, one of which is called “the coherence principle”. It states that all extraneous material should be eliminated.

“We want graphics that are simple, easy to understand, don’t have more information than you need,” says Mayer. “A lot of the time, a simple line drawing is the most effective type of graphic, even though textbooks love to have really complicated photographs or complicated figures, it is better to keep things as simple as you possibly can.”

The quality of the art, he adds, is “probably not that important”.

“In my earlier studies, I looked at trying to explain how a bicycle tyre pump works, and I used very simple line drawings and different frames, so you could see from frame to frame what the changes were,” recalls Mayer. “If the handle goes up in a pump, what happens to the valves? And if it goes down, what happens to the valves? I think graphics that are simple help you to focus on the main idea that we are trying to get across.”

The research literature on this topic is unequivocal: multimedia learning really does aid understanding and comprehension. This is especially true for certain situations.

“There are what I would call ‘boundary conditions’ - which means the principle applies more strongly under certain circumstances,” Mayer explains. “This multimedia principle works best for beginners - students who are not all that knowledgeable about the topic. I think the reason is that experts can form their own images in their minds as they are reading. Beginners might not be able to do that, so we might need to provide them with graphics.”

Despite this drop-off in effectiveness as comprehension improves, the evidence base is solid. Mayer says that, on average, if students are tested on something they have been taught, they will typically perform “between half a standard deviation and a full standard deviation” better if they have had instruction with words and graphics, versus just words.

The wrong kind of ‘wow’

So, is this an area in which tech can bring even more benefits to the model? One of Mayer’s current research projects takes the idea of multimedia learning a step further, into the world of virtual reality, by exploring whether VR could be a useful device for helping pupils to understand scientific concepts.

“In a study we have going on now, we are teaching a biology lesson and a history lesson in immersive virtual reality, so you wear a headset display and you walk around the room,” he says. “We are comparing that with a more conventional approach, where students just watch on a computer or read the text. We are trying to see how we can use virtual reality environments to help people learn.”

The results so far have been mixed. “What we are finding is that virtual reality doesn’t necessarily improve learning per se - in fact, it can actually depress learning because it is too distracting,” reveals Mayer. This is perhaps unsurprising given that even elaborate pictures can prove less effective than simple line drawings.

“We find the students are paying attention to things that have nothing to do with the lesson,” Mayer adds. “Just the ‘wow’ factor of being in VR may distract them from the actual content, so we have to figure out how to make use of the motivational power of these new technologies so that they actually can be used to help people learn.”

The trick, he says, is ensuring that the wow factor in a lesson comes not from the visual quality of the learning aids, or the impressive nature of the technology, but from the comprehension of what is being taught.

“I am sure the pictures I drew [in my previous lessons] didn’t quite have the wow factor of VR, but the wow factor that they got was understanding: there is a concept called ‘cognitive interest’, which says that it is motivating to understand; if you can present material in a way that people can understand, that in itself is actually motivating.”

Intuitive insight

Such motivation creates engagement, and that engagement in turn can lead to greater learning gains. “Effective instruction primes three cognitive processes in learners: selecting the relevant information; organising it into a coherent structure; and integrating it with your relevant prior knowledge,” Mayer says. “The thing about graphics is, I think they are a good way of activating prior knowledge and helping you to see how what is being presented connects to that prior knowledge. Graphics can be little more intuitive than words.”

Despite all the research, Mayer does believe simple dictation still has a role.

“Rote learning has its place - there are certain things that have to be learned by rote,” he says. “But there is also material that we want students to be able to use; we want them to be able to transfer what they have learned to new situations.

“It is becoming increasingly important in the 21st century for students to be able to take what they have learned and use it. We don’t really need people who are repositories of information because we have Google for that.

“What we need is to be able to understand it, use it, integrate it. That is the benefit of meaningful learning. Rote learning is fine if all you have to do is to recite it. But if our goal is transfer, then we need to think about more meaningful methods of instruction.”

Chris Parr is a freelance writer

This article originally appeared in the 8 March 2019 issue under the headline “Tes focus on… Multimedia learning”