I’m really starting to dig Kathy Sierra’s work over at Creating Passionate Users. If you’ve spent any time with us, you know our own passion for understanding brain function and why human beings are neurologically so gifted to attach complex meanings to sounds. We met with a new partner Tuesday who couldn’t stop talking about Broca’s Area of the brain. For us, that’s an awful lot more fun than droning on about demographics and media mix. If you think there just might be something to this brain stuff, too, I urge you to check out Kathy’s site, and maybe even look us up sometime for a chat. You’ve been a great audience … please welcome Kathy Sierra:
The effect of sound on users
By Kathy Sierra
In Hollywood, some say that in a movie the visuals tell you what you’re seeing, while the soundtrack tells you how to feel
about it. I used to teach new media and interaction design at UCLA
Extension’s Entertainment Studies Department, and in the film scoring
classes (which I didn’t teach) students often started out with a
classic exercise: Watch the shower scene in the movie Psycho, without the sound.
Without that Ee-Ee-Ee-Knife-Slashing audio it just doesn’t feel like
the scene that scared me into being more of a bath than shower girl.
And audio has even been shown to affect the audience perception of the quality of a presentation more
than the visuals. I don’t have a link handy, but there’s a study that
showed that the quality of the audio causes people to change their
evaluation of the quality of the visuals, but that it doesn’t work in
reverse. In other words, given a film clip or animation, raising the
quality of the audio caused people to say, "Hey, the visuals are
good!", and when the audio quality sucked, the audience rated the
visuals as worse even when the visual quality was the same in both the bad and good audio versions.
But… changing the quality of the visuals did NOT have that same
power. If the sound sucked but the visuals were fabulous, it didn’t
cause viewers to say, "Wow… good sound too."
So, sound has the power to raise (or lower) audience perception of
visuals, but visual doesn’t have the power to change how the audience
perceives the audio.
But sound is usually the second-class citizen in the
non-professional multimedia world, while visuals take center stage in
everything. (Unless the photographer, videographer, or animator happens
to also be a musician). Everyone has a digital camera now-that makes
everyone at least an amateur photographer. And everyone has some kind
of digital editing software like Photoshop Elements that brings
high-end photo manipulation to the home user. And why stop with still
pictures when digital camcorders are so cheap now? With editing
software like iMovie shipping with every Mac, anyone can become a video
editor now.
But while the emphasis on developing visual sensibilities and skills
has continued to build (almost everyone with a digital camera today
knows design fundamentals like the "rule of thirds" or how not to cut
people off at their joints), what about the poor stepchild audio? Sure
we could all listen to music, but where were the tools that
would bring music creation to non-musicians in the way that the visual
tools (and books and references) brought graphic and photographic
editing to non-photographers and non-graphic designers?
I gave a presention on this discrepancy ten years ago to a new media
group in Los Angeles, and while I stood there ranting about how nobody
had made the Photoshop equivalent of audio (or even the Kid Pix
equivalent) one guy in the audience, Kevin Klinger, started thinking
about it. He went home, thought some more, and decided to start a company to do just that. I was at Mac World many years later, and there was the Smart Sound booth, and Kevin said, "Hey, thanks for the idea." I couldn’t believe it. He actually did it.
SmartSound’s focus is on giving people a way to create sound tracks
for videos, without being a musician. And the "smart" part comes from
the cleverly-engineered ways in which the software fits itself to the
movie. Because another problem with most home movies is that the music
often doesn’t finish, it often just fades out (or worse, cuts
off), because the music was too long for the video. SmartSound is an
amazing program and goes way past what I had in mind when I gave the
talk. But it’s main focus is on making sound tracks for, say, corporate
videos, while I was still waiting for that low-end, home-use music
creation program for non (or very weak) musicians.
Something that would encourage "regular people" to start developing
music and sound sensitivities in the way that we’ve developed our
design and visual awareness and creative skills. In other words, when
will "the rest of us" get to work on the "A" in "AV"?
Of course, Apple’s done that now with the phenomenal Garage Band!
A tool that threatens to turn people who have no business making music
into musicians. (When I say "no business making music" I’m referring to
the notion some have that there must be clear boundaries between those
who create art and those who appreciate it. I think
that’s bullshit… we’re all born creative even if many of us will
never EVER hope to be professionals. Heck, we all spent our first nine
months listening to a 24-7 dance beat.)
I blogged earlier about the way that teens and twenty-somethings today often treat turntabilism the way forty-year olds used to treat guitars. They treat it like an instrument.
Something to use for creating music. So the audio world is definitely
changing, and Garage Band is, I think, the single most important step
in bringing music into the world formerly reserved only for graphics
programs.
The four of us care a lot about sound here, because sound (and
especially music) has a powerful effect on learning, in two ways.
First, it manipulates emotions, and emotions play a huge role in memory
formation. Second, the more senses you can involve in learning, the
greater your chances of retaining and later recalling the knowledge.
Think about it-if you file something in two places instead of
one, you’ve doubled your chances of getting it back again, and when you
remember something as, say, a sound, image, and text-you’ve just given
yourself three potentially different ways in which that info is stored
in the brain. Triple the chances of getting it out when you need it.
But… we’re still doing books and books don’t give us a way to do that. We are
planning some multimedia formats for the fairly near future, but for
now, we’re doing what we can with things we expect you-the reader-to
do to fully realize the power of audio. When you come across a
limerick, poem, or song lyric in the book, for the love of god PLEASE
say it out loud! Don’t just read it silently (although even then, you
often will hear yourself saying it with a rhythm, and even that helps).
Anyway, I’ll have a lot more to say about audio in the future, but
for now, here’s my really BAD version of the film school exercise for
those who aren’t musicians or sound designers. I have three videos, all
with the exact same little scene, but using three different songs. (The
songs are just simple sequences I put together in GarageBand, which
means no copyright issues : ) [Disclaimer: I'm not a musician (which
will be painfully obvious if you play these). The point is the effect the music has, regardless of what you're looking at.]
Your assignment is to play the videos, in order, and with each
one, write down the following info before moving on to the next video:
1) What kind of movie is this? Speculate on what the movie might be about, based on the feeling
you get from the audio. Do NOT use your brain to try to think something
up, just go with what pops into your head based on the feeling.
2) Speculate on what might be happening in this character’s life.
Make up what the next scene might be, based on the feeling you have
from the sound.
Once you’ve done that with all three, play one of them without any
sound at all and see how you feel about it now. (You can experiment by
playing one of the others, and see if the last music you heard affects
how you view it when there’s no music at all.)
Finally, pick an emotion/feeling you want to evoke, and find some
music you have that you think will create that feeling. Play that while
the video is playing, and test it on someone else and see if your
victim gets the feeling.
Here they are, and remember… watch them in this order (note: they are Quicktime movies, about 1.5 Mbs, sorry dial-up users : (
Version one movie
Version two movie
Version three movie
So, how are you using the power of sound in your teaching and learning and communicating?
