More Than Words: Bing TEDActive Virtual Collage Tool Is A Natural Next Step In Communication


The following is a post from Matt Wallaert, a behavioral scientist at Bing.

Earlier this week, members of the Bing team wrote about our TEDActive activation this year, which uses technology developed as part of Microsoft Research’s So.Cl project and created at FUSE Labs. We discussed the way it allows TEDActive attendees to act as citizen reporters, but perhaps more interesting from a scientific angle is the way in which it allows people to express their reactions not by responding directly, but by making mashups of their associations in reaction to each TED Talk.

Imagine, for example, reacting to the TED Talk by Bono. Using, I can put together a video of U2’s “Sunday, Bloody Sunday” alongside a Wikipedia article of the Good Friday Agreement. Throw in pictures of the Egyptian revolution, a video of John Lennon singing “Working Class Hero” and a scholarly essay on poverty and you’re starting to tap into my associations with how we address global issues. And while I could link all those ideas together in an essay to try to explain all the connections that I see between them, it would take far more than the few seconds it takes to create a collage of those sources using the tools at

Conference-goers can assemble online content into collages that express their reactions and associations.

To understand why I think that is important, let me back up a little. Though many people under the age of thirty might not remember it, there was a time when the web was entirely words. The transmission of images via the internet wasn’t yet commonplace, and when it happened, it was usually about sending a file which people then imported into another program to view. Connection speeds meant that it took hours to complete even that process, so the majority of what people made for the web was text-based. It may not have felt natural, but it was the best technology we had.

Today, people expect the web to be visual. Indeed, one of the hallmarks of Bing is our daily photo, a beautiful image that increasingly includes motion (if it took days to download early images, imagine how long it took to download a video!). And even the text of the web is visual; rather than being displayed simply in one giant block, like a massive essay, pages have layouts that display text in complex and subtle structures with bolding, font, and coloring that make them easier to read and understand.

This move to a visual web is no accident. Humans are programmed by thousands of years of evolution to respond to visual stimuli and to draw information from the layout of the world. Babies, from the moment they are born, respond better to drawings that are arranged like human faces than ones that don’t. They learn that words, objects, and images that are visually grouped together are related, that things in a line often represent a development or change (like in a timeline or comic book), and that ideas have motion and can be arranged and rearranged (e.g. “top down”, “bottom up”).

This spatial organization of information is actually a distinguishing feature of human cognition; the human brain is quite literally genetically programmed to see the world as more than the sum of its parts. And increasingly, the web reflects our need to mashup text, images, and video as a way of getting at complex topics that are not easy to express directly. The technology is catching up to the natural way the human mind works.

That ease of creation is a very real part of why I think the TEDActive activation is so important. Just as babies are born ready to perceive faces, the associative collages made by TEDActive attendees are closer to the intrinsic way that people construct meaning in the world. And when we create tools that come closer to natural thought, we not only widen the audience who can use them but increase the value of their output.

This theme of creating tools that are shaped to the way humans actually think and act is one you’ll see increasingly more often at Microsoft, and hopefully across the technological sector, as we push towards more organically human technology. For the past ten years especially, technology has often forced people’s behavior into awkward, unnatural patterns.

For example, where questions naturally occur in patterns like “Who founded the TED organization?” a decade of limited technology has reduced people to keyword searches like “TED organization founder.” It feels awkward for a reason: the human brain isn’t organized along keyword-like retrieval mechanisms and that messy group of keywords isn’t anything like our actual intent. But search technology has pushed people to behave in these highly contrived ways, just like the text-based web sacrificed the natural richness of the world due to inherent limitations in the technology.

At Microsoft, we are pushing the limits of technology to naturalize the way it works. Activations like are part of our commitment to creating a world of tools that make life easier, not harder, and technology that doesn’t feel impoverished or limiting.

We are a long way from the text-only web. In offering free talks in video, audio, and text formats, TED has created a wealth of accessible knowledge that has been viewed over a billion times. And just as TEDActive embraces the profoundly human activity of gathering together to share what is known and to create new knowledge, Bing’s activation is technology that embraces a distinctly human way of thinking: patterns of meaning, linked by association, that create a whole greater than its parts. Humanity is returning to technology; it is about time.

Leave a Reply