This is experience design

my talk on mobile AR experience design at Web2Expo

Tuesday, April 27th, 2010

I’m presenting a session at Web2Expo in San Francisco on May 4th, titled “Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences“. Here’s the blurb:

Mobile augmented reality adds digital overlays and interactivity to the physical world using the sensors and display of your smartphone. Design of mobile AR experiences is complex and takes us well beyond the browser-based web. This session will give you a mix of practical knowledge and new ideas for creating AR experiences, drawing from web design, 3D graphics, games, architecture and stagecraft.

The next generation of mobile augmented reality applications will go well beyond simply overlaying points of interest, floating post-its and 3D models on the video display of your phone. Mobile AR is becoming a sophisticated medium for immersive games, situated storytelling, large-scale visualization, and artistic expression. The combination of physical presence, visual and audio media, sensor datastreams and social environments blended together with web services offers tremendous new creative possibilities. However, the design challenges of creating engaging, exciting and compelling experiences are quite significant.

Research on the design of technology-mediated experiences has shown that compelling experiences often involve a mixture of physical and mental challenge or self-expression, a sense of drama, sensory stimulation, and social interaction. These elements can give us a physical “buzz” by activating the release of adrenaline, endorphins and related neurochemicals.

Mobile AR puts us “where the action is”—in motion through the physical world, surrounded by other people, in a stimulating environment. AR applications additionally provide challenges, stories, information and communication. Factors that AR experience designers need to consider include:

  • Goals of the AR experience
  • Users’ cognitive model of the system
  • Physical environment and context of the experience
  • Social context of the experience
  • Design of interaction models and experience mechanics
  • Story, goals and outcomes
  • Immersion and flow
  • Design of visual and audio assets
  • Non-player characters (“AIs”)
  • Tracking and analytics
  • Technical capabilities and limitations of the AR system
  • Managing the production process (designing an AR experience has much in common with producing a movie on location)

Should be fun, ping me if you’re going to be at the conference!

let’s bury the electronic newspaper

Friday, January 22nd, 2010

In technology, use model and interface metaphors exert a powerful influence on the rhetorical framing innovators adopt for their work. These metaphors can become entrenched schools of thought in product and experience design, making it difficult to imagine alternative approaches. Consider the longevity of the desktop metaphor for personal computing – it has been more than 35 years since the original Xerox PARC Alto, and we are still looking at desktops on many of our screens.

Similarly, the idea of electronic newspapers has been around since at least the 1970s, and now that the print newspaper industry is in dire straits we are seeing that notion thrown around rather more freqently. For example, LG Display is currently showing off an impressive lab prototype of a 19” flexible e-paper display that is 0.3mm thick and weighs just 4 ounces. The prototype measures 40x25cm or around 16×10 inches, making it about the size of a small tabloid newspaper. LG are touting it as “optimized for an e-newspaper and able to convey the feeling of reading an actual newspaper”

lg-19-inch-epaper

LG Display 19" e-paper prototype

As I see it, there’s a fundamental problem with attempts to transplant the design of physical newspapers into an “electronic newspaper” interaction metaphor. The design of print newspapers, and our interaction with them – where, when, what and how we read – arises from the intrinsic properties of the medium. The size of pages, the multicolumn tiled layout, the length of stories, the variety and separation of sections, advertising, subscriptions, deadlines, distribution, local geographic focus, national syndication, editorial viewpoint – all of these factors evolved to their current state largely due to the physical properties and economics of paper. When you remove the paper and substitute a dynamic networked display appliance, you have changed the underlying properties and constraints so radically that the entire newspaper metaphor collapses.

Furthermore, as Clay Shirky and many others have observed, the Internet has spawned a host of disaggregated alternatives to all of the major functions of print newspapers. From online news sites, blogs and social media to Craigslist, Google, spot.us and the Sunlight Foundation, we are evolving new structures, business logic and user experiences based on the properties and economics of connected world technologies. As Shirky wrote in March 2009:

Round and round this goes, with the people committed to saving newspapers demanding to know “If the old model is broken, what will work in its place?” To which the answer is: Nothing. Nothing will work. There is no general model for newspapers to replace the one the internet just broke.

With the old economics destroyed, organizational forms perfected for industrial production have to be replaced with structures optimized for digital data. It makes increasingly less sense even to talk about a publishing industry, because the core problem publishing solves — the incredible difficulty, complexity, and expense of making something available to the public — has stopped being a problem.

When I see electronic versions of print newspapers being sold for Amazon’s Kindle, demonstrated on the Plastic Logic QUE, and mocked up in demos like LG’s flexible plastic e-paper, I see designers and marketers indulging in nostalgia for a bygone era. Newspapers as we know them are pretty much dead. Let’s bury the electronic newspaper metaphor with them.

Postscript:

hakon-lie-monitor

While researching this post I came across Hakon Lie’s 1990 MSc thesis from the MIT Media Lab, titled The Electronic Broadsheet – all the news that fits the display. Lie describes the design and implementation of a broadsheet-sized electronic newspaper on a large high resolution display. Although some of the leading edge technology from 1990 (pre-WWW, pre-flat panel monitors) seems quaint now, Lie’s overview of the Newspaper Metaphor remains relevant and worth reviewing. Lie sought to maintain the best qualities and practices of newspaper reading while augmenting them with the affordances of networked digital media, reifying the whole into a new kind of newspaper. At the time, he did not anticipate the breadth and depth of disruption that would begin in just a few years with the advent of the web. Of course Lie went on to develop CSS in 1994, demonstrating somewhat greater adaptability than the newspaper industry he sought to transform.

experience design for locative media & AR

Thursday, December 3rd, 2009

The next generation of mobile Augmented Reality applications will go well beyond simply overlaying points of interest, floating post-its and 3D models on the video display of your phone. Mobile AR is becoming a sophisticated medium for immersive games, situated storytelling, large-scale visualization, and artistic expression. The combination of physical presence, visual and audio media, sensor datastreams and social environments blended together with web services offers tremendous new creative possibilities. However, the design challenges of creating engaging, exciting and compelling experiences are quite significant. AR experience designers will draw fruitful inspiration and practical lessons from game design, 3D graphics, architecture and stagecraft, as well as the structure, linking, protocols and openness of the web.

Some of the best research to date on experience design for locative media experiences, was done at HP Labs as part of the Mobile Bristol collaboration. You might find these papers useful and applicable to AR design, as I have.

Technology Experiences: What Makes Them Compelling? Alison Kidd, 2000

Experience Design for Pervasive Computing, Richard Hull & Jo Reid

Experience Design Guidelines for Creating Situated Mediascapes, Reid et al, 2005

Magic Moments in Situated Mediascapes, Reid et al