Delivered at BayCHI (March 2009) and the Usability Marathon webinar (October 2009). Although I don't read my presentations, I've included some notes that approximate what I said in delivering the presentation.
In our lifetimes, software has gone from character-based, menu-driven, and monochrome, to mouse-driven visual interfaces and now to colorful, polished visual presentation of tools and content.
This means that designing software increasingly requires mastery of two distinct disciplines --Interaction design, which focuses on defining behavior, structure and flow --visual design, which communicates about on-screen behavior and information There are a couple of strong arguments for each of these being a separate discipline. --each requires specialized skill, so it’s difficult to develop true expertise in both --it’s too slow to do both in sequence
There’s also a productive tension between the two. Both disciplines play a huge role in usability by reducing, prioritizing, and organizing controls and information. But interaction design brings a rigor and logic that’s critical to the effective design of behavior; traditional graphic design approaches are geared toward fairly static design problems and aren’t equipped to handle the complex behavior of software.
On the other hand, design is about usefulness, usability AND emotion. The most successful designs make an emotional connection with their users. Good design turns a consumer product into an object of desire, or makes a medical device look both precise and trustworthy. These are areas in which visual design excels.
But these days more and more software is moving beyond the desktop. Our transportation interfaces, from car dashboards to train ticket kiosks,are both physical and digital.
Our phones, for better or for worse, are taking on more complex functions.
Our healthcare increasingly involves devices with digital interfaces, from diagnosis to treatment and even surgery.
Watching TV, reading, and listening to music all involve complex interactivity.
The tools we use around the house are all becoming smarter, making it possible to use ever more complex parameters to water the garden, manage the climate, or even make dinner.
These days, even the shower may have a complex interface.
One thing you may notice is that all of these examples involve physical as well as graphical interfaces. This means that in order to design today’s products, we have to integrate yet a third discipline, and that’s industrial design.
Users have just one experience of a product; they don’t separate hardware from software, or appearance from behavior, or rational from emotional appeal. When was the last time you heard someone—and I mean normal human beings, not people like us—say that the visual design was good but the behavior was terrible, or the hardware was ugly but the interaction design worked OK?
This is why it seems crazy to me that in most places, interaction and industrial design might as well have a wall between them. In some cases this wall is official, either because of secrecy or a belief that hardware and software are separate systems, or because these groups just don’t know how to work together. And believe it or not, interaction and visual design are still fairly separate in a lot of places, too.
To create a unified experience, the disciplines must collaborate on a daily basis, and must all be at least literate in one another’s fields.
They’ll also be much more effective if they use a single overlapping process that allows each to work separately as needed, but provides shared milestones, a shared language, and a shared basis for decision making. This unified process is what I’d like to talk about tonight.
I assume everyone here is familiar with common interaction design approaches. Our process at Cooper has some unique aspects, but at the highest level, I’m sure it looks pretty familiar. It starts with project planning, a shared understanding of the business problem, and ideally some form of user research. That research is then translated into a shared set of models and requirements to drive design. The framework of the design—core concepts, navigation, and the basic data model--gets roughed out enough that stakeholders, subject matter experts, and engineers can begin to evaluate it. Iterative design cycles, including any usability testing, gradually flesh out the detail until you have an essentially finished spec. Of course, detailed engineering can require minor changes after that. As we’ve integrated visual and industrial design over the last 8 years or so, we’ve found that with some modification, this fundamental approach applies very well. I’ll focus not so much on the interaction design activities, which I assume you’re pretty familiar with, and more on how we integrate the functional and emotional aspects of visual and industrial design.
We always start by interviewing stakeholders to understand the business goals of the project. Then we do whatever amount of user research is appropriate to the design problem, the budget, and the timeline. Our favorite approach relies on ethnographic techniques, combining both interviewing and in-context observation. In most cases, sessions are about an hour, and the sample size might be anywhere from about 5 users to 50 or more, depending on the nature of the design problem. The design team does the research to make sure they’re getting what they need and to minimize information loss between research and design. We try not to have more than 2 or 3 team members in an interview, which usually means the visual and industrial designers attend only a subset of the interviews.
Each team member is looking for something different. For example, take this image of a surgeon using a system to aid in the correct positioning of artificial joints. The interaction designer is looking at the guy holding up a sheet of paper and wondering what’s on it, and why it’s not in the digital system. The industrial designer is noticing whether the device is at the right height and angle to minimize physical strain. The visual designer is noticing how far away from the screen the surgeon is, and the fact that the screen is partly obscured by a plastic drape. Both the visual and industrial designer are also noticing the prevalence of certain colors and materials and the emphasis on cleanliness.
Where the environment is within a user’s control, such as in the home, the visual and industrial designers are also looking for clues to aesthetic and brand leanings. For example, someone who buys modern furnishings at Crate and Barrel and reads Dwell magazine is clearly responding to a different aesthetic than someone who shops at Williams Sonoma, reads Better Homes and Gardens, and buys classic furniture in dark finishes.
Once the data is gathered, the whole team gets together to synthesize the research findings and turn them into models that the entire product team can digest.
Interaction designers may model a few key workflows, but don’t obsess over drawing a million detailed use cases because the details are likely to change.
We might capture data taxonomies, either for our own understanding if it’s a novel field, or if different types of users seemed to have different mental models.
The thing we always do is identify different behavior patterns among the users. These form the basis for a set of personas.
Originally, personas were defined primarily in terms of their goals, skills, and tasks, since they originated as a tool for interaction design.
But these days, we tend to incorporate some other components that are useful to visual and industrial designers. We might include physical characteristics such as height and hand size for hardware products, but issues of that kind tend to be covered pretty well by references like the Measure of and and Woman. Instead, the additional components focus on emotion, brand affinity, and environment. How we communicate this may vary, but one approach we often use is a collage of a persona’s day or lifestyle.
We might create a sketchy representation of each person’s environment. This may provide an idea of how a physical device might fit practical constraints like limited storage space, but it can also provide a taste of a persona’s aesthetic sensibilities.
Perhaps most importantly, we expand the set of goals. For interaction design purposes, persona goals are generally end goals: what the personas hope to accomplish—or at least partially accomplish—by using a product. These might be things like avoiding surprises, going home at 5:00, or optimizing the return on a stock portfolio. However, visual and industrial designers also need to know how people hope to feel. For example, most of us would like to feel safe when we put our money in a bank. Of course, these days it takes more than a navy blue, red, and white color palette to accomplish this, but there’s a reason most banks don’t use orange, purple, and lime green.
Once we’ve got agreement on who the users are, how they think and behave, and what their goals are, the next step is to translate that into a set of high level requirements. These aren’t detailed wish lists that get handed over to engineering; rather, they are meant as discussion points to help the product team arrive at a definition of what the product will and won’t do.
Some requirements may be driven by regulations, persona goals, environmental issues, and so forth, but most are derived from scenarios. These are initially high-level stories that treat the solution as a bit of a black box, then gradually get more detailed ad the design emerges. In the early stages they’re usually technology-agnostic, and they’re described as a sort of conversation between user and product. Unlike agile user stories, they don’t break functionality up into bits and pieces. They’re also not trying to be exhaustive use cases. Instead, they’re a sort of “imagine if…” story that’s meant to push the presumed constraints to some reasonable extent. Again, they’re meant as a discussion tool.
Requirements don’t stop at functionality, though. We want to get consensus on the personality of the experience, too. These requirements are driven by persona experience goals, brand, and stakeholder vision. The visual and industrial designer look for patterns in the brand attributes and key adjectives they heard form stakeholders. We translate these terms into similar terms that are more visual in nature. For example, it’s hard to say what intelligent looks like, but brilliant is easier to get mental picture of. Then we look for overlap between those and the persona experience goals.
These result in a set of what we call experience attributes. There are usually 3 or 4 main attributes that we use to drive the design language, backed up by some related words we also heard. Often, we’re setting up a deliberate tension. For example, if an IT application is too expert, it might be intimidating, so that’s balanced by approachable. The idea is to get agreement on how the product is meant to feel. Using personas to get agreement on who the users are and what they need gives stakeholders a shared basis for making decisions about behavior. In the same way, these terms provide a better basis for making style decisions than “I like this one, I don’t like that one.”
When we have time, we like to help stakeholders begin to envision the implications of these terms by using a collage of images that evoke similar feelings. These start to forecast what we call the design language: how colors, materials, shapes, and so on are used to communicate. For example, if a product is going to make someone feel secure, it has to have some visual weight. It’s probably not going to be bright pink. It’s likely going to use materials that feel sturdy, heavy lines, and so on.
We might also use contrasts to make the meaning of the terms more clear. For example, “clean” can feel cold and forbidding if taken too far.
Once we’ve got agreement on what the product has to accomplish both functionally and from the perspective of design language, we begin to craft what we call the design framework. Ideally, this begins with divergence, and winds up with convergence on a single direction.
In the first few days of design time, the team is focused on fundamental questions of platform and architecture. We define a platform as a basic form factor—for example a portable device of approximately a certain size—plus the input and display methods. For example, you might decide that your device needs a screen of a certain size to display the data, but you might opt for touchscreen input, a 5-in-1 physical control, or soft buttons arranged round the screen for selection of adjacent graphical elements. Once you’ve settled on a platform, you have consider the physical architecture—in other words, the physical arrangement of those elements in relation to one another.
Or you might debate the virtues of a touchscreen versus soft keys.
And you’ll consider whether the type of data is most conducive to a horizontal or vertical display.
Once the team has a small number of possibly viable directions, the interaction designers start using scenarios to flesh out the most critical interactions. This helps figure out of the presumed screen size and orientation will work, and also starts to reveal more pros and cons of different input mechanisms. After some back and forth with the industrial designer to ensure there are one or more solutions that work form a design perspective, there’s generally a feasibility review with the engineers before a bigger group of stakeholders gets involved.
At the same time, the industrial designers begin to think about the physical form in more detail, typically using both hand sketches and rough physical models. They’re starting to think about how much room the necessary internal components will require, usually in consultation with a mechanical engineer and maybe an electrical engineer.
For the sake of communication, we tend to show rough sketches of hardware and software from a functional and structural point of view, plus separate treatment of the design language. We tend to separate function form appearance at this stage because most stakeholders aren’t very good at evaluating both together—they’ll focus on the fact that they don’t like the color when what we really need to know is whether the structure is working, and vice versa.
The design language is essentially communicated in terms of paint chips and fabric swatches.
Here are some examples of visual design language studies focused on the software. You can see that they’re not showing real behavior because that’s not the focus. Each study emphasizes a different experience attribute. For example, this one emphasizes the approachable quality by using bright colors, and this subtle texture in the background might even undulate slightly when the phone isn’t in use to imply touchability.
It’s harder for industrial designers to take a fabric swatch approach because the overall shape of a device is an essential part of the design language. If time is tight, initial style studies might just be hand sketches plus some color and material samples.
When time permits, though, it’s ideal to do some very loose 3D renderings to show overall form, materials, and surfacing. There’s ususally some hand sketching on top of these to show detail. This can get a bit expensive if you’ve got multiple platforms or architectures to depict, so this might only happen once that functional decision has been made. The physical studies are ideally paired with the software visual design studies. For example, this one looks approachable in part because the materials and shape look fairly similar to existing desk phones.
On the other hand, this one seems more exceptional because both hardware and software used a shiny, polished look.
And in this one, you can see that trustworthy gray color makes its way into the physical materials.
Of course, all this takes close collaboration between the visual and industrial designer.
Once there’s agreement on the overall design direction, the team iterates through increasingly detailed versions of the design to work toward a final spec
The interaction designers work closely with the industrial designers on specifying the behavior and states of hardware controls.
But most of their time is spent on software behavior. Most of the widget level and content detail gets worked out at the whiteboard, reviewed with subject mater experts and engineers while it’s still at the whiteboard level, then tweaked and documented in whatever form the spec takes.
This usually means detailed screens rendered in pixels, not wireframes. Although the exact appearance is driven by the visual designers, interaction designers tend to be more successful if they also work in pixels because this clarifies behavior and allows a realistic assessment of how much readable content fits on the screen.
This means that early on, the visual designers are working out a detailed visual system. They try to figure out the structural parts of this quickly so the interaction designers can start to draw screens in pixels. One of the first things to get defined is a grid. This is an underlying structure used to lay out screens or pages. In simple terms, it’s actually very much like the Lego system—if everything is based on a common unit, then it all fits together.
As you can imagine, making the visual and interaction design work in parallel requires a lot of back and forth. Although they may work independently for most of the day, the whole team gets together once or twice a day to review interaction design and visual design work in progress.
Other elements of the visual system, like icons, don’t define the visual structure, so they tend to wait until later in the process.
Ultimately, every screen is specced down to the pixel, or in some cases the visual designer may actually build parts of the presentation layer.
Once the first draft design is documented in whatever form, it’s ready for a more detailed review and usability testing. In some cases, it works to do a paper prototype test before documenting in pixels, though we find that’s much less useful if you have rich data visualization and other things that require detailed visual design to communicate. You also need some kind of functional prototype to test certain kinds of hardware/software interaction. We like to document in pixels before detailed review because we find that documenting the design tends to improve it by identifying holes and inconsistencies. It also allows for more detailed examination of the design.
Ideally, the interaction design work is split up into different topics so review and coding can take place mostly in parallel, though coding should lag behind by one or two topics. This approach works very well in conjunction with agile sprints, provided you have a solid design framework before you start trying to carve things up into chunks. This doesn’t eliminate the need for engineering collaboration at the whiteboard, by the way. Once that review, any testing, and maybe that first engineering sprint are done, the design get tweaked as needed and you can worry about little details like keyboard accelerators and so on. This also allows for refinement of behaviors that really need to be prototyped, like how fast the screen scrolls or exactly how something animates.
At the same time, the industrial designers are collaborating most closely with mechanical engineering to finalize the parts list and assembly of the hardware.
The ME usually owns the final CAD file that gets sent to manufacturing. The ID is typically reviewing the file in detail and asking for adjustments.
In consultation with the visual designer, the ID is also specifying the details of color and materials. Generally, the team builds at least one detailed appearance model to ensure that what they’re envisioning in CAD works as well in the physical world.
And of course, the whole design team is ideally available part time throughout implementation, because as we all know, sometimes things that seemed doable earlier are trickier during construction. As long as there’s sufficient collaboration with engineers throughout, the issues tend to be minor.
It takes some good project management mojo to keep all aspects of the design progressing in step, but the end result is a user experience that’s both seamless and emotionally engaging.
If you find all of this interesting, there’s lots more information in my new book.
