10 posts categorized "Research"

Pick a Number, Any Number—As Long As It’s Five

Five out of Five

The car saleswoman leaned over the desk, placed a sheet of paper down in front of us. “This is a version of the customer satisfaction survey they’re going to send you next week in the mail.” It was a garden-variety survey about the car purchasing process, with a five-point scale that ranged from Excellent (5) to Poor (1). She plucked a pen from the desk before her, then drew a hard blue line down the page between the 4 and the 5.

“If I’m rated anything less than a five,” she said, “then I’m not doing a good job. It’s either a five or I’ve failed.” She continued to describe how even one 3 or 4 in a month could lead to her not being considered a top salesperson. Then she wouldn’t be able to take her family to Hawaii or the Bahamas at the end of the year, courtesy of the car manufacturer, as a thank you for her performance.

Talking about the whole situation after dinner, my wife Mary said:

Buying a car sucks. Everyone knows it. There’s no way in hell this survey thing is working. If they’re all getting fives and the company says less than five is a failure, then why does buying a car still suck? These surveys send employees to Hawaii—they aren’t actually influencing the overall customer experience.

Surveys can be useful instruments for gathering data from people, but survey questions can be bent or manipulated in ways that destroy their benefit for everyone. In the end, is anyone actually using this data to improve customer service? Or are they just using it to punish service providers?

Rather than relying on the illusory difference between a 4.5 and a 4.7* for quality service, I’m more interested in how someone answers the following simple question: “Was this service great? Yes or no, then tell us why.” In the case of car surveys, this type of open-ended question is one of the few places post-purchase where direct service improvements are solicited. Matt Jones at Edumunds says “at the end of Customer Satisfaction Index (CSI) surveys, there is a comment section for the car shopper to address any concerns that may have come up while doing the deal. These comments do not affect the overall scoring of the salesperson. If a car shopper thought the music was too loud in the dealership, for example, saying that in the survey comment would likely be a better option than giving the salesperson an 8.”

So next time you think about using a graduated scale in your survey, know why you’re using it, what biases those delivering the survey may introduce, and whether there might be a better research instrument to achieve a more relevant result.


* We see this in services like Uber, where if a driver's rating dips below 4.X stars they may not be able to participate in the service. Absolute perfection required.


Bringing Users into Your Process Through Participatory Design

We've been seeing an intense pressure on businesses to rapidly make sense of customer needs and demands, then incorporate that feedback into new or existing products. For today's designers, it can be challenging to make well-informed decisions about the large and small details that comprise these products, especially when working within the constraints of an agile/scrum methodology.

At frog, one of the methods we turn to regularly to identify and incorporate user feedback into products is participatory design. Participatory design aims to bring users into the design process by facilitating conversations through the creation and completion of a wide range of activities. We create activities to facilitate sharing and conversation with users, providing them with materials to descriptively discuss their personal experiences and express their desires for ideal solutions. By doing this, we are able to work directly with current and future users of products and services, quickly discovering important criteria to fold into the next iteration of a product, service, or experience strategy.

In the past year, Erin Muntzert and I had a chance to teach our approach for conducting participatory design to UX designers in workshops globally at Interaction 13, UX London, and UX Week. Based on the feedback from participants in those workshops, we wanted to publicly share the slide deck we used in the workshops, which delves into how Erin and I plan, construct, and facilitate participatory activities to incorporate user feedback into product and service solutions.

This workshop was informed by our own practice in designing and conducting hundreds of participatory design sessions, paired with additional input from other leaders within frog's design research practice. Along with frog's open-source Collective Action Toolkit, which Erin and I helped co-author, this workshop is part of our hope to better share the tools and skills that designers use with individuals and organizations around the world.


"A Five-Step Process For Conducting User Research" on Smashing Magazine

The Design Research Spiral

Imagine that this is what you know about me: I am a college-educated male between the ages of 35 and 45. I own a MacBook Pro and an iPhone 5, on which I browse the Internet via the Google Chrome browser. I tweet and blog publicly, where you can discover that I like chocolate and corgis. I’m married. I drive a Toyota Corolla. I have brown hair and brown eyes. My credit-card statement shows where I’ve booked my most recent hotel reservations and where I like to dine out.

If your financial services client provided you with this data, could you tell them why I’ve just decided to move my checking and savings accounts from it to a new bank? This scenario might seem implausible when laid out like this, but you’ve likely been in similar situations as an interactive designer, working with just demographics or website usage metrics.

We can discern plenty of valuable information about a customer from this data, based on what they do and when they do it. That data, however, doesn’t answer the question of why they do it, and how we can design more effective solutions to their problems through our clients’ websites, products and services. We need more context. User research helps to provide that context.

User research helps us to understand how other people live their lives, so that we can respond more effectively to their needs with informed and inspired design solutions. User research also helps us to avoid our own biases, because we frequently have to create design solutions for people who aren’t like us.

So, how does one do user research? Let me share with you a process we use at Frog to plan and conduct user research…

Read the rest of this in-depth article on Smashing Magazine.


Know Thy User: The Role of Research in Great Interactive Design

At the recent HOW Interactive Design Conference in Washington DC, I gave a presentation called "Know Thy User: The Role of Research in Great Interactive Design." This 30-minute high-level talk was intended to provide conference attendees with repeatable processes that will help them integrate user research into their interactive projects. Other presenters at the conference went more in-depth into some of the methods mentioned in this talk, but I felt that it was important for attendees to understand the role of specific methods and activities within the research process on any design project.

When I started working as an user experience designer, I had a thousand questions about how to conduct research. I was lucky to have great mentors and have the chance to collaborate with outside practitioners on a variety of projects. They helped me learn how to do everything from recruiting people to facilitating activities to understanding how we could make sense of the data we gathered from them.

When I came to frog, I discovered a deep global research practice with many designers and strategists who have helped me become a more rigorous and creative researcher. I now have fewer questions about how to practice user research, so I'm doing my best to return the help I'd received over the years and answer some questions for those who are new to the role of research in interactive design.


Where the Roads Become Rivers

Street, Dhaka

The rules of the road? There are no rules. Riding in a fast-moving car, the freeway is a fat, pulsing vein, and we are but one blood cell swirling through the body called Dhaka.

Oncoming traffic doesn't matter, since we can swerve into whatever lane is free to get there a few seconds faster. Right of way isn't important, because we cut off everyone else indiscriminately—three lanes of road packed five to six vehicles across at any moment with rickshaws, mopeds, cars, buses, compressed natural gas (CNG) taxis.

The man who was helping me find my way around joked that when traffic lights are green, you slow down. When they're yellow, you start to speed up. When they turn red, you drive as fast as you can.

 

Stuck in Traffic, Dhaka

Sounds of the road: constant staccato horns warning of impending crashes in a language only locals can understand; strains of late afternoon prayer floating from mosques and prayer rooms; constant pinging of rickshaw bells; intense revving of engines when brief stretches of open road present themselves; painful shrieks of car bumpers on speed bumps that materialize from out of nowhere.

The air is thick with exhaust, oil fumes, plumes of dirt. When I blow my nose, the tissue is stained black.

 

Market on Saturday, Dhaka

If you live in Bangladesh and want robust health services, quality food, access to clean water, careers for those who have gone to university, greater possibilities for entrepreneurship, better schools for your children, you go to Dhaka.

But with the lure of all the benefits the megacity may provide, the costs of gas, food, transportation, and housing are rising on an exponential basis, further deflating the taka, the country's currency. The district around the city proper of Dhaka has seen unparalleled growth over the past decade, doubling its population to over 16 million residents—most moving into the district from rural communities.

 

On the Road, Dhamrai, Bangladesh

Peering into other cars, I see haphazard crates of chickens and rabbits strutting in straw; families of eight squeezed into a car designed for five; piles of narrow gas tanks in battered two-seaters; businessmen in striped suits behind chipped, tinted windows; military men in green-purple camouflage with rifles lazily pointed towards the sky; an entire backseat piled with mounds of bananas, jackfruit, oranges, and coconut.

But it's the buses that take most of my attention. Their sides look like flayed skin from a running child who tripped and fell on the asphalt. Blue birds painted above a landscape are gouged off. Names of transit companies are ripped away. Bright white, cyan, mustard, and cinnamon hues: fresh coats of paint mask the bruises, but don't make them go away. Men on the roadside wave their hands, they pull up and part with a few hundred taka. The buses fill up until men hang outside open doors, out of open windows, and finally, clamber to the top rather than miss a ride.

The people on the buses look at me as intently as I look at them.

 

Sorting Trash, Dhaka

Just as there are no rules of the road, there is little clarity around the boundaries that sustain Dhaka. Much as a flooded river may reach its fingers into any embankment that sustains its flow, the city's efficiencies are eroding in ways that defy their design.

My Dhaka friends tell me this means that if there were any form of major disaster, such as an earthquake or a cyclone, the impact on the city would be catastrophic—and not just because of population density. It has to do with what people have designed to survive, rather than what might satisfy long-term plans.

An example from a CTO of a small software company in Dhaka: "Imagine that a textiles factory employs 3,000 people. When they build the factory, the owners don't install a fire exit. There is no one who can force them to do it. An earthquake strikes the city, the building is damaged, and no one can get out."

He lets this hang in the air, with all of its implications.

 

Girl Wiping Car Window, Dhaka

Children squeeze between the gaps between vehicles, selling the daily news, slices of carrots and cucumber in thin plastic bags, maps, and books. A man with no hands places his arms against our driver's side, peering inside to look at us. Women with hollow-boned faces tap on windows, some holding babies swaddled in their arms: something to eat?

An eight-year-old girl wipes down the windows and headlights of our car with a dirty cloth. "That girl loves what she's doing," a Dhaka resident says from the seat next to me.

"I don't think that's the precise word for it," I say. "Love." Perhaps she does this out of love, I think. The resident considers my words, says the girl likely cleans cars to bring money back to her family, to eat and have a place to sleep on the one day she doesn't have school… if she is able to attend.

The driver gives her 50 taka before the traffic unfurls. She darts to the median strip.

 

Overpass Under Construction, Dhaka

Dhaka is pocketed with ongoing construction: bypasses, overpasses, buildings, living domiciles, a university campus. Everywhere you look, you see the soon-to-be complete. The gaps are always visible in the city's infrastructure, from transit to banking to telecommunications, and the entrepreneurial are finding ways to fill them.

New services for wiring money to local post offices and bKash (mobile money services) are helping businesses and families better share their money without bank accounts. The availability of WiMax networks around the city provides people access to high-speed Internet without the need for physical wiring.

But on an individual level, the users of these services have to work the systems in order to make the best use of them. Cell phone users have adapted to uneven network coverage by purchasing multiple pay-per-use phones for competing networks. Business automation suffers due to the constant heat, humidity, and lengthy rainy seasons, which destroys technology. Perhaps destroys is not the exact word. Eat might be better. During the monsoon season, consume might even be more appropriate.

 

Aarong, Dhaka

Dhaka's exponential growth is reflected in the the fierce celebration of Bangla culture both within and outside the city: in its language, poetry, dance, song, clothing, and crafts. Some businesses are built around curating this cultural output, such as Aarong, a chain of stores started by BRAC that brings together high-quality crafts, textiles, clothing, and other goods sourced from villages in Bangladesh. There is value that Aarong creates that could be replicated across other cultural industries, even though the richness of Bangladesh culture outside of Dhaka is what makes such a business model possible.

And a question lingered in my mind afterward for which I couldn't find an answer: How many people living outside Dhaka would be capable of shopping there?

 

Ekushey Book Fair, Dhaka

My last afternoon in Dhaka, I visit Ekushey Book Fair. It consumes multiple city blocks, runs the entire month of February, and looked to be attended by what I estimated at 50,000 people that single day. How often do you see such a throng rallying around the consumption of any literature, let alone literature only written in Bangla?

Leaving the bookfair, I talk with the software CTO. He asks me: "What was the most powerful observation you've had about Dhaka?"

"Why do you care about what I think about this city?" I say. "I didn't grow up here. I'm not from this culture. Wouldn't someone who's grown up here know it best?"

"You have to be an outsider to have an opinion about Dhaka," he says. "Many of us also moved to Dhaka—we aren't from here. And so much has changed in the past few years, that it's good to see the city through fresh eyes."

I think for a minute. "How small one can feel, in the midst of a city that is so large. Unlike any other city I've been in, you can't wrap your mind around it. And yet, at the same time, you have so much room to create your own path."

"I agree with that," the CTO says. Then he pauses, looks out the window. "But then again, you have to see rural Bangladesh to really understand Dhaka."

 

Roadway at Dusk, Dhaka

I keep returning to the behavior of a river when I think of my time in Dhaka. It has an ability to cleanse, to sustain, to transmit, and to display unwavering tenacity when dammed away.

I was only able to go a few hours outside Dhaka for a single day, and not far enough to fully escape the influence of the city. But even in those few hours, I was immediately struck by the sun shimmering off lush fields of irrigated rice and corn, expanses of water, the scale and space of the city inverted. All roads in Bangladesh may lead to Dhaka, but not the rivers.

Within inefficient systems are always the seeds of their long-term survival. You just have to grapple with the chaos long enough to sense where they will emerge. With patience, you can see what new channels the river will cut into the landscape, and perhaps follow them. Or, what is often more likely: see if you can divert some of that flow without disrupting the source. And when the monsoons come, you hold on to survive. The cycle continues. Within that optimism is a strength that can't be easily diminished.

I think this is why so many people see such promise in Dhaka's expansion. Standing alone on a dusty road, surrounded by millions of people, I return to the word promise in the same way that my mind returns to the river: the promise of a brighter future, the promise of what might happen tomorrow. Promise is a positive word, swollen with hope the way that the outer district of Dhaka is being flooded with people darting between cars, walking from home to work or school, doing their best in the sweeping current of the history of their rivers—and their country—to put back together what has been broken.


Slides from "Designing the Design Problem"

Thanks for everyone who came out (virtually) to see my presentation yesterday at Creativity Oklahoma's online conference on applied creativity in art and design. Scott Belsky did a great job of describing the philosophies behind Behance and the research about how people make ideas happen that became the foundation of his bestselling new book.

While Scott was talking about fulfilling creative projects, I took a different tack and provided methods that frog uses to marry our innate skills in creative problem solving with the evolving practice of "problem making" to better serve both clients and users in crafting compelling products, services, and experiences. As a case study, I shared research data and insights that had been part of frog's initiative to encourage HIV testing in South Africa, Project Masiluleke.

This 20-minute presentation was carved out of a longer work I'm putting together regarding the specific kinds of activities that make up what's called "multi-vector research," which is the secret weapon for any design team that is trying to tackle a complex and systemic business problem or world problem and discern what exactly should be designed to influence it for the better.


On Design Research and Buddhism

Kyoto | Girls at Taito World

I often think about analogues between design research and Buddhism. Not in a practical way—if there is such a thing—but more in a sense of how the process of design attempts to bring a brief moment of permanence to an idea in an ever-fluctuating world. The more meaningful an idea, the more likely it will gain root in the rich soil of our minds.

Ideas are the leavings of an insight—a deeply rooted and observed human truth. Without an insight, good ideas are mere flower petals scattered across the road and apt to float off in a stiff breeze. Beautiful to admire, but no more meaningful than wallpaper.

Continue reading "On Design Research and Buddhism" »


Meets or Exceeds Expectations

Quality of Service

Hello. I'm calling on behalf of your local bank. Would you like to take part in a short survey that will give us insight into your level of satisfaction with your banking experience?

First, we'd like you to identify which banking location you visit most often. Second, we'd like to ask you to rate, on a scale of 1 to 10, with 10 being the best, the quality of service you received from your last bank teller. If you'd like, you can also share his or her name with us.

Sir, while our survey participants can specifically describe their experience in the banking location, what we're really looking for is the number that correctly correlates with their experience in the bank. That number will aid us in improving our customer service and also give us insight into how we can improve for the future.

How will the survey information be used? These numbers will be averaged and analyzed by our corporate office to make crucial decisions about how to improve our customer experience.

No, you can't give us two numbers that identify the range of experiences you've had. We would like the average of those numbers. For example, if you've received exemplary service from one teller, but below average service from another teller, than you may have simply received average service from us in your banking experience.

Yes, you are welcome to describe in detail the overall feeling you'd received from the design of the bank interior, the quality of the furniture you sit on, the leaflets and posters that are scattered through the store, the demeanor of your teller, the speed with which they fulfilled your banking requests, and the wait time while in line to receive help, whether you can talk to a real person while using online banking, the responsiveness of our telephone banking service, the real impression you get about us from our recent television and radio advertisements, and how you feel regarding the demands of our shareholders, but most of what you've described is outside the scope of this survey.

Now, if you could just answer the first question, sir, we can begin...