The “winners” although there are clearly a diverse set of methods that seem to rise to the top – and frankly – why not as none of these are exclusive of the other, potentially all could be utilized in a single application to enhance the experience as needed:

• 19% Fewer Screens // Utilizing AJAX and a single page app layout
• 14% Type-ahead/Auto-Complete Search // Pulling relevant suggestions for quicker completion is definitely helpful
• 13% Context-Sensitive Navigation // Adjusting tools, links, and means to a goal based on context is clearly logical – keeping the cooking  elements in the kitchen and the tools in the garage
• 13% Multi-Variate Testing // Now here’s my vote for the “winner”. Utilizing actual use (along with contextual observation of target users using your application) is a key primary tool to create truly effective, efficient, and enjoyable expereinces – utilizing all the previously mentioned RIA tools.

Utilizing the power of actual use and user preference from multivariate testing, a web app gains valuable response and iterative improvement meeting the needs of the users but also targeting the product owner’s visions for a differentiated product.

It is not surprising, from my perspective, that the social networking tools did not score high… In a business application context, the primary goals are to “get the job done” secondarilly (if at all) would be to share socially in som way.

An interesting poll, Derek. but what I’d rather see is a open question/survey about “What do you think is the most important UX method in creating a successful web application.” And even then, as always, the answer will be IT DEPENDS (context, target users, and design constraints always guide the proper solution).

Just my two centavos for what they’re worth. Other thoughts?

Reviewing this article from 2006 from EUI and Jared Spool struck a few key points:

• “Scavenger-hunt tasks work best when you’ve thoroughly researched the types of things people look for on the site. Our tasks came from extensive interviews and field research. Unfortunately, many times, teams just make up their tasks without doing the research. That’s where the problems begin.”
  The act of creating and selecting tasks needs to flow form solid research – so that your assumptions are verified and correct – otherwise UXD practices can easily get off track.
• “We were quick to see that people who had passion for the tasks behaved quite differently than those that didn’t. People with passion demanded more from the content on the site.”
  So true, many tests with “distant” tasks to the user are just quick run-through’s with no emotional investment in them. Finding that sweet spot where the user has interest, vested value and can clearly express their need and goals and get there is ideal. There is a clear sense of what’s worked and what doesn’t – the user has a clear need they want fulfilled.
• “In interview-based tasks, the participants interested are discovered, not assigned. Unlike scavenger-hunt tasks, the test’s facilitator and participant negotiate the tasks during the tests, instead of proceeding down a list of predefined tasks.”
  Not only getting to know the target user groups as a project goal, but also getting to know the particular person right in front of you in testing add value and the best results.
• “With interview-based tasks, participants take us down paths we never expect to go”
  I love this fact! We are so directed by our own understandings, experiences, and mental models. Quality UXD is in the commitment to LISTEN.
• “Terminology emerges to describe user needs in a way we hadn’t previously thought.”
  An open card sort in the context of usability testing – nice.

As always, thanks Jared for sharing.

The challenge in dealing with conformitory bias – “People can reinforce their existing attitudes by selectively collecting new evidence, by interpreting evidence in a biased way or by selectively recalling information from memory.” – is real. Getting “over yourself” and seeking out valid user testing and feedback is a tough call when you’re doing it all yourself, design and testing.

The points that Paul J. Sherman makes in his followup article that strike me most are these:

• Testing is not a Pass/Fail activity – it is a valuable element in the Design Process. Focusing the testing as a part of the path to a valid, usable solution is key.
• Test early, often, and as lo-fi as possible (keeping the initial investment, emotionally as well as in time/cost down).
• Usability testing serves design – use it to improve design.