Drupal 7 Usability Study at Google: Results and Notes

This is superb! Any plan on how we should go about doing the analysis?

Incomplete observations I made during the testing (in no particular order). Some of these have issues associated with them already.

• multi-step creation processes for field UI, views, etc. REALLY confuse people. would a 1 > 2 > 3 navigation be helpful? or should we simply skip advanced settings and let the tester navigate into edit
• people could not tell how to use menus
• the navigation menu block confuses people terribly
• The "add content" link in the navigation block made people think they'd use that to add items to that menu block
• "tabs" vs. "menus" confused poeple a lot (bartik primary nav theme)
• previews are terribly confusing
• teaser preview in particular was unfamiliar / not clear in its relationship to full content
• links in previews allow people to nuke their content
• filter tip links allow people to nuke their content
• the views save button on the upper right confused every participant
• the seeming distinction between "Add content" and "add new content" confused people
• On the other hand, many participants were confused by scrolling in the overlay, or thought they should not have to scroll to the bottom to save
• the local task links in admin menu confused everyone
• it's important to tag projects with categories on d.o (participant was not able to find date module)
• the words "image styles" do not have much meaning for people, and it was hard for people to see where to create new ones. people expected something in the context of the display settings tab (once they found it)
• manage fields was very, very confusing for everyone.
• people seemed to miss local tasks, or at least took awhile and/or prompting to find them
• lack of wysiwyg came up right away in the standard profile tests
• on the other hand, the combination of filter tips + wysiwyg also confused people in the media profile tests--folks thought they could type markup in the wysiwyg, which would actually have been escaped to HTML entities
• testers had a concept of “developer view” vs. “user view” that corresponded somewhat, but not entirely, to presence/absence of overlay
• almost all testers did not see filter tips in the first 3-4 looks over the forms, or had to be prompted to see them
• several testers found admin_menu somewhat overwhelming, and/or spent a lot of time looking through it instead of prompts in the content area
• testers used toolbar effectively, and generally learned to use “Structure”
• some testers were confused by changing breadcrumbs and/or expected breadcrumbs to reflect their click history rather than navigation structure
• testers “lost” their nodes after they were created
• “add new field” vs “existing field” confused people
• testers tended to overlook the field label box, and were confused by what to put in the field name box
• testers’ attention tended to be drawn to vertical tabs. some clicked through all of them; most looked over the content of the first tab. (so, the first tab should contain the most important info?)
• menu weights confused testers because it was not clear what the weights were relative to. also, one thought the weight selection widget was “crazy” (# of values)
• testers expected to be able to add fields from the node add form (rather than content type form)
• most testers managed to add a term reference field eventually (though it took a lot of searching and prompts), but most did not discover the concept of taxonomy/vocabularies/existing terms
• testers did not really understand the “promoted to front page” concept
• it was often necessary to prompt testers to “just click save” when they were overwhelmed by a new form
• testers expected to create a gallery first and then add content to it, but script used the pattern of creating content and then guiding to views to create the content
• testers all thought views was cool once they understood it a little :)
• testers had trouble finding manage display tab for field formatters
• testers often not sure where views, content, etc. “went” when saved--expected putting them in menus, but generally didn’t see how

I have been watching the posts on these testing, very interesting. Once thing I would like to add is a known issue from almost every user i have talked to. In D6 i created some code to tackle this issue in a custom block. for D7 i created a module to handle it, but in any other words its a massive problem.

How do I create content?
Where is my profile?
(if privatemsg is installed) where are my messages?

I found them in two different XChat logs, so 7 has unix timestamps and 8 has UTC. (I'm not sure why XChat has logs in two places...)

I'm missing the first couple of minutes of user 8, while I was trying to get XChat to start logging again.

The one that gets me the most on Drupal is the Views so many options that unless you spend hours in there setting parameters and going to look you don't have any use for it.

I have over 200 hours with Drupal and I am still daunted by some functions and some not working and other things. Right now I have comments and poll errors popping up on a new install.

The four emerging themes from the study

The issues our users ran into can be categorized by one (or more) of the following problem areas.

• Interface fails: there are a number of issues that reflect obvious broken elements throughout the Drupal interface. Here we are talking about things like users clicking on links in node preview and then losing all their information, or users continually not seeing the save button from the image styles interface.
• Terminology confusion: new users do not understand Drupalspeak. We have documented this problem across Drupal as well as the contrib modules we tested. “Why isn’t Views called Lists?” “What is filtered HTML?”
• Product flow and mental model mismatches: some issues reflect general confusion about where to go next, or how to proceed after taking an action. Users think they are performing one action, yet they are doing something totally different, or they think a function should be found in one place, yet it’s in another. Example: users looking for image styles under manage fields, or the successful installation of a module doesn’t inform a user it still needs to be enabled.
• Conceptual problems: these are the fun ones. These are the issues when users are just not getting it, and Drupal ain’t doing nothing to help them out. “What is a field? Where does my content live?” etc.

To view the high-level usability report from this study, see:
http://groups.drupal.org/node/218959