Goal
The goal of the study was to uncover usability issues with the new prototype for the content creation experience for Drupal 8. The study was conducted with new and existing users.
Methodology & Participant Demographics
- 11 participants ( 6 existing users and 5 new users) were tested for this study. Based on the availability of the participant and flexibility of the contributor, the studies were moderated (5 existing users and 1 new users) or unmoderated (1 existing user and 4 new users) using userfeel.com
- Sessions lasted from 30 minutes to 60 minutes.
- Participants of this study write/edit content on a content management system and write content at least more than once a month. The participant screener can be found here: https://docs.google.com/a/acquia.com/document/d/1KcL3aNUceVKXi9HLAubZv70...
- Participants were asked to think aloud and perform tasks followed by a brief scenario. The study plan can be found here: http://groups.drupal.org/node/223959
- The study was conducted in April 2012.
Executive Summary
-
Overall, participants had a neutral or mildly favorable response to the content creation prototype.
- “It is very reminiscent of Drupal so I don’t have to learn something new... It is neutral good.” (Existing user - participant 1)
- “Did not feel intimidated” (Existing user - participant 4)
- “It is a user friendly interface... I would definitely consider this one as a long term basis” (Existing user - participant 5)
- “I don’t think I have a preference. I will go with the new one because it is the latest.” [Paraphrased Comment] (Existing user - participant 6)
- Participants found it easy to add content (create and edit) and it was clear to participants that the sidebar is related to the content but it is not central to the activity of content creation.
- People expect to see a preview within the site context.
- “A bit disappointed. It does not show it in the site” (New user - participant 5),
- “Want to see how it looks on the website” (Existing user - Participant 6)
- It’s often not immediately clear what the difference is between the two views.
- However, a number of high and medium severity issues were encountered (not necessarily pointed toward the new design).
- To many new and existing users, “Edit Summary” was unclear
- New users did not understand the HTML filters clearly
- Many participants asked for a real preview so that they could see their content in context
- A number of isolated issues exist around the menu options (see below for more details)
- Some new users did not like that “Save” publishes their content.
Contributors
We would like to thank everyone who helped us find participants. And especially thank Dharmesh Mistry (for organizing it all), Danielle Sheffler, Ryan Armstrong, yannisc for helping us test Drupal with the participants and running the analysis.
Follow up
Now we need to solve all of the issues that where found, an detailed report of issues are added to the main issue http://drupal.org/node/1510532 and we have opened the following new ones:
Edit Summary label is unclear to users
Menu settings on content creation are unclear
Its hard for users to provide valid input in the authoring date field
Allow for more images in article
"Sticky on top" is hard to understand