Content Creation UX Study Plan

Introduction

Drupal 8 wants to provide a much-improved workflow for content creators. One of the most critical screens in this is the content creation page. This issue outlines a plan for implementing a new design for this page.

The main issues that this design wants to solve are:

• Make the publishing status of the content item much more obvious: Move the publishing state selection out of vertical tabs and position it closer to the Publish/Save actions. Provide a clear status label in the sidebar.
• Provide clearer seperation between content and settings: Make the vertical tabs less screen-space consuming by moving them into a sidebar as an accordion.
• Update and expand the visual language of the Seven admin theme to express all of the above in a clear and elegant form.

After some great discussions and design mockups shared in the issue queue,the community has come up with a mockup. In order to make sure we do provide a much-improved workflow, validation is required. Hence, we are about to kick off a usability study on this mockup.

This document outlines the following:

1. Goals and methodology for this study
2. How to find the right user for this study? (Screener to qualify participants)
3. How to conduct this study? (A guide of tasks, and questions that we will be asking)
4. How can you help?

1. Goals and Methodology

• Name of the project: Content Creation UX Study
• Focus of the project: Drupal is looking for 10 participants for 30-minute usability sessions to be conducted remotely (online) or in person. We are looking for participants who create content on any CMS-driven (Content Management System) site.
  
  We are looking for participants anywhere in the world.
  
  The study will be conducted on two parts of our audience that will run simultaneously.
  5 Participants: New to Drupal users who write content
  5 Participants: Existing Drupal users who write content

• Type of study:Moderated
• Length of every session: 30 minutes
• Compensation to participants: None
• Recruitment method:Community
• Moderators: UX team / community
• UX Team lead: Dharmesh Mistry (@dcmistry)

Important dates

• Wednesday, April 11: Initial draft of test plan
• Friday, April 13:Final draft of test plan,
• Saturday, April 14 to May 4,2012: Conduct tests
• Sunday, April 22:All testing ends, all results are in the analysis sheet.
• TBD:All results have been analyzed, and final results are posted.

2. How can you find the right user for the study?

(Screener to qualify participants)

We are looking for 5 users who are content creators on a CMS but not on Drupal and 5 users for existing Drupal users. More information can be found in this screener: 

3. How can YOU conduct this study?

Below is a script to use.

Participant Briefing

• Welcome, my name is X and I am helping Drupal on a project
• This session will be a couple questions, a series of tasks, then a couple post-session questions.
• As you go through the session, please think aloud. By that what I mean:
  • Describe your steps… what you are looking for… etc
  • Your comments are what is very important to us – we ask that you give open and candid opinions, both good and bad
  • Ask for clarification if needed, I will be be neutral throughout the test
• Keep in mind we are testing the software, not you. Also, what we are going to see today is a prototype and it is a work in progress. But please continue to work and comment as you would expect a full version to work as.
• My role is going to be of a neutral observer, so anything good or bad you say is not going to affect my feelings. So please feel free to give candid opinion.
• Try to complete the tasks as if you were doing this for real. Spend as little or as much time as you normally would doing these task. It is ok if you cannot complete each task, and we may not get to every task
• We will be recording the session, so that might  share anything relevant with the Drupal community
• We just record your audio and the screen, so use mouse when describing an area of screen.
• Do I have your permission to record this session?
• We may post the highlights from this conversation online. Your full name or any personally identifiable information will not be associated with the data.
• This should take about 30 minutes. The time is now ___ so we should be done around ___.
• Any questions before we begin?

Study Guide

NOTE: This is a combined study guide for “New User Study” and “Existing User Study”. The questions that are specific to “Existing User Study” are marked accordingly.
 

Pre-Session Questionnaire (Estimated time: 7 mins)

1. What is your role when it comes to content creation? Also, tell us briefly if your responsibilities related to the site?
2. If you have to write to a blog or an article, what is the process you follow? (Note to the moderator: We are looking for high level details only, so don’t spend a lot of participant’s time on this question)
3. Do you write the content on your blog/website editor or any other tool and paste it to your site or someway else?
4. How many other content creators (rough estimate) do you have on your website?
5. (If content maintainers) What is your role when it comes to maintaining content on the website?
6. When it comes to adding/editing content, how do each of the users differ from each other?
7. (Ask only for existing users study only) How has been your current experience with creating content on Drupal/ Drupal Gardens? What are the things that you like or do not like?
8. (If time permits): Do you use any assistive editors while creating content? e.g. WYSIWYG, markdown, etc?

Scenario 1: (Estimated time: 17 mins)


You write for a popular food blog. Last weekend, you tried a new recipe for strawberry shortcake and you are excited to share it with your readers.
 

9. Task 1:
   Can you describe how you would go about doing that?
    
10. Subtask 1.1:
    You also want to make sure that your readers have the chance to able to say things regarding the recipe.
    <Note for the participant: You can create dummy content>
    
11. Subtask 1.2:
    Also, You would like to add tag this content with “Dessert” and “Strawberry”. How would you do that?

Probing Questions:

12. Once you are satisfied with the content that you wrote, what would you normally do?
13. If the participant does not comment or use, ask them about it:
    • What is “Edit Summary”? What does it do? Is it helpful or not helpful?
    • Did you notice or not notice the “Filtered HTML/ Plain Text” drop down? What do you think it does?
    • What does “Save”, “Preview” do?
    • What will happen if you hit “Save”? <Try to understand that if the expect that their content will be published by default>
    • Look at the options on the right hand side and tell us what do you think about them?
    • Take a look at each of the options at the bottom of the page and let me know what do you think about them?
    • Are they helpful or not helpful?
    • Are they clear or not clear?

OBSERVE:

• Do people understand the top part of the sidebar (what kind of information it contains)?
• Do participants notice the sidebar options?
• Is it clear that the sidebar contains "meta" information?
• Do people understand that they can open and close the tabs?
• Do people notice the submit buttons?

Post task question:

Scenario 2: (Estimated time: 7 mins)

In an article that you wrote a few weeks back on chocolate chip cookies, you realized that adding one cup of walnut makes the cookies awesomer.

Probing questions:

Post Session questions:

Thank the participant: Thank you again for all the comments and time! Your feedback is very well appreciated.

4. How can I help?

If you have a few hours to contribute between Saturday, April 14, 2012 and Sunday, April 22, 2012, we could really use your help!

You could:

• Help us recruit participants (friends or family members who are willing to spend 30 minutes of their time giving feedback)
• Help us conduct the study: You can conduct the study yourself. It is pretty easy, trust me! Plus, it is a lot of fun. Here is the handbook on how to conduct usability studies

If you want to help out please let us know, in this thread or over e-mail.

Special thanks to @lisarex, @yoroy and @bojhan for the feedback. But we need more … and your feedback would be very much appreciated!