Introduction
Drupal 8 wants to provide a much-improved workflow for content creators. One of the most critical screens in this is the content creation page. This issue outlines a plan for implementing a new design for this page.
The main issues that this design wants to solve are:
- Make the publishing status of the content item much more obvious: Move the publishing state selection out of vertical tabs and position it closer to the Publish/Save actions. Provide a clear status label in the sidebar.
- Provide clearer seperation between content and settings: Make the vertical tabs less screen-space consuming by moving them into a sidebar as an accordion.
- Update and expand the visual language of the Seven admin theme to express all of the above in a clear and elegant form.
After some great discussions and design mockups shared in the issue queue,the community has come up with a mockup. In order to make sure we do provide a much-improved workflow, validation is required. Hence, we are about to kick off a usability study on this mockup.
This document outlines the following:
- Goals and methodology for this study
- How to find the right user for this study? (Screener to qualify participants)
- How to conduct this study? (A guide of tasks, and questions that we will be asking)
- How can you help?
1. Goals and Methodology
- Name of the project: Content Creation UX Study
- Focus of the project: Drupal is looking for 10 participants for 30-minute usability sessions to be conducted remotely (online) or in person. We are looking for participants who create content on any CMS-driven (Content Management System) site.
We are looking for participants anywhere in the world.
The study will be conducted on two parts of our audience that will run simultaneously.
5 Participants: New to Drupal users who write content
5 Participants: Existing Drupal users who write content
- Type of study:Moderated
- Length of every session: 30 minutes
- Compensation to participants: None
- Recruitment method:Community
- Moderators: UX team / community
- UX Team lead: Dharmesh Mistry (@dcmistry)
Important dates
- Wednesday, April 11: Initial draft of test plan
- Friday, April 13:Final draft of test plan,
- Saturday, April 14 to May 4,2012: Conduct tests
- Sunday, April 22:All testing ends, all results are in the analysis sheet.
- TBD:All results have been analyzed, and final results are posted.
2. How can you find the right user for the study?
(Screener to qualify participants)
We are looking for 5 users who are content creators on a CMS but not on Drupal and 5 users for existing Drupal users. More information can be found in this screener:
3. How can YOU conduct this study?
Below is a script to use.
Participant Briefing
- Welcome, my name is X and I am helping Drupal on a project
- This session will be a couple questions, a series of tasks, then a couple post-session questions.
- As you go through the session, please think aloud. By that what I mean:
- Describe your steps… what you are looking for… etc
- Your comments are what is very important to us – we ask that you give open and candid opinions, both good and bad
- Ask for clarification if needed, I will be be neutral throughout the test
- Keep in mind we are testing the software, not you. Also, what we are going to see today is a prototype and it is a work in progress. But please continue to work and comment as you would expect a full version to work as.
- My role is going to be of a neutral observer, so anything good or bad you say is not going to affect my feelings. So please feel free to give candid opinion.
- Try to complete the tasks as if you were doing this for real. Spend as little or as much time as you normally would doing these task. It is ok if you cannot complete each task, and we may not get to every task
- We will be recording the session, so that might share anything relevant with the Drupal community
- We just record your audio and the screen, so use mouse when describing an area of screen.
- Do I have your permission to record this session?
- We may post the highlights from this conversation online. Your full name or any personally identifiable information will not be associated with the data.
- This should take about 30 minutes. The time is now ___ so we should be done around ___.
- Any questions before we begin?
Study Guide
NOTE: This is a combined study guide for “New User Study” and “Existing User Study”. The questions that are specific to “Existing User Study” are marked accordingly.
Pre-Session Questionnaire (Estimated time: 7 mins)
- What is your role when it comes to content creation? Also, tell us briefly if your responsibilities related to the site?
- If you have to write to a blog or an article, what is the process you follow? (Note to the moderator: We are looking for high level details only, so don’t spend a lot of participant’s time on this question)
- Do you write the content on your blog/website editor or any other tool and paste it to your site or someway else?
- How many other content creators (rough estimate) do you have on your website?
- (If content maintainers) What is your role when it comes to maintaining content on the website?
- When it comes to adding/editing content, how do each of the users differ from each other?
- (Ask only for existing users study only) How has been your current experience with creating content on Drupal/ Drupal Gardens? What are the things that you like or do not like?
- (If time permits): Do you use any assistive editors while creating content? e.g. WYSIWYG, markdown, etc?
- Task 1:
Can you describe how you would go about doing that?
- Subtask 1.1:
You also want to make sure that your readers have the chance to able to say things regarding the recipe.
<Note for the participant: You can create dummy content> - Subtask 1.2:
Also, You would like to add tag this content with “Dessert” and “Strawberry”. How would you do that? - Once you are satisfied with the content that you wrote, what would you normally do?
- If the participant does not comment or use, ask them about it:
- What is “Edit Summary”? What does it do? Is it helpful or not helpful?
- Did you notice or not notice the “Filtered HTML/ Plain Text” drop down? What do you think it does?
- What does “Save”, “Preview” do?
- What will happen if you hit “Save”? <Try to understand that if the expect that their content will be published by default>
- Look at the options on the right hand side and tell us what do you think about them?
- Take a look at each of the options at the bottom of the page and let me know what do you think about them?
- Are they helpful or not helpful?
- Are they clear or not clear?
Scenario 1: (Estimated time: 17 mins)
You write for a popular food blog. Last weekend, you tried a new recipe for strawberry shortcake and you are excited to share it with your readers.
Probing Questions:
OBSERVE:
- Do people understand the top part of the sidebar (what kind of information it contains)?
- Do participants notice the sidebar options?
- Is it clear that the sidebar contains "meta" information?
- Do people understand that they can open and close the tabs?
- Do people notice the submit buttons?
Post task question:
Scenario 2: (Estimated time: 7 mins)
In an article that you wrote a few weeks back on chocolate chip cookies, you realized that adding one cup of walnut makes the cookies awesomer.
Probing questions:
Post task questions: (Estimated time: 2 mins)
Post Session questions:
Thank the participant: Thank you again for all the comments and time! Your feedback is very well appreciated.
4. How can I help?
If you have a few hours to contribute between Saturday, April 14, 2012 and Sunday, April 22, 2012, we could really use your help!
You could:
- Help us recruit participants (friends or family members who are willing to spend 30 minutes of their time giving feedback)
- Help us conduct the study: You can conduct the study yourself. It is pretty easy, trust me! Plus, it is a lot of fun. Here is the handbook on how to conduct usability studies
If you want to help out please let us know, in this thread or over e-mail.
Special thanks to @lisarex, @yoroy and @bojhan for the feedback. But we need more … and your feedback would be very much appreciated!
Comments
@dcmistry Thanks for putting
@dcmistry Thanks for putting this out!
We are going to start testing quite soon, if anyone can help us out even by only doing one participant or referring a friend, please let us know!
Testing
I have a strong usability background and have conducted several usability tests with past clients. I'm more than happy to help!
Danielle Sheffler
Engagement Manager at Acquia
http://www.acquia.com
Woho!
Awesome, DSheffler: Looking forward to working with you :)
Dharmesh Mistry
UX Researcher | Acquia,Inc.
We can help
I can definitely recruit a few people to participate from our team/company, including at least one person who uses Drupal daily, and one who use a ahem different CMS.
Updates
Hello @Cherringland,
Any updates? You can send me the e-mails (with time zones, and country) of the participants to me (dharmesh.mistry@acquia.com).
Dharmesh Mistry
UX Researcher | Acquia,Inc.
We can help
I can definitely recruit a few people to participate from our team/company, including at least one person who uses Drupal daily, and one who use a ahem different CMS.
Nice!
@Cherringland,
Fantastico! You have no idea this is going to help us so much with the project.
So, the next steps would be:
Dharmesh Mistry
UX Researcher | Acquia,Inc.
userfeel.com - willing to offer 5 tests
Hello Guys!
I'm the owner of netstudio, a Drupal development services company in Athens, Greece. We have developed www.userfeel.com, a remote usability testing platform.
While remote usability testing is not the same as normal usability testing, it can provide quick and easy to get feedback. The main difference is that the test is not attended, that means you have to provide the whole script in advance and have the user follow the steps by himself. Also, there is a 15 minutes time limit on the tasks that the user should complete. The final result is a video with the user's actions and voice feedback and also written answers to 4 questions.
I'm willing to offer 5 sessions for free if you think that would be helpful for the task described. I'll need the scenario somewhat adapted to remote usability needs. I can do the screening based on your criteria and try to find the right testers among our testers panel.
Please tell me if I should proceed with this.
First remote usability test ready to view.
I got ahead and screened the users. Found 7 that match our criteria and posted the test for them.
Here is the first result: http://www.userfeel.com/result/NGY4Y09ESXdOdz09NGY4Yzdm
I'll have the rest ready in the next hours.
Yannisc, You rock! I am
Yannisc,
You rock!
I am coordinating with other contributors for this study. Can you tell us if they are existing or new users?
Dharmesh Mistry
UX Researcher | Acquia,Inc.
2 of them are existing users,
2 of them are existing users, the others are not.
The one I posted already is not an existing user.
Super!
Sweet!
Can you update this sheet when you get a chance?
Dharmesh Mistry
UX Researcher | Acquia,Inc.
Dharmesh, I updated the sheet
Dharmesh, I updated the sheet with the first 4 tests.
I've also uploaded them to YouTube for easier (maybe) commenting and sharing:
http://youtu.be/PNJCHYGZHFQ
http://youtu.be/MW5EeSTLTYM
http://youtu.be/g7Ciz0mxeMM
http://youtu.be/UYtVuc5a82M
(I've added the URL's in the sheet as well).
@ÿannisc That was fast,
@ÿannisc That was fast, wasn't aware you where already testing (we have actual test instances, which where clean). Next time please catch us, before proceeding because I saw you also didn't change the test plan to fit remote testing (observer notes, where mixed with questions).
Either way, great to see you have already shared the results! I am watching them now, the actual videos are sometimes hard to see, is there any way you could improve the quality?
Bojhan, I have adapted the
Bojhan, I have adapted the scenario to remote testing as I saw fit, but ofcourse I'm open to improvements to this one to use with the next tests.
Please also point me to the clean installation to use that also.
As for the video quality, unfortunately, we cannot have it better. Compression is applied for faster uploading on the tester's side.
thanks,
Yannis
@yannisc Its mostly 13, that
@yannisc Its mostly 13, that might need some changes - I have noticed most of them answered this after completing the task and often skipped over parts. You could consider leaving out that one.
See the sheets for the clear installations to use.
Contributors moderating
Contributors moderating sessions,
I have created an analysis sheet to put your notes from the sessions. Right now it is a private document but will ultimately be made available.
Please drop me a line if you want to access the notes.
Dharmesh Mistry
UX Researcher | Acquia,Inc.
New users for testing
I have some contacts that are content editors that are new to Drupal. Two of them can do this testing with me, if still needed?
We've have done several usability tests with out past clients.
Let me know! Seems like you've got what you need in the Participant Sheet, but I am happy to give some extra input on this if needed :)
Thor Andre Gretland | Frontkom | Twitter: @thorandreg
Thorande, Great! We are
Thorande,
Great! We are looking for participants who haven't used Drupal but use other CMS. We have enough of participants who use Drupal. Would you know of any such participants?
Dharmesh Mistry
UX Researcher | Acquia,Inc.
Sounds great! Unfortunately I
Sounds great! Unfortunately I have missed such nice opportunity so anyone can say me that when it will be held in future?