NOTE: This is the high level test plan for the study. Please feel free to edit and add to it. Although the team has agreed upon the methodology for the study and the focus on the tasks, the wordings for the tasks/scenario have not been finalized yet. WORK IN PROGRESS DOCUMENT.
Project Name: UMN Usability Study
Location: University of Minnesota
Date: May 17-19, 2011
Purpose:
The goal is to learn about the first experience of using Drupal and uncovering usability issues through a community based usability study.
Background:
Usability has been getting increasingly important for Drupal. There is a lot of welcome rush from the community for a more usable Drupal. This project aims to keep this momentum forward. The last usability study was conducted in 2008.
How Can I Help?
While several attending members will be fully or partially sponsored by their employers, others are coming in from overseas, do not directly benefit financially from Drupal, and need help. Your contribution of any amount will help make this event happen and contribute to the ongoing improvement to the Drupal user experience. Thanks!!
Audience:
Undergraduate and graduate students in Computer Science and Design from University of Minnesota with NO experience of using Drupal but have an interest/need of building websites. Note we are focusing on students for this particular study as the study is in a university setting.
Business Goals:
Study Objectives:
- Observe and understand how the first time user interact with Drupal and also learn what is his/her mental model while building websites.
- Evaluate the ease of use of completing common tasks on Drupal with respect to:
- Terminology
- Concept
- Work flow
- Usability
- Mental Model
- Features
- Other
Participants:
We are targeting users who have a need or interest in building websites (site builders) and have never used Drupal before. Our goal is to reserve 8 participants. Participants will receive $45 (USD) each as compensation for their time and comments.
Methodology:
Each session will be one on one with a trained moderator in lab settings at the usability lab at University of Minnesota. The session would be for 75 minutes, and the participants will not be required to sign a non-disclosure agreement (right?). However, they will be required to sign a consent form. The computer screen and the audio of the participants will be recorded. This is done for analysis and to share with the community. The study will be conducted in three parts: background questions, tasks with Drupal, and overall impressions.
Tasks:
Participants will be given a low fidelity wireframe and will be asked to reproduce the wireframe to a website. The wireframes will comprise of 5 important tasks (plus, additional tasks if time permits). Although the goal is to get to all the 5 tasks, the order in which a participant goes through them is not important. Participants can choose to perform the task in the order they would normally do.
Scenario:
You are an active member of an organization. The organization has an upcoming conference. You volunteered to help design the website for the conference.
Task 1: Now that you have installed Drupal, you would like to take some time and explore the system. Take a few moments and let me know what are your first impressions about the system.
Focus: To understand how participants would approach the task and get insights about their work methodology and mental models.
Task 2: You would like to create an “About Us” page that is accessible from the main menu. How would you do that?
Focus: Create a page, link to the main menu
Task 3: You would like to change the visual look of your website. What would you do?
Focus: Themes
Task 4: You would also like to have a small blurb about the conference on the homepage that looks something like this . How would go about doing this?
Focus: Create a block, place a block
Task 5: One of the members of the conference is insisting you to have a calendar or twitter feed (not decided yet) on the website. How would you go about doing that?
Focus: Install a module
Additional Tasks:
Create content types
Create a shortcut