script for usability testing at University of Baltimore

Posted by beccascollan on April 10, 2008 at 8:29pm

An update on usability testing at the University of Baltimore - I ended up putting together a very simple scenario involving page creation, navigation and permissions using the IRT node form - vertical tabs layout. I've tested this script in a session with our campus webmaster, and he was able to complete the tasks within the hour we had, but did have a couple problems. If you have any thoughts on the script, please post.

One question in my head lately - has any Drupal user data ever been collected? Surveys or other research? Or something from within the Drupal developer community? It's usually the first step in this process, and I'd be curious to see what exists.

the Script

Problem Statements:
Is the language and terms used in the administrative interface intuitive to users?
Is the format and language of the proposed IRT node form accessible and intuitive to users?
How long does it take users to perform basic web development tasks with no formal Drupal training?

Introduction [read to participant]
Drupal is an open source content management system developed by a large, international community. The Drupal community would like to make the CMS more useful and easy to use for site administrators such as yourself. We know you're very capable and familiar with working with such applications, but please feel open to letting us know whenever something seems confusing or you're about to hunt around for an answer, and talk aloud any issues as they come up. We're testing the CMS, not your abilities, so don't feel uncomfortable with pointing out anything that makes your work difficult. Anyway, we're not Drupal developers ourselves, so we won't take it to heart.

Process
Tell me about your process for:
- - choosing a content management system [Is it determined by someone else? What do they usually look for? How do you feel about their decision - what matters to you about it?]

Familiarization [starting at welcome page]
Take a moment to look at the main site administration page. Before taking another step:

What are you thinking of doing first?
What is your sense of where you can go, and what you can do from this page?
What is your overall impression of the page - do you feel like you can do what you need to do?

[provide pre-test questionnaire]

Tasks

Introduce scenario & provide site content:

Lets pretend your client is the non-profit art gallery and education center Gallery 53.

You’ve chosen Drupal for the CMS. You’ve uploaded Drupal to the server, configured the database and completed all of the initial installation steps.

The staff at Gallery 53 want to see their mission statement on the home page, an "about us" landing page, and a course schedule page. They would also like their board members to be able to log in and create or edit any of the content on the site, but not play around with anything else.

Task 1:
Put the mission statement text on the site. [Please say aloud when you’ve completed the task.]

[Let them know they don’t have to type in all the text]

Follow up questions
- - review key choices participant made - why did you choose ____ ?
- - where did you expect to find ____ ?
- - what expectations do you have for locating and editing this content now that it's up?

Task 2:
Gallery 53 would like an "About Us" landing page. Create this page with the history and current board members. It's URL should read /about