Toolbar Usability Study Findings

Bojhan and Dharmesh recently conducted a 6-participant moderated usability study on the new toolbar. The design was tested on desktop and mobile (and on the iPad in some cases).

Summary of findings
Overall, the toolbar prototype tested well on desktop and iPhones.

“This is a big improvement from where we are at [right now]” (P4)

Most of the participants (4 out of 6) had positive reaction. The rest of the participants (2) had a neutral response to the change. The mobile version was significantly better received than the desktop experience.

It is also worth noting that participants were mostly neutral about having different mobile vs. desktop versions of the toolbar.

Major Positives:

• Participants thought that the design was clean, usable, with “nice” icons and was visually appealing.
• Participants responded very positively to using it on a mobile phone, it was recognizable and easy to navigate.

Major Issues:

• Collapsing menu items through “>” and the link “Edit shortcuts” are not discoverable. (https://skitch.com/dcmistry/es917/people-simply-recipes).

Detailed Findings
Issues:

• The drop down item “>” isn’t discoverable: All participants failed to notice or did not click on the “>” icon. After receiving assist from the moderator, they liked what it does but did not think the icon was correct.
• Almost all participants (5 of 6 participants) had trouble noticing the "Edit shortcut" and thought need a little more visual emphasis. This is an important one because shortcuts were very important to all of the participants.
• “Extend” was unclear to intermediate participants in this test: Half of the participants thought that “Extend” wasn’t the correct taxonomy label.
• "I have an English degree and I think 'Extend' is not a good word for Modules" (P4)
• Issue on Android phone: When switching to landscape orientation, clicking on a “>” to expand the items below would often select the next item instead of exposing the sub-items. For example: Trying to expand the “structure” menu options, “Appearance” gets selected. The participant tried this several times and yet, the problem persisted.
• The mobile version has icons and no supporting texts for Menu, Home and Account. One participant who saw the design on the mobile first, did not know where to begin as she did not recognize/ understand the menu icon.
• One participant had concern that everything was underneath “Menu” which would reduce productivity. He suggested to bubble up the shortcuts.
• Participants (2 of 6 participants) occasionally triggered the “URL address” bar and its keyboard on the iPhone when tried to tap the “Menu”, because its relatively close to each other.

Other observations/comments:

• We ran into bugs all over the place, from vertical tabs not collapsing nicely, image upload not working, to not always seeing the full menu on the mobile phone, the toolbar being cut off on wide pages, and some android versions missing all icons.
• 2 of 6 participants wanted “Home” to be of the left most side of the interface because that’s what they are used to.
• 2 of 6 participants expected flyouts on hover for “>”
• One participant strongly suggested to reduce the size of the toolbar on the desktop whereas the mobile toolbar size could be the way it is or could expand to take the whole screen.
• One participant wasn’t sure about the benefits of the new design (desktop) over the current experience. And expressed a need to be able to add, more to the top bar items.

Participant profile:

• Intermediate Drupal users who are developers (3), site builders (3), themers (2), trainer (1) and content maintainer (1)
• Participants either use Drupal 7 toolbar or the admin toolbar and thought the respective toolbars did “what it is suppose to do”
• Participant needs to accessing/ updating their website via a phone was to keep it quick and focus on small/ critical tasks only. If they needed something more, they would find a desktop.
• Participants use their smart phones primarily to browse sites, check emails and visit social media.
• We are awaiting release forms from the participants before we make the recordings public.