Usability Report from the Drupal 7 UX Study at Google

You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!

This past February Garen Checkley, Jen Lampton and I ran the Drupal 7 Usability Study at Google. We created a handbook page on drupal.org to share the exhaustive list of usability issues. While we learned about a lot of specific interface and module-level problems, we also saw broad trends that called for a higher-level analysis.

You can view this report here.

We organized the usability issues into four different layers: conceptual, flow, terminology, and interface. In this document we explain what each of these means, and provide key examples and quotes from users. We also propose a quick and concise up-to-speed aid that welcomes users on a new Drupal installation.

Garen, Jen and I will be discussing this report and showing a mock-up of an up-to-speed aid during our core conversation at DrupalCon Denver this Wednesday (3/21/12) at 1pm!

Download the slides from our DrupalCon presentation.
If you want to play around with the mocks we made using FireWorks, you can download the original file.

As we say in the report:
Drupal’s learning curve doesn’t have to be this steep; let’s shape Drupal to become more helpful and supportive for new users.

AttachmentSize
drupal-google-ux-report-2012.pdf1.78 MB
DrupalCon-Denver-2012-03-21-Presentation.pdf6.67 MB
Drupalcon-denver-2012-03-21-up-to-speed-aid-fireworks.png865.3 KB

Comments

Interesting report. However,

greta_drupal's picture

Interesting report. However, I'd have like to see the report clarify who the "user" is in this case. My initial assumption was that it is the content creator; so why would they be creating content types. Apparently, this report was targeting new/prospective site builders, rather than end user. Would be good to make that clarification.

Afterall, there are several types of users of a Drupal:
Drupal Sitebuilder / Developer
Site Admin
Site Staff Content Creators
Site Visitors

I'm always puzzled by Drupal being described as steep learning curve. Perhaps if you are a sitebuilder (although I don't remember it being so difficult) or a developer. But, for the end user, how hard is it to fill in the blanks of a form? And, at the basic user level, this is all that is required.

Beyond that, the care of the sitebuilder (UX/UI) makes a difference.

Good pt

aharown07's picture

greta_drupal's got a solid point... more than one. About four yrs ago I found myself needing to become a site developer having never worked with any kind of CMS before.
After several frustrating attempts to ramp up to various CMS/frameworks over a period of months, got hooked on Drupal mostly because I found it relatively easy to learn. In particular, I needed to be able to figure it out in relatively small pieces.

Site visitors don't have trouble at all with the site experience. Staff content creators have had very little to learn. I had a ton to learn but found it to be pretty accessible.
It may well be that content systems that are easy for inexperienced builders to learn must, by nature, not be so easy for content creators to work with out of the box.

Totally Agree!

danreb's picture

Totally agree with this as I think the one that need to create content types, install modules, create views is the site builder / Developer. I believe content creator or editor, website owners that is not techie don't need to create content types, install modules and create views, you need to create all this things for them if you are the site builder.

Content creator / editor, website owners can easily create new post/page especially if you've installed a WYSIWYG editor and its not that hard to learn, if you want to confuse them then give them all the permission that superuser(user 1) have, but enabling just the necessary permission for each role will eliminate and reduce the confusion.

BTW any improvement with the UX/UI is also good and great for Drupal.

But, for the end user, how

escoles's picture

But, for the end user, how hard is it to fill in the blanks of a form? And, at the basic user level, this is all that is required.

First: If the forms are not designed for usability, very.

Second: No, that's not all that's required, and I've never understood why site builders think that it is. Because, as you point out, there are different classes of user -- but as you don't note, for most drupal sites (the much-maligned "brochureware" and "blog" use-cases probably account for over 80% of drupal sites), there is no "basic user" -- there's a "gets stuck with keeping the site up" user, who has to either call for help to do anything more basic than "filling out the forms" they've been rote-instructed to fill, or take the plunge and try to do it themselves. The former is seldom an attractive prospect when you need to get your work done; the latter is both unattractive and, in the case of Drupal, difficult.

In other words: Yes, there are different user types, but they're not nearly as cleanly-defined as you seem to think they are.

So, bona fides: I've been building Drupal sites for pay since 4.x (have run my personal sites on Drupal since 3.x). I've trained a lot of people to use the system, and have dealt a lot with the question of what happens when we stop getting paid. Because we do stop getting paid, at some point, and "spend time learning the system" is not something clients want to hear at that point.

Thanks for providing this resource

User Advocate's picture

This has been an excellent undertaking and has provided us all with valuable resource info for understanding a range of usability concerns. At Myplanet we're still going through the videos to analyze them. We're approaching this using three conceptual frames of reference:

  • Role analysis (when do user change goals or mindset)
  • User Narratives (how effectively do the textual cues guide users playing various Roles)
  • IA Space (does the user appear to be trying to build the site from the outside)

FYI, I've recently posted a video summary of a project that explored the IA Space concept.

Michael Keara
User Interface Systems Architect,
The User Advocate Group

Awesome

GarenCheckley's picture

We're excited to see your analysis! It's great to hear that others are slicing and dicing the research in different ways :)

Slides and Mockup posted

technicka's picture

We just posted the slides from our presentation in this post as well as the original FireWorks file of the up-to-speed aid to download and play around with.

Looking forward to discussion around this!

GreaT - Reiseversicherung

arton46556's picture

D7 its great

I REALLY like the idea of

nomonstersinme's picture

I REALLY like the idea of this up to speed info for new users, but I think that what information is given here needs a lot of thought. I can see users still being overwhelmed and confused even with the way this is comped now. Especially the recommended modules and description.

I would love to see several more targeted 'up to speed' walk throughs for users based on WHAT they want to do with drupal. Then we could provide an explanation of nodes and fields in a way that makes sense for their focus as well as provide suggestions for modules that make sense. (Eg. WYSIWYG is probably not necessary for someone building a photo based site like a portfolio)

This usability study is very interesting but not surprising at all. I have been using Drupal for several years and I still have trouble finding things sometimes in the admin. This is a huge issue to tackle and I definitely don't know the right solution but I think this is a good start!

We spent a lot of time doing similar tutorials...

Noyz's picture

..for Drupal Gardens. In the end, we never released anything because everyone that looked at the designs had their own ideas as to what should be shown. Net net, the best approach here is to start smallish, cover core concepts, release it for test, and iterate and build up on it based on what users actually want.

Excellent point and you are

nomonstersinme's picture

Excellent point and you are definitely right.

I guess I just don't believe we should be throwing WYSIWYG at people.. maybe thats just because of my design background ;)

Kudos

kenlyle's picture

Kudos to the Drupal community for tackling this issue.

I'd like to point out that every year at CMSExpo, we hear that the CMS are not really in competition with each other, but with "not-CMS" options. Joomla has some of the same issues. It's inherent in the level of indirection between "what's a module?", "what's a module position?", and "on which pages does the module display, and to whom?" where database rules determine which database records make up a page, opposed to the more WYSIWYG approach of Office apps, Frontpage, Dreamweaver, etc.

I think would behoove the D and J communities to team up to produce some basic instructional videos about:
a) The benefits of using a CMS system
b) Terminology
c) How to realize these benefits

modules, blocks, module positions, published, draft, unpublished, trashed, categories, node/article, title, keywords, tags...there are lots of things that CMS systems have in common. I think that it should be possible, and actually helpful, to produce media which explains these basics and perhaps discusses some of the basic differences between J and D.

Crazy? Maybe.

The PDF Presentation

rockrhino27's picture

I really liked the PDF presentation. In my opinion Drupal is not that hard to administer once you get your mind around thinking about content first. I notice that many first time site builders are worried mostly about layout, when in order to be successful with Drupal you need to always be thinking about content.

I agree, but this is a real

escoles's picture

I agree, but this is a real challenge, and in my experience the site-builder is not the person you need to win over.

I've been struggling to get designers and creative directors to think of content first for...um...5 years, now. Hasn't worked so far, not really -- the writers and writing-centric creatives can get their minds around it a little, but I've found that creatives are often really hostile to the idea.

Hello All!

gaurav_m's picture

Just an opinion,
The brains who build the system, while solving the situations, come with solutions. The solution works for them and they push that out onto Live system. The big picture of solution is okay for them. But for the naive users, who have not gone to those altitude of problem solving..as they are consumers only.

Now, with those technical terms... the naive users feel distracted and bogged down. So, being a technical thinker, the builders should feel tempted to dilute there technical terms for naive users.

sys. builder thinking speed is say 50 km/h or some other reasonable unit..
for end user, let say it is 20km/h..

Both need to be somewhere at average. To catch the flow!

What is your opinion community?
(sorry for my grammar :P)

Silence is golden.

There are a few things that

victorv's picture

There are a few things that are missing to make it a true dream for new users.

  1. Standard WYSIWYG with image upload capability. CK Editor is good but but default in filtered HTML mode gives options that get filtered out. And it lacks an image upload capability that is decent.

  2. Canvas editing. Concrete 5 and Umbraco have this and it basically lets a logged in user to click a text field and edit the content live (you browse the site and click what you want to change). You just click the content text field and a WYSIWYG bar appears above the text and you really get what you see since it uses the default classes from the template.

  3. Simpler imagefield calls. There are hundreds of posts on how to render a thumbnail and the original image at the same time. In other CMSes its easier.

Apart from the things above I really like the system. When I meet a customer today I know that with Drupal I will always be able to deliver exactly what I promise and I know that I dont have any real limitations. Also bugs in Drupal along with the modules are way fewer than in most other systems (like Umbraco). Also performance is impressive.

Google Sholud learn from Drupal

emilcohen's picture

Google Sholud learn from Drupal how to build website manage online community first than Usability Report : )

I think the 4 layers they

Ryan258's picture

I think the 4 layers they looked at are very good places to focus out attention as we aim to iron out the Drupal learning curve. Not too long ago I spent a weekend teaching Drupal to a designer friend of mine and looking back on it, the four layers of conceptual, flow, terminology, and interface were really key.

Here are the phases of how it went.

  1. We had a concept something that we had to make. So we had that thing, that concept of a defined end point which brings with it the motivation to learn.

  2. With the concept of what had to be made I basically mind mapped out a road map of how it would be achieved. That's where the flow, accompanied by terminology was covered.

Like "We have these kinds of content types that are going to need different kinds of fields." We were blitzing through an ecomm rebuild so going into product types, products, and product displays so going through the various flows demonstrated patterns that illustrated the concepts of Drupal.

*While this was all fine enough for him to have that temporary knowledge of how things worked, I would not have bet on it sticking... but we still had a site to rebuild and diving right into the interface and into construction was where all that info I shoved in his head was going to have a chance of being retained.

3.) I shared my screen throughout the site building process where we built out the content types, created the content, built views, themed with omega (fortunately he had design/front end skills), and then tackled Commerce.

Tying it all together with the interface wouldn't have been worth anything if there wasn't a purpose. I think when people have a purpose that's when they're truly willing to get their hands dirty and absorb all they can to fulfill it.

4.) Sharing a learning experience...

Commerce was something I had some knowledge of, but my last ecomm site was from the Ubercart era and it was simple. But me showing him how I went about finding solutions, checking into all the resources for answers showed him methods he needed to go in whatever direction he needed.

So in the end I know absolutely possible to walk someone straight through the dreaded Drupal learning curve. In this report I didn't think there was anything I felt was out of order, but Drupal has that natural problem where things that are more comprehensive in options tend to me more difficult to learn because all those options are available.

One thing that slowed my growth during the Drupal 6 era was the constant paranoia of "What don't I know" because with so many options it was like, "If I don't know all of them then I can't sure" which only made any subsequent step that much more intolerably uncertain.

Honestly, recipes were my saving grace, then picking apart distributions. Also tools like, module_filter and admin_module greatly helped pick up the learning pace (less time searching, more time failing forward).

Looking back on it, I'm literally green with envy how what took me a year and a half to get my head around he pretty got covered in a crash course weekend.

Perhaps we need to look back on our own learning experiences and identify the points that held us back because only a piece of it has to do with the interface. To people who wonder "How can anybody not get Content Types!?" think of it as "How can anybody not get their heads around Entities!?" I'm Drupal 25/8 but until I had to really get Commerce, I didn't really understand how to actually use Entities.

But yeah, be a friend, dedicate a weekend to one, and save them a year and half of their life and possibly a lifetime with WordPress and Joomla.

Drupal Developer | Saint Paul | Buenos Aires

Why, o why I can't understand this right away....

dmiric's picture

I'm using Drupal since Drupal 5. And I can remeber how much more complicated it was to start in those days. Now with Drupal 7 everything is pretty easy to pickup.

What would go a long way to getting closer to new users is making something like Quick Start Guide (check http://www.applicationcraft.com/tut-qsg). I spent like 8 hours watching those videos and in the end I felt like I knew enough to start working with Application Craft.

I know there is a lot of videos on Drupal everywhere but having those on official site is what counts.

As far as things go from the google document I would expect someone that works at google to know that he could "GOOGLE IT"

Was there a test scenario?

alex.designworks's picture

I'm not sure that your users had a whole "picture" before test.

Was there a testing scenario?

Because even experienced developer will stumble with a system if he/she does not know what should be built at the end.

If the test scenario would contain exact requirements for the system a user is building, the user would firstly understand WHAT is being built, and only than would try to find they way HOW to achieve this with a specific CMS such as Drupal.

IMHO, such tests do not provide much relevant information about Drupal usability.
Moreover, there is no background information about user level is provided (user from this video http://www.youtube.com/watch?v=pgyRUHRdktY&list=PL1B7D6E805CE3A5A0 does not even understand basic concepts of system - took him 15 mins to understand that Admin bar is 'admin bar'!). This has nothing to do with Drupal's usability - looks like the user is not familiar with web-sites at all.

Drupal's terminology is another issue here (example with 'teaser' perfectly demonstrates it). Most of the terminology can be fixed without much coding as it is only text-based information. Also, contrib modules can be more descriptive about own functionality.

As for the report. The proposal at the end about intro text is great, but according to the videos of testing, some of the users would not read anything written in front of them.
User 1, for example, went to the help page (which totally makes sense) and spend literally 10 seconds there without even reading first 3 items. Maybe it is related to that particular user's way of thinking or maybe (and most likely) it is the way the test was conducted (lack of test scenario) so the user did not even know what to look for.

To sum up, although I agree that UX test are necessary, it is crucial to make sure that test scenarios explain the main goal of everything that is expected to be done by user. It is impossible to make such complex system as Drupal to behave like limited iDevice with 1 button that performs only one function.

I'm not sure how long you

escoles's picture

I'm not sure how long you stay involved with projects after you launch them -- maybe you launch the site and never see it again, maybe you launch it and you or some other person familiar with Drupal is responsible for maintaining it going forward.

My own experience, mostly in B to B marketing, is that businesses buy websites, and they then have to maintain that site. Those businesses are almost never willing to invest in any system that requires them to learn how to use the system in any kind of substantial way. They see it as a waste of time.

So a system that requires them to have the kind of basic familiarity you're assuming they'll have, by definition has a severe usability problem.

The only question really is where the problem resides. Here, it resides in attitude, basically: prior to D7, Drupal's architects have traditionally made the assumption that developers will build their own UX. There's been some attention to UX, but not much, and as a consequence every time someone implements a drupal system it's managed a little differently. For most users, familiarity with one system doesn't translate to a lot of time saved in getting familiar with anything more than basic or anything less than advanced capability.

I'm not sure how your reply

alex.designworks's picture

I'm not sure how your reply is related to my comment above.
I stated that tested users in this particular test did not have proper understanding WHAT they have to do, not even HOW. Therefore this particular tests can not be considered accurate.

I do agree with everything you stated, I just do not agree with the results, because of the way this particular tests were taken.

Installed a module, which I

greta_drupal's picture

Installed a module, which I have never used before, specifically to handle some UI on a current site build. Realized that it offers a HUGE advantage to simply understanding Drupal. Have never heard anyone mention it before for this use.

Tipsy! Perhaps the other tooltip modules equally useful.

But, Tipsy has a setting to automaticallly show form descriptions. I'm finding it very useful as a navigate the use of another new-to-me module, quite complex (Display Suite). Really speeding up the learning curve with that.

Now, I'd be interested to see a Drupal new-user UI/UX study with that module employed.

It's relevant because you

escoles's picture

It's relevant because you have an odd conception of the "what" part. You seem to think the "what" includes details about how the system works. I'm telling you that for most of the users I work with, that's just not the case.

People don't have software they need to use, or even functions they need to execute -- they have tasks they need to perform. Right now, and without pretty extensive customization, Drupal requires significant idiosyncratic education -- it's not very discoverable, if all you know is the task. That's what's being tested here: How discoverable is the system, given only the task. It's a perfectly legitimate mode for user testing, it's been done that way since user testing began. There's nothing illegitimate about these tests.

really useful report! thanks

rommelxcastro's picture

really useful report! thanks for sharing

STRAIGHT EDGE // WEB DEVELOPER

really useful report! thanks

rommelxcastro's picture

really useful report! thanks for sharing

STRAIGHT EDGE // WEB DEVELOPER

You can't save the stupid

dynamicdan's picture

Users are always dumb... that's the POINT. They are not devs. Drupal is a ___? (CMS and Framework). Wordpress feels more like a CMS/blog system. Drupal feels like a web app. The level of technical know how and understanding is greatly increased for Drupal.

I once thought a WYSIWYG system was a brilliant idea. Then when my clients used it I realised it was a curse. Some gave up and emailed me the content changes, others required a lot of hand holding.

The best thing I've seen in Drupal for user support is the community and documentation. I suggest the Skype/GarageBand style intro approach for new users. Users/admins can skip the intro or read through it. It can be a video or a series of visual queues for where to get things done. Perhaps even some "I need a blog" and "I want a catalogue system" links would suffice for getting started and finding ones way around. There are some good points in the study but they are not deal breakers for using Drupal IMHO.

Drupal is a system learnt over time, just like it's modules. Developers are NOT users and never will be. They are simply not equipped with the usability brain. Drupal has a heavily dev/tech oriented community.

By the way, aren't install profiles meant to handle these usability issues and provide a more customised, user friendly experience for the user?


Web Dev, Consulting and Design
DynamicDan.com

So we throw up our hands...

tkoleary's picture

and say "Drupal is for developers" and stop worrying about the glaring usability issues? I think that would spell the death of Drupal.
Usability drives adoption and there is nothing more important if we want to survive.

Drupal has a heavily dev/tech

greta_drupal's picture

Drupal has a heavily dev/tech oriented community.

As a UI/Designer/Sitebuilder Drupaler, I haven't found the Drupal community very welcoming. Posted questions in issue queues, IRC, and gdo related to functionality expectations or suggestions, almost always met with "create a custom module for that". That is definitely not the correct answer, IMO.

On more than one ocassion, I've installed module updates that didn't work -- even broke a site, where previous version did work. Once having posted to issue queue, developer responded 'I don't know about that, I always use Drush".

As a sitebuilder, I use the Drupal UI for everything -- even if something like Drush might save a few seconds. Afterall this is how my clients will use it. Best to be well familiar with it, and tweak it nicely.

IMO, the Drupal community is making a mistake by not (better?) integrating designers, UI/UX folks, and sitebuilders in the process.

As a sitebuilder, I use the

escoles's picture

As a sitebuilder, I use the Drupal UI for everything -- even if something like Drush might save a few seconds. Afterall this is how my clients will use it. Best to be well familiar with it, and tweak it nicely.

Generally a good idea, I think -- I try to do something similar in most of my work. This is why I was banging my head over D7: the outcome of a long and engaged UX process was to make teh system LESS usable. I was having flashbacks to WordPerfect 6, or any time I've been forced to rely on Outlook for email.

Drush is still good for stuff your clients won't do, or for areas where you want to save them money or be more precise or reduce error potential (e.g., updates).

I was banging my head over

greta_drupal's picture

I was banging my head over D7: the outcome of a long and engaged UX process was to make teh system LESS usable.

Yeah, the fact that I must horizontally scroll an overlay or that a close button is rarely ever present at all for D7 makes me wanna just work as a stripper.

Be, no worries, D8 is just up the road. Just follow the ladder to it. Who needs a stable D7. Clients can just rebuild their sites next year.

Moreover,

ry5n's picture

although driving adoption is a worthy outcome, good usability fundamentally means it’s easy for people to use the system for its intended purpose. If it’s great, people will even enjoy using it. The attitude that "users are always dumb" is just plain toxic. If eight tech-savvy Google employees were – at times – scratching their heads at Drupal’s admin UI, we have work to do.

Users are not dumb

dcmistry's picture

I agree with the sentiments shared here that the users are not dumb. There are varied kind of people who use Drupal for varied reasons from blogs to massive sites. They could be site builders, site maintainers, content editors or themers. A lot of them could be "one man army" but there are a lot of people who specialize in one thing. Working on Drupal is not necessarily their 24 x 7 job, it is a tool to improve their efficiency. With different skill sets, their needs are different.

We are moving at a stage in Drupal where we have to consider all angles (just the way, developers want to explore and prioritize the different use case for a particular problem)

@dynamicdan: I agree to your suggestion about getting the user oriented with Drupal. We are working on this. You can find more information here and perhaps provide some feedback: New user onramp proposal

Dharmesh Mistry
UX Researcher | Acquia,Inc.

Perhaps users are just novice

dynamicdan's picture

To better explain my stance on usability for common users, it's probably better to say that they just learn slower and often come into the web/CMS world as novices. One can even ask any existing CMS user to prepare an image for uploading and they'll probably do it the 'wrong' way. About 70% of my work as a web dev and consultant is education. The challenge here is how to educate non-technical users. Technical users have more patience and are happy to 'just google it'.

Admittedly, it does make sense to make drupal more user friendly and decrease the learning curve as much as possible but I feel that the target audience needs to be considered carefully. I don't give admin user access out because I know it will alienate the user. I deliberately cut down the interface and options as much as possible. Over time I try to offload my work as an admin to more experienced users and introduce them to new functions/features gradually.

I'm a big fan of the concept "hide the advanced options until the user looks for them". An example of this is Windows Vs MacOS design for system settings. Many users complain that MacOS is a dumbed-down system with few options for customisation. The actual fact however is that the advanced options buttons are not so obvious. The true mega-advanced options require shell knowledge. If one is competent to use a shell then one is competent to fix the gigantic mess one can make with complex customisations.

Perhaps interface limitation controls make more sense than functional/permission based controls? Let users do everything but make some things more obvious and easier to find.

In short, I would like to see a system where more options and features are brought to the users attention over time so as not to overwhelm them in the beginning.


Web Dev, Consulting and Design
DynamicDan.com

Agreed. I create well-defined

greta_drupal's picture

Agreed. I create well-defined user roles and layer them, when the time is right. Don't give a client admin rights for a very long time -- until they are very comfortable with Drupal.

In fact, after I have done the basic install and client-user UI, I will simply create a 'content producer' account (add, edit nodes) for client and turn them loose without instructions other than "tell me if something doesn't make sense". I want the system (for content creation & management) to be obvious to use. If it is not, that tells me that I have a bit more UI work to do. Rather than train the client how to use the system (for content creation), I let them tell me.

Usability

Group organizers

Group categories

UX topics

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: