DrupalCon Portland took place from May 21st, 2013 to May 24th, 2013. Optional sprinting days bookended the conference on the weekends before and after.
I held an accessibility core conversation about accessibility on Tuesday, May 21st, 2013. In this conversation, I established the need for us, as a community, to develop a testing plan to establish the accessibility support of Drupal 8. On Wednesday, May 22nd, 2013, Wim Leers and I talked about using Backbone.js and several new Drupal 8 features to support aural UI development in Drupal 8. On Thursday May 23rd, 2013, I lead a BoF in order to establish concrete next steps in the effort to establish support and evaluation of accessibility in Drupal.
This post summarizes the discussion of that BoF. Please correct any misrepresented statements I might make and add detail where my recollection might be insufficient or inaccurate.
Drupal 8 Accessibility Support Goals
During the BoF, we established the following four goals.
- Drupal 8 will have zero WCAG 2.0 Single A level conformance violations across all pages. (guideline reference)
- Drupal 8 will strive to have zero WCAG 2.0 Double A conformance violations across all pages. (guideline reference)
- We will establish automated test coverage using the TestSwarm module and QUAIL module through integration of the two. (More details about QUAIL).
- We will establish behavioral test coverage with the support of participating organizations. The test plans will be developed from existing training materials.
Drupal 8 Accessibility Tasks and Considerations
The following points were also made and the group agreed that they are important considerations to keep in mind.
- If we can raise funds, we would like to arrange a third-party audit of Drupal 8 Core. The funds would be managed by the Drupal Association. The audit would be most effective after we had addressed the most obvious and accessibility issues through automated testing.
- Some usability issues will be more critical than others. We reluctantly accept that issues will be prioritized and addressed in priority order. Some issues may not be addressed before Drupal 8 releases.
- Priority of issues will be determined through a combination of applying the WCAG guideline and feedback from users whom these accessibility issues directly effect.
- Review of existing patches and issues is just as important as identifying new issues. Reviewing and testing are just as important as coding.
- Reviewers will be trained to use tools like simplytest.me. jessebeach will set up a training session.
- Developers will be trained to use tools like VoiceOver, JAWS and the Wave Toolbar. How this will happen has not been established.
- Caroline from UC Berkeley will explore the possibility of providing tranining material examples that we can use to develop accessibility behavioral testing materials.
- Automated testing using TestSwarm and QUAIL should eventually be incorporated into the standard testbot tests run on all patches proposed against the Drupal project.
- We will not directly address accessibility of contributed modules and themes in this current effort. Our goals are to establish the accessibility support of Drupal 8 Core and improve it. Once we have accomplished this, we can move our attention to tools to help contrib.
Training
I will be holding a Google Hangout to explain how to use SimplyeTest.me to test Drupal issue patches at 1pm EST on Wednesday, May 29th, 2013. The session will be recorded and I will share the URL here once I have it.
Hangout URL: https://plus.google.com/u/0/events/cpm6cd8gdboaglo4g780k8o2obk
Work that is already underway
Kevin Miller (kevee) has been working on tools to support accessibility in Drupal for many years. During the DrupalCon Portland sprints, he established basic automated tests unsing the QUAIL API and the TestSwarm module. I would love for him to provide details here in this discussion about his work and what he thinks the next steps should be.
Dharmesh Mistry (dcmistry) has proposed we conduct behavioral accessibility testing during Twin Cities Camp in July. He is in contact with the University of Minnesota to set up these tests.
Comments
Great work!
Jesse, thanks for this summary of all that went on at DrupalCon Portland. I do have one suggestion about testing. No, I have two:
Again, thanks for the update—and thanks to everyone who is or ever was participating in the work needed to make Drupal highly accessible and keep it that way.
Manual Testing of Functional Accessibility
I'd be willing to lead the effort to conduct an evaluation of Drupal 8's functional accessibility, and I'm confident I can recruit at least a few higher education accessible IT folks to participate in this effort (for free!). All we need to get started is a set of clear steps that comprise the workflows to be tested. As noted in the meeting summary we can perhaps use Caroline's training materials as a basis for these workflows. Once we have documented workflows, we could step through each workflow using a variety of modalities, including:
The output would be a report that documents the barriers encountered with each modality. We've done similar work for Google Apps, plus various learning management systems. The motivation is that we're charged in our positions with ensuring our campus IT is accessible, and since more of our campuses are using Drupal we're willing and able to justify contributing toward its accessibility, especially if the Drupal community is committed to fixing the problems we identify.
NC State will help
I'm in, as long as I don't have to coordinate it!
Status on automated accessibility testing
Thanks, Jesse. I have setup a sandbox project for integrating QUAIL with TestSwarm, attiks and I are working through the issues right now:
https://drupal.org/sandbox/kevee/2004936
I have also started a sandbox for a general accessibility module, which will be a broader framework for content, theme, and functional testing. This is a D7 replacement for the accessible content module, etc, and will be worked on once the D8 automated testing part is closer to being ready:
https://drupal.org/sandbox/kevee/2000774
Awesome kevee!
Awesome kevee!
Update
Thank you Jesse for the comprehensive run down. I am excited to see you all respond.
One of the accessibility folks at U of MN has agreed to help us with the accessibility sprint at Twin Cities. We are going to have a preliminary call with him on Friday, May 31 1 to 2 pm CST to discuss the scope of this.
If you would like to join the call, please email me at dharmesh.mistry@acquia.com and I would be happy to invite you.
Dharmesh Mistry
UX Researcher | Acquia,Inc.
Meeting notes and next steps
Meeting Notes:
Date: May 31, 2013
Attendees: Tonu Mikk, Chad Fennel, xjm, tkoleary, dcmistry and tkoleary
Goal:
We want to focus on making the core accessible, admin UI should be accessible
To conduct accessibility evaluation on Drupal core with Tonu Mikk and Phil (from the accessibility department at the University of Minnesota).
Tonu’s usual testing process includes:
Accessibility folks:
Raising awareness:
Next Steps:
Dharmesh Mistry
UX Researcher | Acquia,Inc.
Synchronizing Our Testing Proposals
Please see my earlier comment "Manual Testing of Functional Accessibility" (https://groups.drupal.org/node/300238#comment-929668). Note also that Greg Kraus of NC State offered to help with that. I haven't asked yet, but I'm confident I can recruit others as well.
Do you see the methodology I proposed in that comment having a place at, or leading up to, the Minnesota sprint?
When we've done crowdsourced accessibility testing previously (most recently with Google Apps) we did it remotely and asynchronously over a period spanning several weeks. That allows more people to participate, since many who would be interested can't travel to Minneapolis and aren't all available on any single date.
Perhaps if we do testing between now and July 17, your sprint time can be focused on fixing the problems we identify.
Thoughts?
Update
We talked about conducting off-line accessibility evaluations in early July before the camp and conduct the two accessibility evaluation onsite at Twin Cities camp. You mentioned that you had contacts in higher ed accessibility departments who would be willing and interested in participating and giving feedback. If you could make primary introductions with me, that would be really awesome. My email is dharmesh[dot]mistry@acquia.com
Dharmesh Mistry
UX Researcher | Acquia,Inc.
Tasks/Scenarios
I suggest using similar tasks/scenarios as the previous Drupal Usability testing events have. As much as possible it will be good to test the same tasks. Over time this will make the accessibility testing just an extension of the broader usability testing.
Here is the Minnesota testing scenario written for D7:
http://drupal.org/files/Appendix%20C%20-%20Scenarios.odt
(Background info at https://drupal.org/node/1175694)
I'm so glad to hear this accessibility testing on D8 is in the works! Thanks for all you're doing to keep Drupal the most accessible, open source CMS in the world!
www.pixotech.com
Good points!
Thanks bowersox for the comment. I agree on having the accessibility evaluation be task and scenario based. Thinking back, I see some gaps in my own script (the one that you posted). I would like to make changes to the script for maximum value. The participant volunteers for UMN accessibility evaluation 2013 are content creators/ administrators and I want to tweak the script to their use case. By that what I mean focus on the primary tasks of Drupal for content creators/ administrators:
- Create content
- Find content and unpublish the content
- Moderate comments
- Edit content
I would also like to get more information through contextual inquiry about how is that they currently do such things? what is the process? what is working and not working for them?
Do we have this information? If so, can you point me in the right direction. The last thing we want to do is duplicate efforts :)
My personal goal is to continue these efforts through multiple evaluations and partnering with others to build a solid pool of information that is actionable.
Thoughts?
Dharmesh Mistry
UX Researcher | Acquia,Inc.
Re: Good points!
Hi Dharmesh,
I would suggest adding two additional tasks into your list:
- Add a page into the site's menu
- Swap the order of two items in the site's menu
Traditionally, these menu operations have been very difficult for users with disabilities in Drupal. Because the menu re-ordering is based on drag-n-drop, that was totally inaccessible in Drupal 6. We used to recommend to blind users that they turn off Javascript in their browser (!) so they would get the numerical weight fields. In Drupal 7 we added the "Show row weights" link in tabledrag. It does help some people, but it also creates UX confusion.
The 4 primary tasks that you have listed are generally accessible in Drupal 7. During the "create content" task I would expect keyboard-only users and screenreader users to have confusion/frustration around the vertical tabs. "Find content" is also going to be difficult for blind users because the UI in Drupal 7 puts the ability to search for content in a different place from the ability to filter for it (Administer > Content).
That is just my gut instinct. None of this has ever been systemically tested in a public way for the whole Drupal community. Many of us have experienced our own quirks with our own sub-sets of users.
P.S. That's awesome that you were part of the previous usability testing! You're a rock star!
www.pixotech.com
Great point!
Hello terrill,
Synchronizing efforts completely makes sense. I am having a meeting with the folks tomorrow and I will propose this idea and I am certain that will be as exciting as I am. The only issue that I can see is time commitment on our part. Just thinking aloud if this does pan out, perhaps we do these asynchronous evaluations in early July and that way we have a solid list of issues for the sprint.
I will give an update tomorrow.
Dharmesh Mistry
UX Researcher | Acquia,Inc.
Just wanted to mention that
Just wanted to mention that behavioral testing, like we're proposing, is done in addition to testing of pages against accessibility standards. We need to do both ultimately. Testing against standards can be done with machines for the bulk of issues. I'd rather we engage humans for complex tasks, just as you're all proposing here.
Meeting notes
Meeting notes
- Date: June 5, 2013
- Attendees: Dcmistry, Jessebeach, Chad Fennel, Tonu, xjm, lisarex
- Conduct accessibility evaluation around “Content” (Create/ Manage/ Edit) on July 17 and 18. Findings to be analyzed and issues to be put in the queue and to be worked at the sprint on July 21. Accessibility evalution to focus on making the experience easier rather than just WCAG compliance
- Dcmistry to write the first draft of the script which will have tasks/scenarios and contextual interviews about their process
- Involve Terrill and conduct a few accessibility evaluations off-site before the accessibility evaluation at U of MN
- U of MN to provide access to usability lab to conduct and record evaluations
- Need to understand how are we going to combine usability and accessibility together without adding cognitive load to the participants. One idea is to focus on the reflection phase after a task. A moderator from our team will guide the participants on how-to aspect of task completion
- Involve Catherine from Phase 2 in the sprint if she is available
- We are going to provide the participants with URLs, usernames and password
- Dharmesh to reach out to Jen Lampton and ask if she would be available and be involved in the sprint
- Jesse to reach out to Mgifford and ask if any front-end devs would be at Twin Cities camp and will be interested in participating at the sprint
Dharmesh Mistry
UX Researcher | Acquia,Inc.
Accessibility Evaluation Script Draft
Hello all,
We are getting excited with the preparations for the accessibility evaluation. Here is the first draft of the script: https://docs.google.com/a/acquia.com/document/d/1sxe16mB2CRKz2fraJLVWHp7...
Please provide feedback via comments in the document by end of day Sunday (June 30).
Let me know if you have any questions.
Dharmesh Mistry
UX Researcher | Acquia,Inc.
URL?
Dharmesh, can you re-post the URL for that? Google says that document does not exist.
www.pixotech.com
Ops
Try this: https://docs.google.com/a/acquia.com/document/d/1sxe16mB2CRKz2fraJLVWHp7...
Dharmesh Mistry
UX Researcher | Acquia,Inc.