Accessibility Testing for Drupal 8

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Jeff Burnz's picture

In the BoF the other night the issue of testing accessibility was raised, I thought this was very interesting and we certainly need to move forward form the ad-hoc methods we used in Drupal 7 development and I think formalize our testing procedures and methods.

Examples include standardizing on software used in testing, for example a defined list of AT such as screen-readers and versions.

We certainly need to raise our game with regards to user testing also, perhaps looking to arrange defined rounds of end user testing at various points during development.

Finally we I think we can partner or collaborate with third parties on testing - such as large organizations outside of Drupal with a vested interest in Drupal 8's accessibility standards.

Comments

Unit tests

Jeff Burnz's picture

I should add there is one other possibility and thats writing unit tests for accessibility - I don't know what this would look like or what we could use unit tests for, but its worth looking into.

"Unit tests"

Cliff's picture

At the risk of exposing my ignorance, but could you explain briefly what a "unit test" is?

...

Jeff Burnz's picture

An automated test that tests code programatically - D7 was developed in a "test driven environment", so tests are written to test a specific feature, function or method etc. What I am asking here is if this might be possible for accessibility? In other words we write code that tests the accessibility of something. Maybe it's doable, maybe its not, but its worth looking at and thinking about.

Got it!

Cliff's picture

Thanks!

There are already some online

mungai's picture

There are already some online accessibility test sites of various quality. For example:
http://wave.webaim.org
http://www.cynthiasays.com

These are by no means perfect. For example, computers are dumb, so they can only look to see if there is an alt tag, not check to make sure there is "good" data in the alt tag.

Is this the sort of thing you had in mind? Would there need to be something specific for drupal?

Not really what I am talking

Jeff Burnz's picture

Not really what I am talking about exactly, I mean automated testing within the Drupal test framework.

Let me give a concrete example - lets say we could parse CSS files (probably going to happen in D8), and we have a theme, lets say Seven, then we know the values of things like colors and background colors. We can hard code those values into a test, so if someone tries to change them they have to change the test, initially the patch will fail because the test detects a failed match. When they go to change the test they are going to get a big doc block telling them why this test exists (contrast ratios) - so this raises awareness to all developers + saves us having to explain contrast ratio's (again...) and prevents the change happening and us finding out about it later. I'm not saying we should do this, but it could well be possible, and could be very useful - we just need to think it though and see where this sort of thing could be applicable in Drupal core.

I am not really looking for

mfratkin's picture

I am not really looking for automated testing tools, as we have already evaluated them and find them to be of only limited use. As you said the alt tag is a good example of a toll not being able to determine whether the description is appropriate or not. It can only tell whether it is present or not.

We have a very well designed testing methodoly that we emply testing 200-300 applications per year for accessibilty in the federal government. We use testing without assistive technology and with screen readers, magnifiers and voice input. So, we are trying to determine if a Drupal site is developed with all of the accessibility components utilized, will it pass our tests for Section 508 compliace. We can only do this if there are sites that are claimed to be comopliant and then we can use our testing methodolgy.

Alternately, we can try to create our own site but this would probably require a significant learning curve and I would think that there are people with more experience that have already done this. Thanks.

Mike

...

Jeff Burnz's picture

Your post seems oddly out of place in this discussion - this is about developing testing for Drupal 8, I think you are looking for existing Drupal 7 sites you can test against? Pretty sure Mike Gifford pointed you to at least two sites you can evaluate for testing in the other thread you posted - http://groups.drupal.org/node/132764

Would some module like the

Shyamala's picture

Would some module like the Coder module work? A module that evaluates the accessibility of any modules by parsing the module.

End User Testing

dcmistry's picture

Jeff - To your point, end user testing could be a good thing to do. At least it will uncover the bigger issues and may be we will be able to detect a pattern and fix it. We could do a variety of things (perhaps a usability study) or interviews. I would be happy to consider one of those studies for people with disabilities and understanding what their concerns are. For this to happen the obvious questions would be :

  1. Do we have such data from the past?
  2. How helpful would it be to have this data?
  3. How the community feels about this?

Dharmesh Mistry
UX Researcher | Acquia,Inc.

re: End User Testing

bowersox's picture
  1. We only have very limited findings from the past. I believe Katherine Lynch (from Drexel) organized a prior round of accessibility testing on Drupal 6, and she has videos which I have watched clips of. But I don't believe there is an extensive write-up with prioritized, actionable findings.

  2. It would be very very helpful to have this data.

  3. Everyone I've talked to in the community, both the accessibility group and the rest of the Drupal community, would prefer that we have formal testing results as the basis for our accessibility work. It will help us know what the highest-priority problems are. It will give us a more objective justification for our work. And it will give us a personal window into the individual users who are impacted by the current accessibility limitations of Drupal.