Drupal 8 multilingual user testing in Zagreb

The study was conducted after Gábor Hojtsy made an open call for the multilingual user testing results. It was originally planned to run as many tests as possible on the second day of Drupal Balkan Summit in Zagreb, but the snow prevented me from going to day one, and a personal emergency had me out of the second day within an hour of getting to the summit. This started a trend, as every other session was plagued by something or other. The results are compiled from my notes and backup audio recording I made over my phone.

Timing and the focus of the study

The study was spread out during the 3rd and 4th weeks of December 2012. The three key questions were:

• Can users add a foreign language to an existing site?
• Can users configure translatability / language options for entities?
• Can users do a basic node translation?

Participants

There were ⅘ participants in total. All of them worked in IT and have at least basic familiarity with web based applications. All of them had at least working knowledge of a web based CMS (in a range of products) or similar. One used both Drupal 6 and 7 extensively, and helped administer a small Drupal 7 Commerce site.

Task 1: Adding Croatian to an English Site

In this task, what we have been looking for is that people realize that language setup is provided by a module that needs to be enabled, understand how to add a language and explain the three special languages in core that were added in Drupal 8 (not applicable, not specified and multiple).

Every participant behaved like the participant’s in Gábor’s study: all of them looked in the Regional and Languages area of the Configuration page first. Following that, they would look into Content, Structure with Extend being the distant third (and in once case fourth when the user opened up Appearance before).

• The “wall plugin” comment first noted in Gábor’s study was also heard a couple of times.
• I have a feeling that everyone expected that the CMS would be able to handle multiple languages and that “you do not need to extend it to do that”.
• Three participants looked and enabled the Languages straight away. Of the three, two noticed Entity
• Translation but did not associate it with the task at hand and left it disabled, because they did not understand what it would do. The third enabled it “just in case”.
• Again, just like in the Hungarian study, none of the participants noticed or enabled the language switcher block.

After the language was enabled all of the participants concluded that now they need to go to Configuration page.

When asked to comment on the process, most of the participants expressed confusion as to why they had to enable the entity translation module, as they expected that adding a language to the site implied that the content will be translatable as well.

The second comment most had was that it would be more logical to enable the language module, and get taken to the configuration page right away.

Task 2: Making Things Translatable

The cornerstone of this task was the patch from http://drupal.org/node/1810386 which we applied to the test environment.

From this point on everything went fine for my participants. Clicking around for task one got them all comfortable enough with the logic of the GUI, and all of them went straight to the configuration page and found the Content translation settings link with ease and immediately figured out what it was.

The settings page was easy to navigate and to use for all of them. They did ignore the configuration options, and when asked could accurately figure out what they do (the default language option and the hide language selector option) as all of them actually read the descriptions next to both the options.

The only issue, as mentioned in task 1, was the need for the few participants to go back and enable Entity Translation module. All of my participants commented that it looked like an extra unnecessary step in the process.

Task 3: Translating Content

Just when I thought that the previous task was uniform and easy, we got to the third task. All of the participants immediately went to the content section and could easily translate the sample article.

All of the realised that the title was not translatable, and could not understand why. The warning label was noticed, so no one went to the translation configuration page to enable it, but it was not a good experience for any of them.

While doing the test most of the participants wanted to see the full experience and created their own content and translated it. All of them got the interface easily, but could not understand why you needed to save the content and then add a translation. Again, all who tried did figure out that that is how it works without any guidance, but did not like it.