Comparative Multilingual Content Edit Experience Usability Study Findings

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
dcmistry's picture

Comparative Multilingual Content Edit Experience Study

Focus of the study: To get answers to the following high-level questions:

  • How do users current translate content on Drupal?
  • How do users translate content on current Drupal 8 and current Spark translation feature?
  • Which version do the users prefer? Why?

Number of participants: 3 (Acquia Internal), 3 (Partners/ Customers)
Participant profile:

  • Participants who create/ translate multilingual content or have a need to do so in the near future
  • Participants have not used the current Drupal 8 and Spark translation feature

Study Date: June 25-26, 2013

Recruitment: Participants were recruited through internal e-mail and Acquia Sales contacts

Compensation: No compensation was provided to the users

Methodology:

  • Participants have not used the current Drupal 8 and Spark translation feature
  • Moderated usability session for 30-45 minutes each
  • Participant performed same set of tasks on each of the designs
  • Half of the participants were shown the Drupal 8 UI and then Spark UI. The order was reversed for the other half of the participants. This was done to eliminate the bias from the order

Overall Summary

5 of 6 participants preferred Spark design over Drupal 8 multilingual design and 1 participant did not have a preference.
How would you split 100 points giving more points to the design you like? (Average of 6 participants)

  • Spark - 75 points
  • Drupal 8 - 25 points

However several critical issues were found in the Spark design

  • Unclear if the settings are for source or target or both
  • Sync scroll may not work for languages with different text lengths for source and target
  • Issues around “Fill with source text”: Label, Discoverability, Intent

Detailed Findings

Key Highlights of Spark design

Positives

  • The two column layout makes it easy to translate content
  • Text sync is cool and useful
  • Having fields empty by default on target content made sense to the users
  • The design is optimized for mobile/ tablet. The order of fields for mobile/ tablet made sense to the participants.
  • Design offers a way to switch to current way of editing multilingual content via “Hide source text” so that the users do not have to necessarily learn news if they don’t want to. (One participant gave 30 extra points out of 100 to this design because of this feature)

Issues & Bugs

Critical Issue: Participants were unsure if the settings were for the source language or target language or all translations. Participants commented that usually they do retain the same settings across translations however, there could be some edge cases where they would want to have settings/ translation. One of the participants suggested to provide a check box that will have the settings will be applied to all translations via a checkbox.

Critical Issue: Although participant liked that the scroll syncs in desktop/tablet/ mobile mode. However, the text sync becomes a problem when the source/ target languages are of different lengths making it difficult to translate content.

Critical Issue (Needs more research): Corresponding fields one below the other make it easier to translate content on a smaller screen for the users. However, scrolling via mouse vs. scrolling on a phone/ tablet provide different interaction experiences. Study needs to be conducted on mobile/ tablet devices to fully understand the issue.

Critical Issue: Confusion around “Fill with source text”:

  • Will “Fill with source text" copy the markup/ HTML format from the source field to the target field as well?
  • If fields from the source field (e.g. Body) have images/ positioning, will “Fill with source text”, copy them too?

The label “Fill with source text” made participant think that only the text will be copied. Participants expected “Fill with source text” will copy the formatting/ styling from the source.

Moderate Issue: The target language should clearly indicate the language of the content. Participants also want the labels of the fields in the target section be in the target language (i.e. Title, Body, Tags, etc.) so that they can get a sense of how content will be in the target language

Moderate Issue: “Fill with source text” and “Hide source text” not noticeable

Critical Bug: “Save” is missing from options

Critical Bug: Ability to copy content/ images from source to target is missing for other fields besides Body.

Key Highlights of Drupal 8 UI

Positives

Participants easily got to the multilingual edit page easily. I took a few seconds to notice the pencil (as they were used to the gear) but it wasn’t a hurdle.

The design informs the participants that the settings are going to affect all translations

Issues

Critical Issue: Participants did not like that changing the title in target content will change the title for all translations

Critical Issue: Participants did not like that the content on the target language edit page was mixed between source and target language. Participants want the UI to be in the global language by default as opposed to switching to the translated language by default because:

  • Sometimes content editors are entering content and they aren’t necessarily familiar with the language
  • For translators who understand Drupal may have training of Drupal terminology in English/ Global language of the site and translation may yield terms which they may not fully understand or be unsure about it

Critical Issue: Since the settings affects all the translation, one participant was confused as to what does “Delete” delete - source/ target/ all?

Recommendation: Provide confirmatory message as best practice when a user tries to delete content.

Minor Issue: One participant had a minor frustration that every setting option was followed by “(All languages)”. Although, it is necessary information to have but it was making the page more text heavy and adding cognitive load to the new users.

AttachmentSize
Critical Issue: Spark Settings Confusion206.75 KB
Critical Issue: Spark Sync Scroll 144.92 KB
Critical Issue: Spark Sync Scroll (for tablet) 452.17 KB
Critical Bug: Spark Save missing 48.96 KB
Critical Issue: Drupal 8 Mix Source & Target Language 140.7 KB
Minor Issue: Drupal 8 "All languages" repeated226.2 KB

Usability

Group organizers

Group categories

UX topics

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: