Comparative Multilingual Content Edit Experience Study
Focus of the study: To get answers to the following high-level questions:
- How do users current translate content on Drupal?
- How do users translate content on current Drupal 8 and current Spark translation feature?
- Which version do the users prefer? Why?
Number of participants: 3 (Acquia Internal), 3 (Partners/ Customers)
Participant profile:
- Participants who create/ translate multilingual content or have a need to do so in the near future
- Participants have not used the current Drupal 8 and Spark translation feature
Study Date: June 25-26, 2013
Recruitment: Participants were recruited through internal e-mail and Acquia Sales contacts
Compensation: No compensation was provided to the users
Methodology:
- Participants have not used the current Drupal 8 and Spark translation feature
- Moderated usability session for 30-45 minutes each
- Participant performed same set of tasks on each of the designs
- Half of the participants were shown the Drupal 8 UI and then Spark UI. The order was reversed for the other half of the participants. This was done to eliminate the bias from the order
Overall Summary
5 of 6 participants preferred Spark design over Drupal 8 multilingual design and 1 participant did not have a preference.
How would you split 100 points giving more points to the design you like? (Average of 6 participants)
- Spark - 75 points
- Drupal 8 - 25 points
However several critical issues were found in the Spark design
- Unclear if the settings are for source or target or both
- Sync scroll may not work for languages with different text lengths for source and target
- Issues around “Fill with source text”: Label, Discoverability, Intent
Detailed Findings
Key Highlights of Spark design
Positives
- The two column layout makes it easy to translate content
- Text sync is cool and useful
- Having fields empty by default on target content made sense to the users
- The design is optimized for mobile/ tablet. The order of fields for mobile/ tablet made sense to the participants.
- Design offers a way to switch to current way of editing multilingual content via “Hide source text” so that the users do not have to necessarily learn news if they don’t want to. (One participant gave 30 extra points out of 100 to this design because of this feature)
Issues & Bugs
Critical Issue: Participants were unsure if the settings were for the source language or target language or all translations. Participants commented that usually they do retain the same settings across translations however, there could be some edge cases where they would want to have settings/ translation. One of the participants suggested to provide a check box that will have the settings will be applied to all translations via a checkbox.
Critical Issue: Although participant liked that the scroll syncs in desktop/tablet/ mobile mode. However, the text sync becomes a problem when the source/ target languages are of different lengths making it difficult to translate content.
Critical Issue (Needs more research): Corresponding fields one below the other make it easier to translate content on a smaller screen for the users. However, scrolling via mouse vs. scrolling on a phone/ tablet provide different interaction experiences. Study needs to be conducted on mobile/ tablet devices to fully understand the issue.
Critical Issue: Confusion around “Fill with source text”:
- Will “Fill with source text" copy the markup/ HTML format from the source field to the target field as well?
- If fields from the source field (e.g. Body) have images/ positioning, will “Fill with source text”, copy them too?
The label “Fill with source text” made participant think that only the text will be copied. Participants expected “Fill with source text” will copy the formatting/ styling from the source.
Moderate Issue: The target language should clearly indicate the language of the content. Participants also want the labels of the fields in the target section be in the target language (i.e. Title, Body, Tags, etc.) so that they can get a sense of how content will be in the target language
Moderate Issue: “Fill with source text” and “Hide source text” not noticeable
Critical Bug: “Save” is missing from options
Critical Bug: Ability to copy content/ images from source to target is missing for other fields besides Body.
Key Highlights of Drupal 8 UI
Positives
Participants easily got to the multilingual edit page easily. I took a few seconds to notice the pencil (as they were used to the gear) but it wasn’t a hurdle.
The design informs the participants that the settings are going to affect all translations
Issues
Critical Issue: Participants did not like that changing the title in target content will change the title for all translations
Critical Issue: Participants did not like that the content on the target language edit page was mixed between source and target language. Participants want the UI to be in the global language by default as opposed to switching to the translated language by default because:
- Sometimes content editors are entering content and they aren’t necessarily familiar with the language
- For translators who understand Drupal may have training of Drupal terminology in English/ Global language of the site and translation may yield terms which they may not fully understand or be unsure about it
Critical Issue: Since the settings affects all the translation, one participant was confused as to what does “Delete” delete - source/ target/ all?
Recommendation: Provide confirmatory message as best practice when a user tries to delete content.
Minor Issue: One participant had a minor frustration that every setting option was followed by “(All languages)”. Although, it is necessary information to have but it was making the page more text heavy and adding cognitive load to the new users.
Attachment | Size |
---|---|
Critical Issue: Spark Settings Confusion | 206.75 KB |
Critical Issue: Spark Sync Scroll | 144.92 KB |
Critical Issue: Spark Sync Scroll (for tablet) | 452.17 KB |
Critical Bug: Spark Save missing | 48.96 KB |
Critical Issue: Drupal 8 Mix Source & Target Language | 140.7 KB |
Minor Issue: Drupal 8 "All languages" repeated | 226.2 KB |