Cutting through the input format clutter

Gábor Hojtsy's picture

As part of working on implementing easier to use input formats in Drupal 7 (reaching to WYSIWYG sooner or later), I started with cutting through the clutter. A possible rule to follow to simplify the content editing user experience is to remove (hide) filter options when appropriate. How does this work in Drupal?

Drupal core

Drupal allows setting up multiple input formats, each with different filters. The filters have specific (possibly different) configuration in each input format. The default Drupal core options allow admins to restrict filter format access to certain roles, and there should be one format accessible to all roles, so if someone has content submission permissions, she can add content to the site in the default format at least.

Input format clutter

There comes the input format clutter. Many sites have multiple input formats. Least restricted formats are used for site pages, about boxes, blocks, commonly stuff produced by the administrator for. There might be a bbcode format for forums and comments, a wiki format for wiki pages, etc. Some roles have access to multiple input formats, and choosing the right one each time complicates their life. So several contributed modules emerged to solve this problem, hiding certain input formats or prioritizing them by setting the default intelligently depending on configured circumstances. I reviewed four modules in the hopes that we can get some of this functionality into Drupal 7.

Remember filter

http://drupal.org/project/remember_filter

The most simple module of all reviewed, it just saves the last used node editing format for each user id, and loads it back when editing new nodes. It has a limitation to only deal with nodes, so input formats used elsewhere (blocks, comments, etc) are not looked at. Also, if the user edits a bbcode formatted forum topic, his next static page submission will also default to bbcode, which might not be desired. So remembering input formats this way does not look like a desired core functionality to me.

Filters by node type

http://drupal.org/project/filterbynodetype

This is a bit more complicated, presenting a matrix with all the content types and input formats available on the site. Then the admin can decide on the input formats to be possible to use with each content type. The core system on the role based restrictions on top of this setting, so it is possible to set up wiki markup only wiki pages, bbcode only forums as well as full HTML static pages. This module eliminates the need to select input formats at all. Again, it is restricted by its limitation to only deal with content types, but this is the most common use case.

Default filter

http://drupal.org/project/default_filter

This module sets a more ambitious goal. Setting default input formats per user role, per content type. Because one user might have multiple roles on the site, the module also allows setting weights for these assignments. So if a user has two roles, and there are different default input formats set up for the two roles, the user gets the "lighter" input format (the input format with less weight set).

Filter default

http://drupal.org/project/filter_default

This module allows setting of default input formats per role with weights, regardless of node types. Yes, this is a much older module, which does a similar thing as the previous module, but it lacks the node type angle. On the other hand, it comes with a slightly more comprehensible UI (it still needs some docs to be understandable though). The weights are not configurable here, but there is one weight value for each role, so the result is effectively the same.

The Better Formats module

By far the most promising module to yet emerge in this battle for ease of textual-input-format selection, the Better Formats module is a really good looking attempt to replace at least three other non-compatible, yet needed modules; Filter Default, Default Filter, and Filter By Node Type. If it gets popular, it'll be a good solution to keep an eye on for inclusion into core.

What's core worthy for Drupal 7?

From the list of modules developed here, there is a clear need for better input format limits in Drupal 7 core, so admins can cut through the clutter and offer a simpler user interface for their users easily. A generic fixed list of input formats for all possible input fields does not really cut it.

What do you think? How should we approach this? How can we provide a comprehensible user interface to limit input formats? What level of limitations are needed for core?

In the meantime, to bootstrap configurability for input formats, I submitted weight and drag and drop support for the input formats themselfs, which should allow admins to customize the order of input formats to their preference. Check out the issue: http://drupal.org/node/222183

Ps. Note that all the above modules got the "filter" name wrong. Filters are the individual components of input formats, and all above modules deal with input formats not filters. This might be misleading for some.

Comments

End-user UI

drumm's picture

Currently users see 'Input format' under text boxes. This is not very helpful since it does not make the current state clear. I would like to see the current filter tips listed with a 'Change formatting' link which would swap in the options.

agreed - fieldset summaries

moshe weitzman's picture

i agree that collapsed fieldsets hide too much. there is a massive usability discussion on this topic, and I am hopeful of big changes coming. see http://drupal.org/node/190946

I think that filters by node

earnie's picture

I think that filters by node type is what I desire. So wiki content type can have only one input filter and I don't want wiki format for page or story content types. Also, the content type should be able to pick the input filter to apply to comments for that type. If I allow comments on the wiki page then wiki format should apply to the comments.

First, a big +++1 to the

WorldFallz's picture

First, a big +++1 to the fact that all the contribs you mention are named correctly-- that's bothered my UI sensibility for a long time. Glad I'm not the only one, LOL.

OK, my functional requirements for my biggest use case (corporate intranet with all kinds of pages including very specific node types for corporate policies & procedures) are:

  1. there are applicable Input Formats based on node type (i.e. wiki page, image as node page, formal documentation pages, etc). These should apply first, regardless of role. I'd like to use mediawiki pear for wiki pages and more formal filtered html for formal documentation, bbcode for forums, etc.). I don't want users to be able to change this.

  2. then, once the set of allowable filters has been narrowed down by node type, it often requires being narrowed yet again (really just trying to fine tune allowable functionality) by role. "Admin" (admin != user1 on my site) vs. editor vs. contributor.

I'm torn as to how the UI should work though. On the one hand I'm a big fan of consolidating per node type settings right on the note type edit form-- clear, easy, and users will always know where to go when they want to specify something by node type.

But then where does the Input Filter by role info go? On the Roles edit form? Are the Input Filters by role independent of the node type form or are they a branch of a tree that begins with the allowable filters by node? This normally be the kind of complex functionality I would want to evaluate with usability testing-- mock up 2 or 3 UIs and see what real users think.

I can think about this a bit more and mockup some wireframe options for the ui if you like.

I agree that setting filters

catch's picture

I agree that setting filters per node type, then narrowing that list down with role permissions ought to work pretty well. Certainly on sites I run this would suit me very well.

Crell's point further down about it being on the field level also makes sense to me - if we change 'node type' to "body field per node type" then the two approaches look quite compatible.

Copy & adapt from other granular mechanisms?

John Bryan@drupal.org-gdo's picture

There is already a functionality in drupal which is multi tiered in it's control, that can be modified by global-site, node-type and user-group(ish) associations - CSS. I don't think that a file based system would be appropriate to this, and with CSS it is using an existing technology. But it is a multi-tiered system with a system of precedence controlling large numbers finite display "output" settings.

The question here is how to control a possibly large number of finite "input" display settings whilst reducing the complexity for the user. Looking at CSS and other non-drupal systems (e.g. Microsoft's AD Policy control etc.) may give some direction. For example there are a large number of CSS style levels applied to the user (e.g. module, node-type etc.) but the user only, if even then, sees/gets to choose one level - which user theme they get displayed. Some concepts:-

FILTERS and/or FORMATS:

(where an "Input Format" is a collection of individual filters being enabled) At what level of granuality should the input controls operate? Similar systems, e.g. CSS style sheets operate at a collection level where a collection can cosist of many settings or just a single control, this seems a good model.

"INPUT POLICY":

For (hopefully) clarity sake, so as not to confuse with what a user currently can see as an "Input Format", I will refer to a collection of input-filters as an "Input Policy". That is, an input-format that is not visible as a choice to the user during editing (unless it is a "user" input-policy).

"RESULTANT SET"

To use a common computer industry term - we are talking about a system where at the end a "Resultant Set" of individual input-filters is being calculated and applied when an edited form field is rendered. This resultant-set is to be a compilation of possibly over-lapping "Input Policies".

"INPUT CONTROL LEVELS":

An "Input Control Level" would be a layer of permission/input control such as "Site", "User", "Node-Type" and "Role". How many different levels of control should there be? I suggest "global", "site", "module", "role", "node-type" & "user" (possibly down to field level?).

"PRECEDENCE":

Should a "Site" control level setting over-ride a user role setting or vice versa. I suggest the sam order as above where the first "input control level" can be over ruled by the "input control levels" below it.:-

  • GLOBAL: All sites within the Drupal installation (not sure about the practicality of this one)
  • SITE: Effectively the the default site setting
  • MODULE: Allowing a module to control settings of pre-requisite filters, e.g. a module for mathematical manipulation may require a 3rd party higher-maths input filter enabled
  • ROLE: If a role based input-format/collection is created then it would be listed and enabled by the normal "Permissions" screen.
  • NODE-TYPE:
  • USER: i.e. an "Input Policy" that acts as a traditional, user visible, Input Format. (Later I suggest that this be a visibility setting rather than a seperate 'level' mechanism).

CONTROL OPTIONS:

Currently an input format can only enable an input-filter but can not disable an input-filter. With the limited operation of input-formats where only one input-format is active at a time this is not a problem, if a filter is not enabled then it is there effectively disabled. But once you go to a more comprehensive system this falls apart. Therefore an "Input Policy" will need to be capable of not just switching a filter on but also off. Taking examples from other systems an input policy should be able to set an input filter to:-

  • ENABLE
  • DISABLE
  • PROHIBIT: Prevent a filter from being enabled
  • FORCE: Enable despite a prohibit

e.g. the site policy might 'enable' a widget-filter but a role policy 'disable' it, the end result being that the widget filter is disabled. Or the site policy might 'prohibit' a widget-filter but a role policy 'enable' it, but the end result is that the widget filter is still disabled. Or the site policy might 'disable' a widget-filter but a role policy 'enable' it, and the end result is that the widget filter is enabled.

Sound complex?

Not neccessarily as complex as it seems. It is a re-iterative system i.e. the same basic mechanism repeated to give the granuality needed and expands existing tables and forms (see below).

Administration

At what ever level they are applied a policy could be created in the same form (a variation of the Input Format screen) with an additional or "visible" setting (a visible input-policy being one that is user selectable in the edit screens i.e. a traditional "Input Format" behaviour). Once policies are created a "select input policies" section could be added to the "Content Type", "Site Configuration" & "Permission" screens etc. would provide the list of input policies can be set to active, inactive.

Implimentation

The "filters" table would need to be supplimented with a "setting" column to allow a filter to be set to 'enable', 'disable', 'prohibit' or 'force' rather than an entry in the the "module" column only acting as an 'enable' setting. Also a "visible" column ("1" or "0") to hold whether it is a user controllable "Input Format" type policy rather than a hidden policy.
The "filter_formats" table (columns: 'format', 'name', 'roles', 'cache') would need to be supplimented with "node_type" (array of node-type names), "site" ("1" or "0"), "module" columns etc.

Despite the added processing involved in checking whether a filter is applicable it would only impact during edit forms opening so should not impact overall site performance.

A step further?

Sometimes expressed is the wish to apply filter control to blocks and individual fields within a node-type etc. Therefore additional "filter_formats" table columns might be considered e.g. "field_name" and "block" where if the field_name column is null then the input-policy will apply to all fields in the node etc.

Regards

John Bryan
www.ALT2.com

Agree Your Suggestions

robinfly's picture

I agree most of your suggestions, but i think it will a big change, may be cann't be done all. Hope to see more better changes like you mentioned.

Wow, the parent comment is a

aidanlis's picture

Wow, the parent comment is a load of bollocks.

Yes!

Crell's picture

As the author of filterbynodetype, let me say THANK YOU for taking this on! Someone definitely needs to, and I don't expect to have the time to devote to it.

IMO, the core system should only manage the creation of input formats. The application of input formats should be left to the implementing modules. That is, On a node type edit page the admin can select which input formats apply to the body, and what order they're in. Possibly role-based control here as well. A CCK textfield field should itself specify which input formats it will support, possibly by what roles, on its config page. The block module should have a settings page where the site admin can establish who can use what input formats. Etc.

There's a patch in the filterbynodetype issue queue to add support for CCK fields, but I've not looked at it closely and it scares me a little. filterbynodetype is based on form-altering the form in wacky and strange ways that have a habit of breaking if a node type doesn't follow the core node type form's structure exactly. Adding even more fragile form-alters is not a good solution.

Rather, what we need is some good mechanism for a module (node, cck, block, foo, etc.) to easily manage its own input format logic directly, using the input formats defined by the core filter module. What that mechanism is I'm not really sure, but one is definitely needed.

Personally I've always found per-node-type or per-field input formats to make a lot more sense than per-role. The per-role I've only seen used as a sort of poor-man's access control.

The per-role I've only seen

WorldFallz's picture

The per-role I've only seen used as a sort of poor-man's access control.

Yes and no-- it can be used that way. In my use case it's more related to job function and skill set. Contributors tend to be lower skilled and are more writers than html enthusiasts. Their job is to get the content in-- period. They only required the barest of html tags or wiki syntax. Editors are more advanced-- in some cases graphic designers and/or html jocks. They require pretty much everything just short of PHP. Then admins have it all. I could see my case existing without by node type before it could live without role type. Then there's the odd case of a very specific formal node type where no one should get anything more than a very restricted filtered html or forums which should be limited to bbcode or some such. I guess it could be addressed with training, but it really should be enforcible via the tool itself.

With our site we transferred

catch's picture

With our site we transferred a bunch of static pages from a dozen different sources into one or two content type. Some of these were thousands of lines with plenty of html markup, some plain text. This was way back in Drupal 4.5, and we ended up using about 5 different input formats to deal with the variety of material - probably would've stripped most tags before entry now but I didn't know any better at the time. This same content type is used by regular site users with hardly any tags allowed at all now. So it is handy to have per-role input formats when you're dealing with legacy and/or user contributed content.

Ps. Note that all the above

gaele's picture

Ps. Note that all the above modules got the "filter" name wrong. Filters are the individual components of input formats, and all above modules deal with input formats not filters. This might be misleading for some.

Might this be an indication that input format itself is misleading?

Can I add a vote for sorting

DanielJohnston's picture

Can I add a vote for sorting this out? I've got enough trouble getting site users used to the idea of forums and wiki, without them having to select the right input format each time. A key need is to separate the access controls for input and output as well as input formats by node type and user role - it makes no sense to only allow people to read stuff I've written if they also have permission to write it.

Also, is there a way of hiding the input format selection box in post editing without hacking Drupal core files?

Personally on almost all of

tjholowaychuk's picture

Personally on almost all of our sites I simply set it at the default I want and then hide it with CSS for the appropriate role(s).

vision media
350designs
Print Huge Edmonton Printing Services
Design Inspiration Gallery

I have been testing each of these solutions...

Shane Birley's picture

And I think if there is any direction, I prefer http://drupal.org/project/default_filter over the other contrib modules listed.

This type of functionality is the easiest to understand and configure and is the most flexible and could easily be rolled into the input formats core functionality. It plays well with blog api settings, security settings, and unlike the others, I never had a "gotcha moment" where I did with all the others in my test environment.

Of course, it may be be the perfect implementation but there are my two cents...

Something definitely needs

Dave Reid's picture

Something definitely needs to be improved. In the meantime, I've been liking the Preferred Format module.

Senior Drupal Developer for Lullabot | www.davereid.net | @davereid

One temporary solution with jQuery

raspberryman-gdo's picture

I wanted to force the filter, and hide the format options, based on a custom filter permission. Making this work for webforms, nodes, and CCK fields was a pain, so I picked jQuery instead. This is how I've been hacking around this in Drupal 6:

<?php
/**
* Implementation of hook_init().
*/
function mymodule_init() {
  if (
user_access('submit full HTML without filtering')) ? $filter = 2 : $filter = 1;
 
drupal_add_js(array('force_filter' => $filter), 'setting');
}
?>

And then, via jQuery:

<?php
 
$("label:contains('Filtered HTML')").parent().parent().parent().hide().find('label input').each(function () {
    $(
this).val(Drupal.settings.force_filter);
  });
?>

Better Formats

PeterZ's picture

Another new one is Better Formats (http://drupal.org/project/better_formats) which seemed to work well for my needs. From the project page:

This module was created to replace and expand upon these modules:
Filter Default
Default Filter
Filter by Node Type

The Better Formats module -

momper's picture

The Better Formats module - i love it :) - it solved me hours ...

diferent formats for different fields in a form

candelas's picture

i would like to be able to pick as default different formats for different cck fields in the same form.
i wasnt able to do it or find a way.
the case is that i need two different formats for diferent fields and this formats are specific for that field, i dont need the possibility to switch formats in each field, nor that the user does it.

i hope this gets in next realeasses.

D7 better, but needs to go further

crimsondryad's picture

We are pretty concerned about security and while D7 does a better job with text formats (as they're now called), there is still a lot of confusion about how they work. I suspect a lot of users just get frustrated and go with Full HTML, which is a problem.

Text formats work great when you don't have any overlap. For instance, external non-authenticated users should always use plain text, authenticated but mostly untrusted should use filtered html, etc. The issue becomes when you have multiple groups with different permissions working on the same content. Or, as mentioned above, when you have legacy content with a bunch of icky old html in it. We're struggling with both these scenarios right now.

One of the major issues is the all-or-nothing approach to script tags and js. Currently, if limiting html tags is on, script and style tags are stripped, no matter what text format you have permissions for. That seems like a throwing the baby out with the bathwater approach. I understand the security implications of having script and style tags on. However, we have devs who don't have full html perms ( especially due to the number of interactive html 5 tags ), but who do need to be able to put script and css in certain pieces of content. This is particularly painful on legacy content we are migrating in from an old site. I would love the luxury of cleaning it all up before import, but the reality is that we have difficult deadlines and there is no way we can clean out thousands and thousands of pieces of content.

The implementation is also uneven. In some instances, if there are tags in the content that the current text format doesn't allow, it just greys out the field. In others, it strips the tags entirely, which can completely bork the content and doesn't really give you a method of getting those tags back.

We also noticed that if you change the weight of the text formats, it will also change the defaults for every content type. That makes it hard to set some content types to have tighter or more permissive text formats than others. The user would then have to choose between available text formats and they just aren't technical enough to do that. I realize it would be more complex to set the text format per content type, but it's really confusing when you change the weight and suddenly all your content types are using a different format.

The CKEditor group says they are using HTML Purifier for more flexibility, I plan to kick the tires on that. Also, they have evolved better formats for D7 to handle some of the issues that still exist. I've been reading a lot of the D8 initiatives and I realize none of these topics have easy answers. I appreciate all the work that has gone into making D7 so much better in a lot of ways than D6. :)

"Plain text format" applied to every content by default

Frank Ralf's picture

One cannot disable the "Plain text format" in D7:

All roles for this text format must be enabled and cannot be changed.

So this text format is applied to every content by default (which in effect makes it another filter) , in most cases in addition to some other text format. Given the default text format order of a D7 installation (HTML > Plain text) this leads to issues when a text field contains HTML code, because the "Plain text format" will escape all the HTML the "HTML filter" allowed.

IMHO this is a bug of Drupal core. And the documentation on Text filters and Input Formats has to be upgraded to reflect these substantial change in Drupal 7.

One cannot disable the "Plain

David_Rothstein's picture

One cannot disable the "Plain text format" in D7

Yes, like previous versions, Drupal 7 requires that you have at least one text format that is available to all users. However, nothing prevents you from renaming and reconfiguring it if you want it to behave differently (e.g., rename it to "Limited HTML" and configure it to allow a limited set of HTML tags)...

So this text format is applied to every content by default (which in effect makes it another filter) , in most cases in addition to some other text format.

Not sure what you mean by that. Only one text format is applied to each piece of content (whichever text format it was saved with). No piece of data should ever have two text formats applied at once... If you know of a bug with that, please file it and link to it from here :)

Double escaped HTML

Frank Ralf's picture

Thanks for the quick reply.

Only one text format is applied to each piece of content (whichever text format it was saved with). No piece of data should ever have two text formats applied at once...

Yes, that was the behavior we expected. But somehow we got double escaped HTML like the following for a text field which was set to use the HTML format. We could only amend this by disabling all settings in the Plain text filter.

&lt;a href="<a class="ext" href="http://mydomain.de/rezepte/thunfisch-maki-rollen/&quot;>Thunfisch-Maki-Rollen</a>">http://mydomain.de/rezepte/thunfisch-maki-rollen/"&gt;Thunfisch-Maki-Roll...</a>

I will have a closer look and file a bug if necessary. It might have something to do with this content being generated by an import script.

EDIT:
Usability: "Plain text" is confusing... seems to be the relevant issue:

Remaining tasks
Clarification is needed on what "plain text" means in Drupal in the documentation; Plain text functionality has changed in D7 (compared to D6). Please also add screenshots (before and after). At least one work around is also needed.

EDIT 2:
It definitely has something to do with the content being imported. Just saving the node again manually gets rid of the double escaped text.