The future of tokens?

Posted by eaton on April 27, 2007 at 4:39am

Wow. Pathauto finally made the move!

Token module was conceived when I was writing custom_breadcrumbs and custom_links, and wanted to generate the kind of simple [foo] and [bar] links that had been popularized by Pathauto. I was this close to listing pathauto as a dependency for both modules when I ran into greggles in #drupal and started chatting with him about the need for a centralized 'Just The Tokens' utility that many modules could benefit from.

With the current release of Token module, and the current release of Pathauto, quite a few modules can now plug into "the system."

Now, though, with more modules using the tokens in more scenarios, problems are beginning to appear that we didn't anticipate. Tokens, it appears, are not JUST about a snippet of text and a description. More information, like the current language. Metadata ABOUT tokens (like whether a given token is url-safe, html-safe, completely unfiltered, etc) and so on are all part of the mix. In addition, some ideas that have been kicked around include dynamic generation of tokens -- parsing the substitution text for tokens and replacing only the ones needed, rather than generating ALL tokens and looking to see if they appear. Indeed, the entire 'context' system that token currently uses may be unnecessarily limiting. Should tokens appear as [node:title] and [user:name] rather than simply [title] and [name]?

It's time to start brainstorming. How can token.module's relative simplicity be preserved while supporting the flexibility that more modules will want?

Comments

indeed - so many uses

Posted by greggles on April 27, 2007 at 3:09pm

Indeed, it's been quite a pleasure to replace the token parsing code from pathauto.

Here are the benefits to pathauto from the migration:

The line count in pathauto module went from 1444 down to 1089 ~25% reduction :)
The number of tokens available to users more than doubled
The reliability of those tokens is much better

So...what isn't there to love about the concept of tokens?

The module is really easy to work with and a big part of my goal in discussing token so broadly is to get other developers to use the module. It will make your life better. Seriously!

One really good example of how to do this migration in the case of pathauto is the contrib/pathauto_node.inc. For most modules migrating to token, that example shows exactly what you need to do.

--
Knaddisons Denver Life | mmm Free Range Burritos

knaddison blog | Morris Animal Foundation

I caught wind of this module

Posted by rszrama on May 24, 2007 at 8:24pm

I caught wind of this module somehow or another (I can't really remember) and switched to using it instead of a custom solution for tokens in Ubercart invoices. Our on-site invoices and e-mail notifications are now taking advantage of the module, and I'll include it wherever else possible. The changes won't be committed until the next release (alpha 6c?), and it was a lot of fun to put in!

I was glad to see token_replace_multiple() when browsing the code, as I was actually going in to see if token_replace() would have to be tweaked for multiple type replacements. Might wanna just put a mention of it into API.txt.

Many thanks, and I'd love to see Ubercart up on the module list in the project description. ; )

Wish I was in Boulder.