Implementation ideas

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Jeff Veit's picture

I think that there are several key elements to growing Drupal into a capable registration system:

  1. The holding space for the events related information
  2. The storage of data related to the event
  3. Tracking people related to the event
  4. Tracking attendance
  5. The registration process
  6. The process of setting up an event

Points 3 and 4 are interrelated - if you only track attendance, then point 3 is redundant. Point 3 applies when you track other relationships, for example, whether someone is a prospective attendee.

I'll write about implementing each of these separately in the comments.

Comments

Holding space for events related information

Jeff Veit's picture

For each event you need one or more pages for the event, one or more registration forms, a payment mechanism if required, and administration pages and reports.

For small events, a single node might do as the centralization place. The Event module gives you an events listing. Used together with the Signup module, you have a single node acting as the centralization space. It includes the registration form on the node, and provides tabs for the aggregated information like attendee names.

For a one-shot event site, this isn't an issue: the site is all about the event. Drupalcons are examples - they each get their own sites. Extra pages can handle the administrator information.

For larger, multiple events, organic groups might be suitable.

I can imagine something similar to organic groups, specialized for event handling, but I think that working with Organic Groups has more value.

I think that events enabling modules should work with all these scenarios.

Are there any other suggestions?

Storing data related to the event

Jeff Veit's picture

When someone registers for an event, you often require some event specific information, for example food preferences, whether they need a parking space, or which sessions they want to attend.

This information has little value beyond the event, but for smooth running of the event, it's essential. Most often there is some information that is captured for all attendees regardless of ticket type or attendee type, for instance their badge name, and usually this common information is processed in a way that's the same for all . E.g. producing badges.

I see several possibilities:

  1. Store it in some custom table. Write a suitable module for capturing the data and outputting as required. This has the advantage of being able to customize the data structure. For instance, it would make group bookings easy. More code is needed than leveraging an already working project. The lovely Form Builder might be persuaded to work with this. Multipage data captures might be difficult; the custom table might be more than one table. Pageroute might be used.

  2. Store the information in a webform. This has the advantage of some data security: webforms are not content and are not displayed as nodes. The webform is never indexed for search. But webforms are flat structures: a group booking might need another structure to track the group. Different ticket types/ attendee types at events often have different data gathering requirements. Sometimes a webform can be shared between these ticket types/attendee types, other times they can't. Non-shared forms make reporting difficult. The Webform Report module, with fixes for the security issues might help. Multipage webforms might be achievable using the Pageroute module.

  3. Store the data as a CCK node type. Upside: CCK is good at handling shared data and has lots of widgets. Upside: excellent Views integration. Again Pageroute could be used. Downside: nodes are searchable and this could lead to information leakage. Downside: this data is transient and once the event has concluded it should be discarded. That means deleting this information.

  4. Store the information in a Signup. Signup lets you include forms and can serialize the form into the signup log. describes this. Upsides: Signup already exists, and there are some useful add on modules. It's integrated with Views. Downside: Signup doesn't do group bookings. Reporting on serialised data is expensive. Maybe Webform Report could be extended to deal with Signup data.

Is form builder ready for

joachim's picture

Is form builder ready for this sort of thing?
I was thinking webform, but it needs a fair bit of bending and stapling to use outside of its own node as the webform_associate module shows.

Dunno

Jeff Veit's picture

I've never used it for anything serious.

Two related points:

The signup module serialises fields I think. Webform too, I think. Needs checking. There has been talk often on the signup issues list about the lack of forms. I think theres been an experimental version, or a patched version that used webforms for fields. Currently dww, the author, is holding off fields because they will be part of D7, and there should be a uniform interface for adding them.

There's also some discussion related to signup on how to use cck to store many forms in one node, and this is maybe ehat is needed. Still to investigate.

Webform stores all data in a

joachim's picture

Webform stores all data in a single table like this:

Node ID, Component ID, Data

This is the same as Profile module.
So it's accessible to Views by joining on both the nid and cid. AFAIK there isn't Views support yet, but it's doable.

Tracking people related to the event

Jeff Veit's picture

In my view we have:

  • People and organisations we know something about, who are not yet users of the system. Drupal has no concept for this sort of person. CivicCRM does. We may have an email address, but not necessarily. An example might be people on an invitation list; we will want to produce a letter or mail for these people, but they haven't registered.

  • People who have registered for an event, who also want an account on the system. It's for events where there's a good amount of information on the web, which is only available to participants or registered users. Or where the event is an offshoot of a community site. If someone has a login, then we can use profile data to fill in event fields during registration.

  • People who have registered for an event, who don't want or need an account: anonymous registrations. Organizers want to know about these people for the future. And of course need to track their attendance. But the attendee isn't interested in a longer term connection to the site/community. We can create accounts for these people (although we might not have their email addresses) but they should be noted as dormant. This info is not available during subsequent event registrations. We may also have multiple registrations of the same person, so it may be useful to have some account reconciliation tools to combine accounts.

  • Registrations at the event. This is usually similar to normal registration, but data input is optimized for speed, and includes badging, and often this data will need to be backported to the live site, if it was collected offline. IMO it should be collected in the same structure as pre-registrations.

Constraints: Drupal only has the notion of a user as an account holder. It wants every account to be associated with an email address.

Implementation: I can see three possibilities -

  • Have a separate table for people known to the system, who are not registered users, and refer to this for in the wider events handling. Simplenews maintains it's subscribers table, independently of the subscribers Drupal site registration. RSVP module keeps a list of invites to an event, and connects where possible the invitation to a known user. Signup similarly, I think, can deal with anonymous registrations.

  • Everyone known to the system has an account. Where the email address is not known, a dummy that follows a well-known pattern is used. People who have never registered but are known to the system are given a particular role, e.g. the 'prospect' role. The system would need account reconciliation mechanisms to allow people to register on the system despite already being known to the system. This would be especially required where they were already known to the system as a prospect with an email address since Drupal requires email address to be unique. (Why??)

  • Don't track prospects and people who have indicated that they are not interested. Disallow anonymous registrations. (May be a problem with group bookings.) To the extent that you want to talk to these people, hold them in Simplenews. Create event registrations using the system you use (CCK, webform or whatever) and only create accounts for people who indicate that they want them, or if they are required to participate in the event.

Have a separate table for

joachim's picture

Have a separate table for people known to the system, who are not registered users

There's an API module for this but I don't know how ready it is: http://drupal.org/project/visitor
There's also something in the works for D7: http://drupal.org/node/355513

However, we need more than this as we need to know about people who might never actually touch the site themselves (case of office dogsbody registering 10 people from their organization for a conference).

Tracking attendance

Jeff Veit's picture

I can see several ways in which it can be done:

  • Everyone who registers gets a user account. Nobody who has not registered has an account. This works fine for single event sites where the event is the site. It doesn't scale.

  • Or, when someone registers if they aren't a registered user create an account. Give them a role that confers registration.

    This can work for a small number of events, but you get a proliferation of roles when you run lots of events. If your site is complex with lots of modules, then permissions for the roles and making sure that they are right can be tricky. It might be okay if the role creation was automated. For example, a new attendee role for a conference could be based on the authorized user role + permissions relating to the particular event. You may need more than one role for the same event, to represent different ticket types. Views works nicely.

    Event data could be stored separately from the person, but you would need a link module to create Views. Or if event data is stored as a CCK type, then you needn't. You would end up with one CCK type for each ticket type, and different event might need different node types to store the different data collections. CCK would work nicely with the shared data gathering.

  • Or, when someone registers, give them an account if necessary, and a user flag.

    This means that everyone who is an attendee needs an account. You get a proliferation of flags. Creating lists of people is easy though. And Views works nicely.

    Everything above about storing event data separately or in CCK still holds true.

  • Or, use a User Queue. One per event. It works like Node Queue.

  • Or, when someone registers, they get an entry in a webform. Different events have different webforms. A user account is not required to make a webform entry.

  • Or, store the registration in a CCK, giving anonymous users the right to create bookings, a CCK node type. User reference to link to people who are registered users. Either create a new CCK per event, or use nodereferences to link to the event.

  • Or, use the signup module to track registrations. Extend to group registrations.

  • Or, build a custom module for registrations. (We already have Signup, but it's not group friendly, and holds form data as serialized.)

  • Or, through Ubercart, you have a product purchase.

Other ideas?

I was pondering this kind of

joachim's picture

I was pondering this kind of flow:

  1. Event node. (not event module though; that's not compatible with date/calendar modules is it?)
  2. Link to a form that takes an email for the person who is making the registration. This creates an account so they can return to their registration later.
  3. Ask how many people they want to register, and ask for general options that apply to the whole group: which days of a multiple event for example.
  4. For each participant, take the registrant through a form that takes:

- name, contact details,
- event-specific options: meals, accommodation, etc etc. This is the part that needs to be buildable by a site admin for each event: webform / formbuilder?

If the event is paid, this needs then to push through to add a product to the UC basket and direct them to pay.
We also then need to store all manner of data, push stuff to Simplenews for campaigns, etc etc.

check out the UC demo

Jeff Veit's picture

UC has a demo of events which is interesting. Look at UC2 demo for D6.

You mean this one?

joachim's picture

You mean this one? http://demo.ubercart.org/product/one-day-uberevent

That's using UC product attributes to alter the price. Definitely something we want to make use of.
Though whether we use the actual UC form, or make our own form and then feed the data to UC I don't know.
UC serializes the product options, which is damn ugly for our reporting purposes (and for UC too!). I'm inclined to store everything ourselves and pass on stuff that affects price to UC through the API.

Yep

Jeff Veit's picture

Yes that's the one I meant.

The registration process

Jeff Veit's picture

See http://groups.drupal.org/node/24458#comment-85692

But I think registration should be a modular process. Some people want to be able to push registration out to an external provider. The system should make that easy.

Ubercart can handle the registration process in terms of products. With each product you can have subproducts (e.g the ability to add an additional track) and you can have required details, for instance attendee name. A UC integration isn't so great for free events but the idea of a cart to create a group booking is attractive: it lets you set up the booking and then submit it as an atomic element. This might not be so good where there is competition for resources - e.g. tickets - because your tickets aren't reserved - they are only booked at the final step.

The process of setting up an event

Jeff Veit's picture

If the event is a one-off, with a custom site, then this isn't likely to be an issue: everything can be hand-built.

If the event is one of a series of free events, without complexity, then installing Signup, enabling it for a node type and setting up each event as a node is not too hard.

For larger scale, complex, repeatable events, setting up everything from scratch is overhead. Ideally you either want a template (not in the Drupal design sense), with tickboxes to say 'I want an RSVP and a wait list for this event', or you want to be able to clone the structures used in a previous event.

For events built around OG, Node Clone might work. (Not tested.)

Possibly a combination of Space, Features and Context might work to provide a platform for event repeatability.

Or Patterns.

I think some of the event

joachim's picture

I think some of the event stuff will be new modules.
For stuff that wraps around existing modules like simplenews, UC, etc, we'll need glue modules to connect. Features would be a good way to get started on those.

Note that CiviEvent does a fair amount of the above ...

lobo's picture

and i do think that its a fairly non-trivial piece of work

lobo

Easy at the low end

Jeff Veit's picture

Yes, Civic does do basic event registration. The backend isn't strong.

One thing it is very good at is contact management - it differentiates between the prospects and the registrations. It's also good at mailing - naturally.

your probably should take another look at CiviEvent 3.0 ..

lobo's picture

Its becoming more and more of a full featured event management system and has a fairly large set of features that are being discussed here

not sure what you mean by "the backend isnt strong"

lobo

Backend needs

Jeff Veit's picture

Last time I looked, at least 6 months ago, and probably longer, the registration part was ok, although I can't remember if it dealt well with different ticket types, but the admin part of the system was weaker. I remember it giving a list of participants, and the ability to change the configuration of events, but not a lot more.

I've just looked at the current demo and it's much much better than I remember.

Strengths:
Does most of what we want
Includes roles
Includes ticket types
Includes a store
Fantastic at dealing with prospects and attendees

Weaknesses:
It's not a Drupal module & not under Drupal licence
Requires a CivicCRM too

Bottom line for me is that

joachim's picture

Bottom line for me is that CiviCRM just doesn't play ball with Drupal.

Good model

Jeff Veit's picture

Civic is a good model; the registration process is good. The backend is well organised too. But it's not easy for people to use as part of Drupal.

lobo's picture

"not easy for people to use as part of Drupal."

note that the recent versions have added a lot of hooks (i.e. easier to extend like Drupal), integration with Drupal modules (the Views integration is quite good and is getting better. there are attempts to do CCK integration also which i suspect will happen sometime soon)

Yes, it does use a different form library and a different templating system. But overall its not too different :)

And finally, we do have a working version of a pretty decent Event system :)

lobo

Sure

Jeff Veit's picture

What I mean is that once you have base Drupal up and running, adding features is normally: download module, enable, change module settings if necessary, create content where appropriate, and you are done.

Using Civic introduces a big step: install Civic. And a subsequent bunch of smaller ones - deal with the Civic templating, deal with civic configuration and administration.

Lots of this is handled by signup.module

dww's picture

I only had a chance to briefly skim this post, but LOTS of what's being discussed here is all handled by the Signup module. Before anyone spends too much effort redesigning an entire system and implementing something from scrath, please take a look at what signup (and a host of add-on modules that integrate with it) can already accomplish. If anything is missing, I'd be very happy and willing to work with people to either add it to signup itself or a separate module that works with it. Let's avoid Druplication if we can...

Thanks!
-Derek

I'm all for reusing /

joachim's picture

I'm all for reusing / expanding existing components! :)
Will take a look at signup soon.

It makes sense to me.

goose2000's picture

It makes sense to me.

Signup is good

Jeff Veit's picture

I agree that signup is good, and it's got lots of additional modules which add great functionality.

Strengths:
- There's a good basis
- There's loads of embedded knowledge in it
- Good selection of extra modules
- It has some backend facilities
- It deals with anonymous and registered users

Weaknesses:
- It's aimed at single user registering for themselves
- it's aimed at free registrations, but there's an addon with Ubercart
- It doesn't do fields fully - no ability to design
- it doesn't have an idea of different types of registration
- No notion of prospects etc, although RSVP module adds the start of a campaign system
- It has a fixed idea of language - signup vs registration vs notification - can be fixed
- It's less of an API than it could be

Most of the weaknesses are relatively easily remidied.

Webform could replace the built-in registration. This has the advantage that webform is widely used and I expect that it will use fields in d7.
The idea of a booking could be introduced to deal with group registrations.
It could use Ajax in-place editing for the decision branching. (Problem - corporations sometimes have problems with Javascripy on IE.)

The biggest problem is likely to be ticket types and the need for different fields for ticket types, and the way in which this could be dealt with. One solution might be to have more than one webform for a ticket type. Then the parts that were common would be represented using one form and the parts that were different woul be dealt wth using another.

But dealing with mixed webforms in Views would be much messier.

IMO basing the work on signup is one of the strongest options.

I was listening to a

goose2000's picture

I was listening to a lullabot podcast today about install profiles and distributions. I'm starting to think that this is what would be built? With signup module and the other main stays like CCK, Views and others?

Not quite what I have in mind

Jeff Veit's picture

No, that's not what I was imagining; a profile gives a fixed set of functionality as a base. While this could be useful for single event sites, I don't want to maintain anything like that.

What I'm thinking about is a way to replicate events and have different event types within the same site. For instance, an organisation may well run day long classes on how to program as well as week-long conferences bringing together programmers from lots of cities. Further they may repeat the classes in different cities, regularly, but only run the conference every second year.

It seems sensible to me to have a mechanism that can template or replicate whole events, sometimes classses, and sometimes a conference, that lets more than one event exist in a site.

We'd need an import/export

joachim's picture

We'd need an import/export system/loading defaults from code, and that would entail cloning.

Good review -- a few replies

dww's picture

Thanks for the review. Re: the (mostly very accurate) list of weaknesses:

- It's aimed at single user registering for themselves

Not necessarily. There's a mechanism for admins to signup other users, and some sites use it entirely in this capacity.

- it's aimed at free registrations, but there's an addon with Ubercart

Right. I want to keep the base module as "simple" as possible and allow for add-ons to provide more complicated work-flows. There are at least 3 different pay-per-signup options I know of, including most recently uc_signup.

- It doesn't do fields fully - no ability to design

Yeah, this totally sucks. ;) There's been lots of discussion about this over the years at #29568: Flexible number and type of fields. You can customize the form with a theme function, but it's insane. Signup status basically is an add-on module to give you 1 real customizable, feature-rich field on your signups (which in turn has its own set of add-ons for other functionality based on this field). However, I'd like to just convert signup to use core fields in the D7 port...

- it doesn't have an idea of different types of registration
- No notion of prospects etc, although RSVP module adds the start of a campaign system

Not exactly sure what you mean by these. Can you elaborate?

- It has a fixed idea of language - signup vs registration vs notification - can be fixed

Right. Signup is (finally) very themeable, although some of the URLs are going to still contain "signup", even if you theme (or translate) the UI.

- It's less of an API than it could be

This is less and less true. For example, uc_signup makes heavy use of the Signup API (such as it is). It's less explicit about being an API, but it's there....

Most of the weaknesses are relatively easily remidied.

Agreed! I'd love to collaborate with folks who want to tackle any of the above...

Cheers,
-Derek

Do you have any thoughts on

joachim's picture

Do you have any thoughts on how to collect more information on signup?
Fields in D7 would be great, but what about D6?

For example:
- a 2-day conference allows people to sign up for either or both days
- event registrants can specify their meal preferences

I was thinking of bringing in webform somehow, but I'm open to ideas?

Webform

Jeff Veit's picture

My opinion is that signup needs to delegate to webform for data collection.

Advantages:
* Webform has promised a D7 version, and I expect that it's going to be based on fields. (I don't know this but I think they'd be mad to skip using common infrastructure.)
* Webform has an interface for adding fields.

Disadvantages:
* Dependencies introduce problems
* Could be complex - e.g. badge name is a common collection requirement across different attendee types. A good system makes it easy to produce a complete list of badge names, and badges. Either this means pulling data from different webforms or it means having a CCK style relationship between webforms - with shared parts and non-shared parts. For example badge name would be in one webform, but car registration number (for speaker parking places when there's limited parking) would be in another webform available only to speakers.

Webform is most probably not

joachim's picture

Webform is most probably not moving to fields: http://drupal.org/node/118984#comment-1872692 --

At this point a CCK/Field-based version of webform seems very, very unlikely. In D7, Fields-in-core creates a table for EVERY field. Considering I've seen sites that have 300+ fields on a SINGLE webform, that would mean 300(!) tables for that webform alone. This would also mean it would 300 JOINs to retrieve the results, or 300 individual selects to get a single result. Basically, a CCK-based Webform would be impossible from an efficiency standpoint and will likely never happen at this point.

Fields in core engine

Jeff Veit's picture

I think they'll change their mind:

Firstly does it matter if there are hundreds of tables, if they are managed? It's not like they are touched by human hand. Are joins an issue? Maybe not because every read creates a cached copy (for nodes anyway) and this is an extensible strategy. After the cached copy is created, it's a single read I think.

But if that wasn't enough...

Fields in core is pluggable, so there are three(?) ways that the hundreds of tables problem can be overcome:

A hook allows you to change the storage on a per-field or per-bundle basis. So instead of having multiple tables, webform could use it's traditional form.

In fact there is already a module that implements strorage in a different way: it's called the PBS/Per bundle storage module. This stores fields in a form much like CCK does.

And lastly the materialized views strategy is also available to make fast-read versions of the data.

The benefits of using fields will be enormous - hundreds of interesting widgets and probably a standardised UI for creating forms -so I'd be surprised if there wasn't lots of pressure for them to change.

Clarifications

Jeff Veit's picture
  • it doesn't have an idea of different types of registration

I meant that different sorts of registration for the same event sometimes (often?) require different collection of information. For instance, speakers at a conference may need to provide a bio, whereas ordinary attendees may not need to. These different roles may also have different process rules applying to them - for instance speakers may need a code to verify they are who they say they are. While the system can hold different attendance lists for each role, there are parts of a registration system where the different roles come together - for instance in active entrance control - you need a list of all attendees or a system that can verify against the attendance list.

There's a second meaning, but I wasn't thinking this when I wrote the comment but it's relevant... different types of event have different language, registration requirements and interaction rules. What does that mean? An online RSVP for a family party is very different in the language of the interaction to a conference registration; you wouldn't call a guest an attendee. But they both fit the signup framework, so each event needs control over the descriptive language of the interface. Registration requirements: a conference requires much more data collection than a party - ideally a party RSVP needs no info from the guest. Interaction rules: a family party RSVP requires a light touch - ideally the single click because of the relationship between the guest and whoever is holding the party. In contrast relationship between organizer and prospective attendee means that the attendee is prepared to go through a multi-stage process to join a conference. But these two extremes should ideally co-exist in the same framework, with both types of event active at the same time. I think that they can if active portions are chosen on an event by event basis.

  • No notion of prospects etc, although RSVP module adds the start of a campaign system

Lots of events need to advertise themselves. It's typically more than a one-shot process and repetition is boring. Different people also sign up for information about the event at different stages in the lifecycle of the event. Before they've registered for the event, they are prospects. A campaign takes note of who knows what about the event, and delivers information that's relevant to each person. For instance if the event is a barcamp style event, then planning might take place in public. People who register interest before the cut off date for session proposals should be told about what planning is going on, and what sessions have been proposed. This is true for anyone who registers before the cut-off date. People who register interest after the cut-off date should not receive this mail. People who have signedup should not receive any further marketing style mail. There's a sort of workflow of who should receive what information. That's what I meant by a campaign system.

Backend needs

Jeff Veit's picture

Oops - wrong place - comment moved so it makes sense. No way to delete it.