Activity stream modules

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!

Activity stream functionality (eg Facebook's activity stream) is an increasingly popular feature for social websites. It's also a very complex feature both in terms of functionality and architecture, so it's unfortunate that we've got developer fragmentation in this area.

Luckily, we have the power to collaborate! The goal of this analysis is to understand the differences of the main activity stream modules and ideally, spur cooperation between the maintainers on fewer projects, or to at least understand why four essentially competing projects are necessary.

Here's a comparison about the functionality and architecture of the main activity stream modules. See below for a list of modules that were not included and reasons why they were not included.

Some background threads on these modules:
Ultimate activity feed - programming, technical -- This thread is particularly useful as it includes several of the developers from the below modules. However, it doesn't fully explain how we ended up with so many modules!

Ultimate activity feed - features, UI
Activity, Heartbeat, Activity Stream, Activity Log

. Activity Activity log Heartbeat Message
Project creation date 9/22/2007 9/22/2008 11/9/2008 4/1/2010
Total reported installs *1 2667 44 1350 1400
Maintainer(s) cvangysel , Scott Reynolds, Sirkitree IceCreamYou Stalski Amitaibu (Maintains OG)
Last commit 4 Weeks ago 8 hours ago 2 Days ago 1 week ago
Has D7 release Dev No Recommended Recommended
Maintainer stated D7 commitment ? Plans to update Yes Yes
Logs activity based on Core triggers - more limited than Rules Rules Rules & an API Rules & an API
Access control method Has 'nid' column on {activity} - How is this used for node access? Views displays and Digests enforce node access restriction in hook_views_pre_render() by way of a db_rewrite_sql_query against the list of nids being displayed. Access control not built-into message architecture. Own algorithms (stream access, message template restrictions and user scope (groups/related/friends/whoever-implements-hook), node_access when displaying links to nodes In progress
Visibility/realm control *2 ? Currently: a list of viewer ids (uid, group id, etc) are generated for each message and messages visibility is determined at time of message generation. Visibility to be re-architected so that it is determined at time of viewing: http://drupal.org/node/1259306 Currently: a list of viewer ids (uid, group id, etc) are generated for each message and messages visibility is determined at time of message viewing In progress
Messages are rendered when (generated vs viewed) Generated Generated Viewed (but with variables at log-time) Viewed (messages are not cached so they are generated dynamically each time.)
Supports message grouping *3 ? Yes, configurable. Supports grouping of messages grouped by some variable (by user, by node, by term). Configurable Views Group By
Variable are accurate as of (activity logged time vs view time)* ? View time Log time View time
General Internationalization support ? Somewhat - Not currently working - See http://drupal.org/node/1261914 Yes, via t() (D6: activity inherits the language from nodes or logs in all languages if language undefined) Yes, via I18N
Documentation of support for Spanish verb forms (eg, config steps, screencast or info on a workaround) Possible problems with this: http://drupal.org/node/411814 Potential support but not 100% ? Potential support but not 100%.
Can comment on activity ? Yes yes (in node context configurable to switch to real comments) Reply module
Layout manageable [needs clarification] ? ? yes (via templates and Display Suite default layout and settings) Display Suite, or other entity render techniques.
Features integration [could use elaboration on what's exportable] ? ? yes (through ctools) yes (through ctools)
Exportable templates ? ? yes (through ctools) yes (through ctools)
Exposes entity form to push activity in the stream no no yes doable, via via entity API module
Stream polls for newer activity ? ? Yes, configurable Views autorefresh
Nodejs integration ? ? yes (instead of polling for newer messages) no
API documentation Yes Yes Yes Yes
Latest schema activity.install activity_log.install heartbeat.install message.install

*As of 13, Sept, 2011
*2 (eg, activity messages for members of group x)
*3 eg Bob joined group x, y and z, or Mary, Jason and Billy like this comment)
*4 If message elements such as node titles, usernames or user pictures change after the activity is logged, will the updated (title, username, picture etc) be used in the activity stream message? If the variables are substituted at View time then yes. If at render time then no, unless the update action triggers a re-rendering of the message. Substituting the variables when the message is viewed, rather than when it is rendered has likely performance costs.

Modules not included in this analysis:

http://drupal.org/project/FriendFeed - Integrates an external service, no stable release, last commit 1 year ago
http://drupal.org/project/lifestream - No release, last commit 2 years ago
http://drupal.org/project/status_router - No release, last commit 2 years ago, 3 site installs
http://drupal.org/project/activitystream - No commits for almost a year, every item in the stream is a node, which is likely to have negative performance impact. 1542 reported installs.

Comments

Some notes

Michelle's picture

Activity and activity log do cover the same territory, but activity log does it using rules and actions, I belive, and started at a time when activity seemed to be dead. Because it was such a drastically different approach, it didn't make sense to take over activity. The maintainer said it was more of an experiment and I don't know if he plans to continue it or not. Might not be worth it now that activity is showing life again.

User activity is just a number, if I'm understanding right, and doesn't list activity at all, so that's only related by name.

Friend feed is integration with a 3rd party service, which is totally different.

Lifestream looks dead and I don't know what it is.

Activity stream gathers information from a bunch of 3rd party sites. If that could be made to show Drupal activity as well, then that might be a good candidate to rule them all.

From a quick look, heartbeat looks like a straight dupe of activity. :(

Michelle


See my Drupal articles and tutorials or come check out the Coulee Region

motivation to write message?

kyle_mathews's picture

Amitaibu -- could you write a bit more about your motivation for writing Message? I'll need to pick one of these at some point in the next 3-6 months and the constant multiplication of choices is quite confusing.

Kyle Mathews

@kyle_mathews, I've written

amitaibu's picture

@kyle_mathews,
I've written on the Message page some differences, and posted a movie as-well.

But I'll try to summarize it in one liners:

1) Activity - Currently, most popular module. The messages are hardcoded and assigned to users. In case of multiple languages or the need to assign to different enteties, its disadvantages are more visible.
2) Heartbeat - I'm not too familiar with Heartbeat's code. The main reason is that it is in my opinion too complicated, and has too many assumptions on how it will be used.
3) Message - Built on top of Ctools, thus is more lightweight (on the other hand there are less features). As messages are not hardcoded, and placeholders are being used, one may use PHP to populate the message (which of course comes with a performance hit). Messages are translatable and exportable.

exportables key

kyle_mathews's picture

I really like that Message is exportable. Cool, thanks for the great work there. I might be using it fairly soon.

Kyle Mathews

About complexibility

Stalski's picture

@Amitaibu : i know it's rather complex and i worked a lot on the "making it easier" part. The problem is that there are some features that bring a lot of complexibility with it, that i don't see in other activity stream builders.
However, the problem you mention "too many assumptions on how it will be used", is a real one i realize. Could you point me in the direction where i could make improvements to make it more drupalish? People already told me a video would help a lot ... maybe i really need to do that.
Soon i will start working on the drupal7 organic groups integrations. Looking forward to it after seeing your session in Copenhagen.

General in this discussion:
Heartbeat has exportables, rules integration, features integration, organic groups and friends api integration (flag_friend, UR, friendlist), shouts as small twittery system but it works with facebook status styles as well.
Heartbeat negative point is complexibility (meaning you need to see examples for custom stuff) at first sight.
Heartbeat positive point is flexibility (you alter everything really) and merged activity system.

regards,
Stalski

@Stalski, Heya, I am happy

amitaibu's picture

@Stalski,
Heya, I am happy you raise those issues. My main "problem" with Heartbeat is that it's trying to do most of the heavy lifting by itself. This is a design decision in the core of Heartbeat, so I can't really point to a simple solution.

Message module is taking a different path - it says, we've got CTools and Views (and in D7 entity API) - so let them do all the hard work.

I think, and encourage you to take a look at Message (I'm now days working on porting it to D7, and fago is also going to help). Maybe if you'll like the concept of the module we can join forces. I think you'll find Message pretty flexible and easily extendable (e.g. this is all the code needed to integrate with OG):

<?php
if (module_exists('og')) {
 
// Message plugin deceleration.
 
$plugin = array(
   
'realm' => 'og',
   
'title' => t('Organic groups realm'),
   
'description' => t("Determine access by a user's membership in a group."),
  );
}

/**
* Get all groups a user belongs to.
*/
function message_plugin_og_accessible_ids($account) {
  return !empty(
$account->og_groups) ? array_keys($account->og_groups) : array();
}
?>

here's a blog post I wrote to show as an example Github like functionality -- http://gizra.com/content/message-example-github-news-feed

Join forces is needed

Stalski's picture

@Amitaibu: i would very much like to join forces. However, i have a few opinions on it as well. I also think - as you noticed that heartbeat covers a lot and adds new community features - my expertise will be handy as i work more than two years on this.
I think you can compare activity with heartbeat and everyone can decide on their own what fits their needs best. As activity is the oldest module, than heartbeat, then message, it is kind of obvious that that module needs to rules the rest for drupal7. I am always for collaboration and joining forces.
I asked the people of Acquia why the chose heartbeat over message and activity and i liked what i've read there. I can send you the mail if you like with detailed discussion they had intternal.
Besides, even that little code you showed is not needed and works out of the box in heartbeat as og does the hard work. I only provide a couple of specific stream with their configuration to achieve groups context, group context and member of a group context (all feature requests through time).

I will try to ping you on IRC since it's been months now that i wanted to talk to other people about the community activity. AT the moment, every maintainer of communities lives on his own island and i think we should improve that.
We all now the great modules that are out there: ctools, rules, og, ... but we still need to make choices on "connected to each other part". I think heartbeat has the best way since it integrates with the community features (using UR or flag_friend) where message separated this into it's own follow-us feature, making it unusable to most communities.
So here heartbeat uses out of the box friendslist notion and message wrote its own version.

But like i said, maybe a discussion on IRC with you and fago (which i consider a guru as well) will improve the start communication. I have also set-up another module "apiframework" which can help us get around ugly checks.

Sidenote: this text assumes we leave d6 for what it is, and collaborate for d7. Heartbeat is already ported in an early stage and uses entities with display of extra fields. This works like a charm. Exportables are not in heartbeat core anymore but taking over by ctools (and features for the rest). So i think that all activity modules would end up going the same direction in a d7 environment. Fields are everywhere ;)

even

nithinkolekar's picture

even https://www.drupal.org/project/openlucius distribution (updated 11 Jan 2017) is actively using heartbeat module even though its code base is very old.
Curious what makes them to choose Heartbeat over message :)

Besides, even that little

amitaibu's picture

Besides, even that little code you showed is not needed and works out of the box in heartbeat as og does the hard work

Can you explain how you accomplish this?

node_access is the one

Stalski's picture

node_access is the one checker itself, using the normal realms. To make sure heartbeat does not have to node_load for this, i give a dummy node object that node_access needs.
The og-activity module only adds tokens i did not find in og core and creates streams. A stream in heartbeat is nothing more than a querybuilder (you can extend existing ones or create a complete new one). But out of the box, it should already work. The streams created by og_activity are "activity within all groups", "activity within a group" or "activity performed by members of a given group".
Even further there is also a connected activity stream that used "the friends of the actor", which again heartbeat has to calculated itself :(. Here there is a hook that can be implemented to create the array of uids for a user. So in a project, i needed users that were connected in the following way "friends, followers and even if a user is in a group where i am member as well ... he's connected".

The only thing that is very difficult in heartbeat is views. Views does a query and knows then how much messages will be shown. Heartbeat does not. This is because there messages can be restricted afterwards or even use the great feature 'merge similar activity'.

So what i am trying to explain here, is that heartbeat itself is not so complex but integration everything to use the features of other community modules, makes it hard to understand. I think friendlist modules explain it all (see APIFramework for a sollution).

we'll take these conversations further at IRC with the people we know, want to be involved. So i was thinking, you, me, fago, and maybe the activity maintainers that are willing? I will try to make a blog post, explaining most basic features in heartbeat for D7.
thx for your quick and constructive replies.

Let me bump into..

ilo's picture

the conversation, and explain my opinion.

Recently we have been working in a 'custom' notification system for something close to the Heartbeat/Activity/Message functionality, but the project required not only this activity stream to be notified, also other types of messages. So, we built a little custom message push and storage system to store all the messages being translatable, and including a 'realm' field that could be used later to filter what messages should be shown. We create our _push() message to insert messages in the system and modules implementing the push hook could keep track of them, altering, firing additional actions - or rules - and turning this notification system into something that could be Access Controlled. Actually, all this is the 'prosa' stuff we sell to the customer, but we just copied/renamed/put-some-love to a copy of the core's dblog module wich is exactly what we need for this as basic functionality to work.

And it is not so different in functionality from amitaibu's Message module from what I can see and does not overhaul the module system nor requirements. Tt does not require ctools also, but you can still use views to create your view query if you need to show the notices in a more fancy way than using the -default-built-in- table view (also copied from dblog). We found that the module is working fine, being integrated with growl module to send notifications usin growl desktop applications (as a hook implementation). This system is working as 'site activity' stream provider including also notices, errors and whatever you want to push on the notification table when you have a complete action-rule set. This system builds a full site-activity stream and is able to include private or global messages as part of this stream.

What I'm seeing here is that now Message module (a generic name) is going to be used to build user/group/node/comment.. activity streams. Perhaps the name may confuse at first sight because one would expect to find 'Message' to be focused on messages or part of a message system in Drupal, but this could be solved with a good module description. Backward search would be more difficult because when I type activity stream in the Drupal's search box I'm expecting to find modules not related to messages.

Heartbeat does its purpose, we used it for a while and had no problems at all. It is not an overkilling module. Message is in the list, still have to test it, I'll update the comment when I do have more feeling of the module, just downloaded right now.

nice

Stalski's picture

Nice, however, again, why not join forces.
Heartbeat is designed to do exactly as you describe. I will give a short indication on what i think we need to ease the process.

Let's think out of the box (since installing heartbeat, activity and message will hopefully push basic messages into the streams, node, comment, account, terms, ... )
Developers need more than an hour to understand heartbeat, which i don't like. So if i can improve that, whiiiii
Implementers mostly don't understand heartbeat, and if i take the issue with me, than lots of them come from the fact that they did not read how to use the rules UI. So telling them to rtfm sort of speak, fixes the problem. The next part that is very difficult is that heartbeat takes on all permissions checks facebook streams have, with lot's of code that is difficult to capture. i am trying very hard to fix this in D7.

The other part i'll have to work on a lot, is providing a screencast video and a real good one-page-tuturial for heartbeat. I did lot's of post, code examples, demo site, but i realize more and more that the information is scattered, due to my good intentions :p
So what you say above is true and funds my statement. The reason why you did not use heartbeat and created a custom module is bad for heartbeat. It means that i did not accomplish what i wanted: make it accessible to everyone.

more on this to come.

So what you say above is true

ilo's picture

So what you say above is true and funds my statement.

And this is the reson why I commented, Stalski.

The reason why you did not use heartbeat and created a custom module is bad for heartbeat. It means that i did not accomplish what i wanted: make it accessible to everyone.

No it is not, it was a custom requirement. Actually, Message is more the way the customer wanted to go, but I've had hard times trying to explain him why views+ctools in size is as big as Drupal core, but this is out of the scope of this issue, and Heartbeat looks for the customer a 'distracting' stream, where site activity is the minor usage. The customer thought that it would take more time to make heartbeat streaming other notification messages (non-related to site-users activity) than creating a custom module and include some site-actitivy events.

So just to settle my opinion, a push/pop message and storage system is required, call it activity or message or whatever, and a customization to discard non-site-users-related-activity is where Heartbeat must limit its scope. Heartbeat must not be responsible of creating/storing the messages, but to show them in a way activity streams makes sense. The same happens with Message module, it is very 'generic' in scope, so it must not focus on providing activity streams, because it is a very specific use case, letting other more specific modules provide these streams: activity, development, security, and so on. Seems that instead of joining forces, you two (heartbeat and message) should split scopes, IMO.

Message module, it is very

amitaibu's picture

Message module, it is very 'generic' in scope, so it must not focus on providing activity streams

But it does it through Views and Views filters that provide the access control (see the message_example module, or the blog post I've mentioned above).

you two (heartbeat and message) should split scopes, IMO.

That might happen if we won't agree on design decisions, but we should at least find out first. As I stated before, Heartbeat is doing all the heavy lifting for itself and provides more functionality out of the box (e.g. a page to see the streams). Message is just an API that is exposes it's data to Views, assuming that people already know how to deal and theme Views, and don't want to learn a new system.

Agreed

Stalski's picture

Certainly in d7, I tried to meet the people's desire of : "what can I do to make heartbeat streams look (exactly) the same as facebook". I heard that so many times I chose that road.
Amitai is correct, we should talk api and design.

First of all:
- I am trying to make heartbeat activity a full entity (not yet on entity dependant, sorry fago ;) ).
- Then I 'll check message for d7 again and see how we can work out a solution for iew vs non-views. The reason here is that what call in heartbeat "attachments" are harder to load. But the whole concept could change into fields and extra fields.
- The display suite approach I would like to keep
- Work out which module it's going to be (message or heartbeat).
- Talking to ezra-g since he wants to collaborate with activity_log and activity as well. We'll see if that's a bridge to far or not.

More on this later.

The question is then,

ilo's picture

The question is then, Amitaibu, what is Message module going to be? 'just an API' or an alternative to activity stream modules? I can't see the point of the module, where is the end. Will it just create and store messages? or are you interested in providing the use cases also (in the case of this issue, the site activity stream? And I don't mean what it can do, because dblog is already an 'activity stream', very poor, but it is.

@ilo, 'just an API' or an

amitaibu's picture

@ilo,

'just an API' or an alternative to activity stream modules?

It's an alternative built as an API :) The Github example shows how to leverage the API to create meaningful, translatable and access aware messages that can be shown via views.

Example modules are not ready

ilo's picture

Example modules are not ready for final use, no matter how far they reach in concept terms. I think the question is not yet answered, but I'll try to keep on, because afaik, heartbeat is already focused in activity streams, while Message tries to cover too much space I'd say. Definitelly you should try to merge efforts and split scopes so there is a chance to work in the same 'project' line.

Thanks for your comments, Amitaibu, Stalski.

Example modules are not ready

amitaibu's picture

Example modules are not ready for final use, no matter how far they reach in concept terms.
while Message tries to cover too much space I'd say.

I disagree. Views is an API allowing users/ implemeting modules to leverage it's API. Message is trying the same (only on a much smaller scale) - allow users or implementing modules to create a message. IMO, It's a pretty focused definition.

Oh, I agree in and I love

ilo's picture

Oh, I agree in and I love this approach, because Views will never be an 'Archive', a 'Most viewed Posts' or a 'blog', it is just an API that other modules use to build their functionality on top of it. So, in this case, Message is just the API requiring additional modules to provide the functionality, a 'Site activity Stream' module, call it heartbeat, call it 'Site Activity' or whatever you want to. Is that what you mean?

The question that I'm asking is, will Message be a replacement for 'Heartbeat'? and by 'will' I don't mean 'can be', I mean, will such a module using that API exist apart from an example module of what can be done? That is a key question, because it may help to understand what is the final shape of Message as module: provide an API, provide 'Site Activity' or 'Friend related' streams, or both..

I hope you don't dissagree that 'examples modules are not ready for final use'. :)

If I can butt in

kyle_mathews's picture

It seems like ilo is saying that there should be a "features" module like the message_example module but more polished so that people who want a really nice activity-stream setup can get one without having to figure out how to use an API-type module which while powerful, is beyond the capabilities of most people to setup.

And I agree for fwiw. I'm planning on using Message for my Eduglu distro as I like very much that it's just an API and will let me build exactly what I want. But for many more casual users of the module, that'd be too much work and they'd like something more plug-n-play.

Kyle Mathews