I'm currently planning the rule based custom logging module, which I'm going to write after rules 1.0 is out. I post this to the activity group too, because I think there is quite a overlap to 2.x version, which is currently under development.
There is already a rule based custom logging module for 5.x, it's called "per entity logging" and ships with workflow-ng. It builds already upon views for the display part and one can already use it for logging activity messages too. For 6.x I want to create a separate project and implement some improvements, like multi-lingual messages, a more modular design and so support for possible arbitrary entities. More on the details later, but yes it would support logging activities too, as well as everything else one wants to log.
So as you can see, I'm really in favor of using an action for logging messages. Activity 2.x goes this way too, but builds upon the trigger module. Unfortunately this requires modules to still implement a hook so that their hooks are compatibly with activity. If you would build upon rules, this wouldn't be necessary and any supported event could be used. Apart from that you could easily provide default rules... :)
I thought a bit about the differences between activity 2.x and the module I plan. Apart from working with rules I think the biggest difference is that my planned modules is going to handle messages related to any entity (nodes, comments, users,..) not only users - but it would handle the "user case" fine too.
So what do you think about this? What about joining forces and doing together the "rule-based custom logging module", or however we like to call it. We could also do an "activity log" module built upon it, which provides further needed features or just some useful default configurations. What do you think?
Note: I'm going to post a concrete implementation draft soon.
Comments
Draft for the implementation plan
Database tables:
I'd propose two tables, one for logging the actual events and one for storing further infos per message.
Table for logging event messages:
(message_id, date, uid, entity_id, variables)
where variables is a serialized array containing the values for the replacements.
Table for storing message infos:
(message_id, msg_type, msg_category, entity_type, text)
where msg_type could be 'activity', category 'Profile', entity_type 'user' and text the actual log message.
Having two tables makes sure the messages are stored in a normalized way, which reduces storage overhead and makes it easy to change things afterward, e.g. change the message category or text.
Display:
Build upon views.
For the default output, one could just pass the message text and the variables to tt() of the i18n module or just replace the replacements.
Supported entities:
Basically all what's need is a mapping from views2 base tables to drupal data types. Probably this mapping is added to rules as it's useful also for other stuff (vbo, ...). So for entity to be supported only views integration and a mapping to rules would be required. At least users and nodes should be supported out of the box - best also comments.
Hooks:
The module should offer a hook for extending the configuration form of the action. For changing the display it probably suffices for the module to just add another view formatter. Also add a hook before actually storing the message to the database, so modules could detect duplicates or so.
So the provided hooks should make it possible for a module to add further different message texts based upon certain criteria (e.g. user roles), add per-message access control, or detect multiple occurring events.
So I have taken a first pass
So I have taken a first pass at looking at this. I will require a less foggy brain before commenting more intelligently.
But doing translation and token replacement at the View layer is very slow. Really really slow. I would prefer not to save the variables into a serialized column. Instead the table for storing message infos should have a primary key of (message_id, language). So what you do on message creation is to log away all the possible languages at the same time. This is much faster then logging away the variables and doing the replacement later. Furthermore, storing the node, user or what have you for the token replacement is really inefficient.
Whats not mentioned in here, and what I built for Activity 2, is the access control system. How do you push logged messages to another user, and how do you prevent a log message from showing that the user doesn't have access too? There are two distinct problems around access control.
I think a lesson from Activity 1.0 is that you need to plan access control right away.
But doing translation and
I think you got this wrong. I don't think of storing the whole node or whatever in the variables column, but the strings ready to replace as you usually pass it to t(), e.g. array('!link' => 'node/4'); So on display only simple replacements are needed, but firing up token is not.
Storing the ready translated messages would be another possible way to go, but I think in the end tt() does this for us, and so we don't need to reinvent the wheel.
Yep, that's true. I have thought a little bit about that and I think for the purposes I planned the module an access system isn't needed necessarily - but having one as extension might make sense. However I'm not sure whether the db_rewrite_sql() approach is the right one (as written below).
token and t()
After some more discussion about translation I'd like to give a synopsis of our plan here.
Token should not be required, but it should be supported. The majority of users need this sort of flexibility in crafting their messages. Token replacement should also be done on the recording side, not display. This is how it is currently implemented.
t() function should be used on display as it is lighter and would provide translation. This will require a little extra work; adding another column to {activity_messages} and making sure t() is used in all of the appropriate places.
So in theory, both should be used, and we could possibly handle a message like this:
[user-link] updated %title.
Which would be recorded in {activity_messages} as:
user name updated %title. | serialize(array('%title' => 'node title'))
yep
This is exactly how I planned the "variables" thing.
However note, that one shouldn't run t() on user-provided strings. For this one should use tt() provided by i18n. Then your "user name" needs to be also such a replacement variable, as else one would have to translate the message once for each occuring user name... ;)
Well I'm glad we agree on
Well I'm glad we agree on the fact that actions should be used to log activity messages and the fact that Views should be used to display them.
When it comes to an Activity vs. Rules line of thinking, I thought that Rules could implement any action within a system. Therefor, Rules should be able to work with the action that Activity provides, correct? I tried this out and it didn't seem to work, so though Rules proposes that it can work with any action, in fact, it seems cannot. Please correct me here if I'm wrong as I'd like them to be compatible.
I guess overall, I would really like to see the approach Activity is taking be developed further and make so that it is compatible with Rules rather then have another separate project out there.
Activity also has a means of allowing other modules which define relationships provide that as criteria when it comes to the display of activities. So out of the box you can record and display whatever you want, but as soon as there is a 'relationship' context on the same system, Activity can then behave within that context, showing only the activity within that relationship context. I'm not sure how you intend to handle such a thing with your approach, or for that matter why you should have to if Activity already does so.
Basically I don't want to give up my end of the rope here as I think Activity has a lot to offer already that does not need to be rethought and I think it would be more beneficial if you could help us make sure that Activity can work with Rules. Activity's action is not only based around users but can be based on other entities as well.
The fact that you can have default Rules is a huge plus, and I wish Trigger had something similar, but one of my key goals in this project is to have it work out of the box with core modules and not have too many dependencies. We already depend on token module, but I hear that is a likely candidate for core at some point. The Trigger module has some obvious drawbacks, but it's in core and therefor, though limiting, I feel is important to work with at first to provide basic functionality. Additional functionality can perhaps be handled by recommending people install Rules in order to do X, Y, and Z.
Please re-think building an entirely new contrib module and instead consider helping us out with Activity 2.x
thanks for the fast reply
thanks for the fast reply :)
Rules has core action support, more exact the support depends on the type of the action. Node and user actions are supported out of the box, for other action types one has to tell rules how to deal with them - so for activity. Have a look at the documentation for it here: http://drupal.org/node/299055
Hm, what would be such a relationship? I don't think I understand this right. Is it more than a way to categorize the activities?
I can understand that.
However I'm not sure about doing it the other way round, basing the rules modules on activity. My module is not only intented for activities, it's also intended as general purpose logging system for any entity - e.g. also for a workflow related log for nodes. So how would we build multi-entity support into activty? How do we make it translatable? I think this would require some drastic changes. Then most of the current code doesn't help me much, as its specific to triggers. :(
Not related that, I'm not sure if the activity access like that is a good idea. It hasn't proven for nodes, so I'm not sure if it's a good idea to mimic that.
Yeah, that's the way to go :)
access
Hm, thinking more about it, it might be the best way to go though. But what's the actual use-case for it? Is it really needed? I wonder about that, because it adds quite some complexity.
friends
Social networks are big users of Activity module, usually trying to replicate the 'news' feed of Facebook, or as a general way of keeping up-to-date with what your friends are doing on the site. So in this context, which Activity is originally based upon, access control to these feeds based on those with whom you have a relationship is paramount.
Here is an issue in which the mechanism itself is described in greater detail.
That being said, if another module does not implement these hooks, and you have permission to view user profiles, then you basically have access to everything Activity has recorded for that user.
For getting the activities
For getting the activities of all my friends, a view (with relationships of a buddy module) should be fine.
So it's probably only about viewing profiles of other users, which contain their activities. I think one could just use a separate view for that, so you can easily filter out activities you don't want to list there. So it's up to the user, which activities he lists where (by using views).
Ya we actually talked about
Ya we actually talked about doing this. I seem to remember the one issue I had with that is how do you do OG postings? Have to write a relationship for that but that causes another issue as I think relationships are "AND" 'd together, therefore creating other issues. And you would have to JOIN together two tables, thus actually making it less efficient then the UNION'd OR query we currently generate.
Perhaps a quick look at how Views handles relationships warrants a glance, but pretty certain thats how it works.I don't think it warrants another look anymore more. I originally wrote it but Im still haven't had my morning coffee. Someone else can but I don't think Im am going too. Even it doesn't necessarly AND'd them together, the relationship building can get unwieldy.
@group messages: One could
@group messages:
One could log the messages directly for the group node and then just show them to members of the group. So one could list them in a separate view easily. I don't know for sure if it's possible within one view, but I think views2 provides ways to do that too.
So you are going to actually store the message X times, where X is the number of users who should see it? Hm, I think this would create quite some amounts of duplicate logged messages.
I don't think that joining tables is so inefficient as long it uses only indexed columns and we do not need more sophisticated stuff like sorting over two columns living in two different tables. Moreover you save a lot of headache, as you don't have to care about the messages from a user when creates/looses a friendship or joins/leaves a group.
wait, you should read the
wait, you should read the link sirkitree posted. Cause all those things you just said, arn't true in activity.
we don't store the message X times, we store it once
We don't need to keep track of who creates/looses a friendship or group at all either. Really, i promise we arn't that silly.
edit: group and friend activity messages are possible in one view using activity. I know cause Im looking at it ;)
we don't store the message
oh, I see. Now I think I understand what your access system is really about. Thanks for all the explanation .. :)
I see now, that the access system allows relating the messages to arbitrary entites too. Interesting!
;)
I wasn't able to see og integration code in the repo? Where can I find that? From where does the og integration module know which messages belong to it?
Like I was planning it, one would select the entity to log for, when the action is configured. So the user can log for, whatever he wants. While I thought about doing only views relationships for pulling messages out, for activities I see that your system makes perfectly sense!
Idea: Have you thought about integrating your system with the help of a (custom written) views filter? So you could use the views query object for modifying the query instead of relying on db_rewrite_sql(). If you plan to demand views for display, I think there is no need to go with the regexes...
Ya perhaps the Views filter
Ya perhaps the Views filter might make sense. We could even go as far as Spaces goes with its Views Filter. But going this route, we arn't tied down to Views. We can create a page callback without the use of Views. Maybe?
RE: OG Integration
Sirkitree and myself use a custom group implementation. But the idea is very much the same thing. OG integration goes like this
<?php
//Implementation of hook_activity_grants().
function og_activity_activity_grants($activity) {
if (!empty($activity->nid) {
return array_keys(og_group_posted_in($activity->nid);
}
return array();
}
// Implemenation of hook_activity_access_grants()
function og_activity_activity_access_grants($account) {
return array_keys(og_get_user_groups($account));
}
?>
yep. Ya perhaps the Views
yep.
Yep, that's the question. Whether this possibility needs to be there.
@og: I see, thanks.
sirkitree and i talked about
sirkitree and i talked about this yesterday and I think thats the way we are going to go. The filter will have an options_form with checkboxes for each activity_grant module (friends, og etc) so your able to configure the View. You would also be able to say, "Everyones activities".
The db_rewrite_sql() will stay though as it adds in the node_access stuff as well, preventing private nodes and comments on private nodes from surfacing.
Just need to slow down Earth's rotation to add in a few more hours in the day :-D
sirkitree and i talked
cool!
I wondered if it's necessary to store all the ids in the access table again. Apart from duplicating things you might ran into synchronization issues - what if the author changes a post to don't be in the group any more?
So I tested the performance of some (multiple) joins on quite big tables - they are pretty fast as long as you join only two tables per column (and we won't need that). Also I tried two optional joins, where at least one is required - it's fast :) So instead of letting modules return access grants, they could just alter the query object directly and add their (optional) join - but of course this would be views only.
Re: performance Performance
Re: performance
Performance has little to do with the number of columns to tables. Usually the bottlenecks are not having indices for the right columns. And it this case Im not worried. If this code was deployed on MySQL 4.x it would be slower, as ORs aren't turned into UNIONS like they are in 5.x by the optimizer
Removing the access table will make the SQL to complex. And as you noted, complex SQL will break node_access() which we are trying hard to make use of (never show activity on nodes the user doesn't have access to). And it also removes the ability to really make some cool access stuff. You could right a small module that broadcasts all of user 1's content to all users really really easily.
Not related that, I'm not
was hoping you were going to explain that a lil bit. Cause comments like this always make me roll my eyes.
Im all for a better access control system, but no1 has pointed me to one that works. I haven't found a system that works better then the current object based system. Im all for changing it, and I have looked but there is nothing that I can find.
And Im not sure how its not proven for nodes. I know its a silly oft-the-cuft statement, but that always irks me.
Even this two page article from xaprb (one of the authors of High Performance MySQL) implements his system just like ours, with the exception of he uses UNIX style bit operations, where we use simple columns. And he says
So while I know you didn't mean something negative, I hope I can change your mind about the node_access and other access implementations. It is with high probability the best implementation out there.
If you have an idea, I would love to explore it though. Most of the time, my ideas become just like the existing system just using different words.
was hoping you were going
As I already noted above, I can't think of a better solution for it and it's probably the way to go, if one wants to implement access control per message.
The problem is that db_rewrite_sql() isn't clean, it does query rewriting by regular expressions, which sometimes results in troubles - in particular if queries are getting more complicated. For d7 the new db API is going to help us here.. :)
I have started working on
I have started working on message module (currently on github). It's based on the idea of realms, that you can assign the message to. So if you assign it to a group realm, only members of that group will be able to see it. Currently updated user, role, og and og_admin realm. And it's very easy to extend it, using CTools plugins.
The module has a dependency on CTools, which means it's pretty slim, and it's exportable by Features!
It also has an integration with i18n, so one can translate their messages, and of course to Views (but not as a dependency). It's still on early stages, but the core is already working.
I plan to add rules integration, so you can set up actions (and export them using features), and a message_ui to allow adding message templates via UI.
I'm Currently using
I'm Currently using heartbeat, what's the advantage of message over heartbeat?
Interesting stuff. Couple
Interesting stuff. Couple quick comments.
I would make sure message templates are exportable, its one the things I wish Activity had.
Thanks for the comments
Thanks for the comments guys:
Heartbeat in its concept is closer to message, than activity is. I have looked at it's code and played with it, but I indeed need to learn it better to tell all differences. Message is built on CTools which brings a lot of good stuff and reduces lot of the complexity I see in hertbeat.
There isn't really the notion of "activity", just a message which is related to a UID and possibly to an entity. So basically you can have many message templates and trigger them on a single event.
Messge's API and crud is mostly copy/ paste from the great Context module, which has all those caching. But, I'll go over it once again once I'll get some other things done in the module.
that's interesting -- can you explain more. I myself (as you probably see from my english :P) am not a English native speaker, and I can't think of "non-translatble" message.
They already are! You can alreay create a Feature and export the message :) That's one of the most important features currently in the module.
that's interesting -- can you
I believe this was the one issue, and its still confusing to me. http://drupal.org/node/519356 . What I took away from that issue is that the pronouns are difficult.
But looking at it now, I remember the real rational. With Activity2, we decided that the database would hold the fully tokenized message (i.e. run token_replace and store that message into the database). These means the message itself has things like node title which are variable. So running t() over them becomes impossible. tt() though needs to be able to take the message and return the translation. So it has to have a mapping "Scott created blog Fun stuff" -> SOME MESSAGE IN THE USERS LANGUAGE right? So then that means that for every message, a translation needs to be CREATED and stored.
With Activity2, the message template is created with multiple languages and then with token this means the pre-tokenized message is translated for you therefore the stored message will be translated. This means that the translated message is then just stored and removes the need to create translations.
And it also means that after querying the database for all the messages, each message doesn't have to be processed by a translation function which means its faster.
Here is the issue for the translatable messages: http://drupal.org/node/493102
@translation: From what I can
@translation:
From what I can see the module just translates full sentences like "%author has created a node." - which is fine also for languages like German (I speak it).
a translation needs to be
Still, I think translating once is better then duplicating the message.
Indeed, it is faster. However, it's less flexible, and this is the case I think, that speed is second - otherwise, fast as it may, it didn't serve my needs. Actually, Message now comes in two modes - one is a plain text replacement, and the other is using callback functions. so you can use for example
url()to return an aliased URL instead of hardcoding the href.I have released the module in d.o -- http://drupal.org/project/message -- you can enable the example message to see how it works (ain't it fun that it's all exportable?) :)
Awesome!
I've not tested it yet, but this all looks really awesome. Great work!
Though I'm not sure whether I got the 'realm' concept right. Is it different than a "tag" for a message you can use for filtering?
Does the module support multiple different messages depending on who created the event or would one have to log it multiple times? (You have post a node. vs fago created a node)?
I've recently talked to klausi how that would look like in D7. I think there messages should be fieldable entities. While it might feel strange in the first place, there is no downside from having an entity, but you are able to add any field on it as you like, maybe even based upon message type. Flexibility++!
this allows multiple messages
I have thought about it some more, nd I'll dd an "extra_identfier" to the {message_instance}, so one we'll be able to act on messages regarding the same event.