Hey everybody!
I thought it was long past time to make the first post. I've just backported a new bug fix regarding node revisions, and I took the time to go back and look at some of the newer issues.
There are quite a few patches to HEAD that need more testing from people in this group so that we can mark them as RTBC and get them committed.
Once those outstanding bug fixes are there, then I think we have a more than stable enough version of og2list that we could make a 5.x-1.0 release of it at last.
Beyond that, I know that I have a boat load of features to backport from our private og2list version, and they are all going to require help testing from everyone.
There is also some interesting work going on about multi site configurations at http://drupal.org/node/68408 that is going to need testing.
If anyone is lacking a test environment or has been thinking about setting it up, now is the time!!
Onwards to a release!
D.
Comments
Where is it?
Where is it to install an test?
Check this discussion
Check this discussion here.
Best.
actually
most of that stuff is over a year olde and has been implemented in Post Carbon's private version.
what we need to get tested is the version that is in CVS. It is a Drupal 5 port of the old 4.7 og2list with bug fixes.
Creating a test environment for og2list is no trivial thing however, so I've come up with an idea that may help with patch testing. I am going to stage two og2list sites on the Post Carbon development server, an og2list-before and an og2list-after. The idea will be that we can use it for testing patches one by one - I can appliy them to the after and then the difference in behaviour should be clear.
I'll post the links to these sites when I get them up and running.
Or, anyone who has a local or test instance of og2list running can help test.
To download the latest version, visit the project download page for og2list and then choose "View All Releases". That will let you download CVS HEAD which is the current version.
Then there are some patches that need to be looked at that are waiting in the project's issue queue.
After we can get a 5.x-1.0 out, I can then look at backporting and/or somehow getting into CVS all of the features that we have added in the Post carbon private version. The list is a couple pages long.... and we are finally almost at a production roll out in our testing. Once I've rolled our private version out and fixed any problems then it will be time to get those features back into the public domain. But first we have to even get a tested 5.x release out.
I'll post back again when the test domains are ready, and pick a patch to start off the testing with.
D.
Let's do this?
I've messed with mailhandler and listhandler for a 4.7 site, and now want real organic groups integrated e-mail lists as soon as possible. I can jump in with a bit of work and/or money because we'll be needing this for upcoming projects also. (Long term we need to write our own modular mailing list software, I think.) For now and og2list, please point me at the code and give me some words of advice or encouragement.
Here's the immediate request from World Social Forum 2008:
So, looking for that promised update and. one way or another, to get something like this operating.
Thanks Damien and anyone else who can provide input!
benjamin, Agaric Design Collective
benjamin, agaric
let's do it!!! - 3 x og2list demo/test sites coming today!
Happy Holidays everyone!
@Benmajin,
are you going to be doing this in 4.7 or 5.x? The Postcarbon sponsored 5.x version is SO nearly ready.. and I mean SO nearly ready! We are preparing to roll it out jan 1st to more production groups... all major bugs seem to be fixed or bearable anyway.
I am going to have some time to spend on og2list around new years.
However, I will try and bring our three demo instances to life on the postcarbon beta server today. We will have:
1) a CVS HEAD version, before a patch
2) a CVS HEAD version, after a patch
... these two can be used so that people in this group can help with testing patches, without having to have a full blown working og2list test environment. I hope to use these two to get the rest of the essential fixes in the patch queue committed to the 5.x-head branch.
then i want to see what's in CVS + the oustanding fixes (there are a couple that need testing and I still have to backport my fix for the node revisions issue)
3) the postcarbon private version - for demonstration purposes
I will use this one to demo the enhancements we put into our version. Hopefully this can become the 5.x-2.0 branch. Otherwise it will probably be privately available on an unofficial basis and it will be up to people on here and/or the drupal.org community to backport whatever features they want and go through the process of atomically patching whatever is in HEAD.
All three of these will be on public test sites that you guys can sign up to for testing and play.
Once we roll out Postcarbon's version to a few more postcarbon groups and/or do some testing with you guys on my demo #3, we can look at releasing that private version to those that need/want it (or hopefully even better getting it into CVS as 5.x-2.x).
Going forward, we have determined a lot of things we'd like to change about og2list... to the point where we feel the best way forward is a fairly complete rewrite for Drupal 6, casting out alot that doesn't work or is no longer needed, and keeping all the good working code (which was a lot more sweat and tears figuring out exactly what works than number of lines).
We'd like to involve interested parties in that and do it the proper drupal.org way. Here are some of the ideas we've come up with as lessons learned from the past year of hacking on the 4.7 version to get where we are. I think any of these would lend themselves well to being attacked individually by a person or team of persons in the usual patch-test-revise-release cycle. However first we need the stable and production ready base to work on (the PCI private version)
1) outgoing mail should be moved into drupal. mail-out.pl is inefficient, confusing, and makes it very hard to do good formatting on outgoing e-mails (such as: customized footer links on each email, proper rendering and node revisions handling using the API, fancy digests with linked/themed everthing like yahoo groups provides, customized per-user digest schedules like one user wanting digests hourly and one wanting them weekly). mail-out.pl does provide integration with multilple mail transports, but we do not feel that functionality is even required really given that Drupal already does plenty of outgoing smtp mail and works just fine. Mail can still be batched as it is in og to not overload cron jobs. Or, outgoing mail-out could be handled by a different mail handler, but that should be some external module and not part of og2list.
2) the dependency on og_forum should not be required. Right now the only thing og2list requires og_forum for is creating new posts somewhere - i.e. if a user mails the list with the first post in a thread, then og_forum is used to make the thread a group forum thread. Other than this, our og2list already cleanly supports any content type that drupal does, including CCK. Rather than depending on forums, the admin should be able to choose any content type they want, even on a per-group basis (see below). Those using og_forum would certainly be able to use a forum type, which should work properly with the newer og_forum releases for 5.x.
3) All og2list options should have a master override in admin/og/og2list and ALSO be group-by-group settable. Currently, most configuration settings are global for all groups.
4) og2list needs decent user instructions/help somewhere. Right now, to create a mailing list you have to edit/create a group and add the Mailing List Recepient field (buried away with all the other node authoring form fields) and then the mailing list tab appears on your group and you can edit more settings there. It is NOT intuitive. Basic functions and tips and hints for group managers should also be available somehow... important little tips like "Don't post the group list address in a node, because spammers will spider it and start hammering on the list address". There are several more of these little gems we have determined as a result of our production testing.
5) UI cleanup - the user interface is inconsistent - all the fields in mailing list should be part of the node authoring form really, with og2list control settings in a seperate tab as appropriate and consistent with other drupal modules.
6) Remove old / unused code - there are hundreds if not over a thousand lines of weird code that is doing something with commenting or printing links for publsh/unpublish/spam. I found these almost entirely by accident, they are about 3/4 of the way through the module source code. These look like functions borrowed from some other 4.7 module to provide some sort of UI functionality - which I'm not sure should even be og2list's responsibility??? It needs more investigation and testing to see if they are even running, or needed, or need updating. I will provide more details on this later, would love for someone to tell us what if anything these are doing.
.. anyway there is more I'm sure, but that's a nice starter list off the top of my head. I am all for fixing these problems going forward by committee... I am not so hot on the idea of going back and trying to bust a year of work into atomic patches.... mainly because we already did a boat load of testing with real users that will just have to be repeated, and so far we haven't got a lot of movement on testing / comitting patches in the queue because it's so hard to make an og2list test environment.
So hopefully once you all see some of the demo in action you will agree that it's stable enough to just become a -2.0 version (it needs some cleanup first but the first step is your feedback). Then we can spend our effort fixing these problems and moving forward for a really cool d6+ version. Meanwhile, people can use 2.x in production, or people with really simple needs (like a small internal intranet) could also use 1.x in production. Finally, those really insistent on doing it the atomic way could backport stuff as they wanted from 2.x towards a 1.1 or whatever. I'm fine with that as well and can provide help/support for anyone doing it.
another route we might end up going if we can't get the 2.x version back in, is stabilize HEAD to make that 1.0 release and Postcarbon just releases their version privately,and it's up to people here to backport what they want while work begins on a drupal 6 version trough the project on drupal.org. However that seems kind of silly, at least to me :)
So - I will try and set those versions up today, the DNS entries are all ready, I just have to make some drupal site instances and hook it all up and test it. I'll post again here with the location of the demo list, if everyone can join and sign up to that then in a few days once everyone's on it I'll start to explain and show off some of the new features on that list, then summarize the features here with links to the posts.
Wow Damien! Awesome
I need this. World Social Forum 2008 needs this. Agaric Design needs this. People Who Give a Damn needs this and a whole lot more (democratically moderated mailing list). So I'm saying I will absolutely test and help maintain the Drupal.org project.
Your proposals look very solid, although I had assumed that mail-out.pl was created for efficiency reasons.
Isn't Drupal 5 pretty much wide open? There is no available release tagged with 5. In any case I request you be made a maintainer of og2list in order to manage the 2.x version.
Also for the long-term goals, it may be useful to look at the mailing list proposal by Jamie McClelland of May First People Link which can work as sort of a mailing list API: these are the pieces OG2List and other Drupal solutions should work with, which can then be replaced with PHP solutions as proposed by Jamie or other tools.
Very much looking forward to getting a look at the results of your blood, sweat, and tears!
Thanks!
benjamin, Agaric Design Collective
benjamin, agaric
COOL! We have eagerly been waiting for og2list.
Getting og2list working again will be a big help to my client, Partners In Health. When og2list stopped working with Drupal 5, we switched to using Google Groups. Surprise! Google Groups has a storage limit! The group got filled up with uploaded files and there is no way to buy more storage. We need og2list!
I think og2list is potentially one of the most significant Drupal modules, and thanks to Damien and Postcarbon for getting it working.
Performance testing
Hello, one of the goals of OG2List was to be performant. I'll have to check with Gerhard, but I think we moved code into mail-out.pl to enable us to have 10 000 subscribers. This could not be done in Drupal as Drupal would crawl to a halt preventing any sort of real time mailing list discussion from happening.
You should track down Gerhard and make sure you understand some of the architecture decisions before you attempt a rewrite. You might find yourself discovering paths we already implemented and had to rewrite once before.
Thanks for maintaining the module. I suspect the UI changes are probably all sensible, but could have been limited at the time by very specific implementations of OG in 4.7. Those limitations might no longer be in place in OG 5.
Cheers,
Kieran
To seek, to strive, to find, and not to yield
mail-out and peformance
Everyone - sorry, I ran out of time over the holidays to get those test instances up. They are coming though!! I'll be back at HQ tomorrow!
@Amazon,
yes I assumed there were definitely some design goals behind the choice of mail-out.pl.... I'm not out to knee-jerk remove it so much as to revisit it with proper discussion for the next Drupal 6 version. Specifically, I am not convinced there aren't better alternatives to mail-out for doing the mass mailing - surely other modules exist for this?
anyway the most frustrating issues with mail-out.pl revolve around the way it does its querying - the 2 passes to create the outgong mails, rather than getting everthing prepared while stil in Drupal. While this isn't a problem for the basic functionality, it does make it hard to add features asked for repeatedly by users (improved digesting, footer links, footer text etc etc etc).
Anyway, as outlined above I just want to get a stable version people can actually use in production for drupal 5. We can redesign and revisit some of the other areas by committee. My vision of a re-write is more of a "pluck out all the code that works really well into a better design, and leave the rest" than a "re-write the code from scratch".
one thing I definitely learned this last year is that you can't think of everything ahead of time when it comes to mailing lists! The use cases and how people actually expect it to work (and also what they -don't- find intuitive) are often quite surprising from the developer's perspective. We are lucky at post carbon in that we have lots of groups to test with, many of them with users who are non-computer savvy, so we've been hitting a lot of these use case problems... Although I'll admit I didn't always feel lucky at the time ;)
Definitely before redesigning anything I'd like to do that on the drupal.org project infrastructure, and also have feedback from people who've actually used the 5.x version, because all of you who do will have some lucky fun times of your own exploring your users' use cases!
anyway.. just a few more days patience, I promise... :) To get rolling we will get testing some of those open patches on my two test instances, just having you guys here look at them will be a huge help.
Happy new year!
D.
one more thing re: release schedule
Also, one more thing I thought of,
with Drupal 6 in RC and given a realistic view of how fast things move, the sensible thing might be to make a D6 port of what we have now and postpone redesigns/rewrites until Drupal 7.... By no means is that idea cast in stone time wise, there is no reason we can't make a d6 port of what is working if people are using D6 by that time.
D.
og2list for Drupal 6 please!...
How can I help get og2list for Drupal 6? Would cash help? Let me know...
Cheers Daniel
need a 5 version first
A 6 version is not in the cards until we even get a 5 version out. Until someone other than me helps test the outstanding patches we are not going to even have that.
I continue to try and get this done, and am actively testing and fixing bugs in my private version. I can make some of that stuff available but not until we get past where we've been stuck for the last year+.... which is that no one other than me seems to be able to do any testing?
Anyway the steps to a drupal 6 version are...
I need person(s) other than me testing the outstanding patches to get them RTBC so that HEAD can contain a stable version
we have to get official control / co-maintaining over control of the project and get a release out. Post carbon is willing to sponsor development hence forth along these lines and we have at least two developers who are familiar with the module can co-maintain - a third is getting up to speed now.
get some of the more important functionality of my private version into HEAD somehow, as a new version or as atomic patches. (I don't have much hope for the latter seeing as 3 simple patches have sat in the queue for a year with no one testing them)
At this point someone can attempt a drupal 6 port. There really is no point attempting a port from the code in CVS presently as it's buggy and quite frankly nowhere near ready for prime time on the internet (I have a laundry list of missing features that are required to operate it safely and securely in public. It works pretty good for us at Post Carbon, but not great).
There are some rays of sunshine though!
Drupalcon boston is coming up at the end of this month and Mack from Post Carbon is going to be there. I'm once again trying to get this stuff closed off and a public test environment made on the postcarbon servers so it'll be easier for people to test. I'm hoping Mack's meetings with Moshe and other prominent figures will help get some life breathed into og2list in the form of more testing / community support.
I do know for sure that once everyone starts using this thing, more bugs will be found. It's an extremely complicated module. There is lots of stuff running in there that probably no longer needs to be there, legacy from the early Drupal versions (before my time on og2list). This a good thing, it will move it forward towards prime time.
So, my plea once again, please test these patches!!! These ones are things I KNOW are broken and I have fixed... it is the first step. the patches are:
http://drupal.org/node/148713 <-- this one will take FIVE minutes to test! But still no one has looked at it yet ?!?!?!?!?
http://drupal.org/node/165299 and http://drupal.org/node/161393 <-- these are related. One involved a fix to og which got tested / committed in days. The og2list side is still sitting here needing more testing. Notifications do NOT work properly with og without this.
also there is this: http://drupal.org/node/188403, which I am still working on. I just found another bug in my implemented fix on the private version that is causing blank node bodies for CCK types that are edited before the notification gets sent out by the perl scripts (i.e. in the first 2 minutes after creating it). It will be fixed soon, and then it too will need testing.
So... let's get those first three tested! I'm not sure if it's the og2list installation that is so daunting for people and preventing testing?? Hopefully my public test groups can help. Meanwhile if you are trying to install it, please please please test these patches for me and comment on them on the appropriate threads. It's by far the best thing you can do to move the module forward along the yellow brick road to Drupal 6!
thanks!
So these patches apply against HEAD?
Which can be found by clicking "all releases" and downloading http://ftp.drupal.org/files/projects/og2list-HEAD.tar.gz (last updated August 5?)
I think that's what's holding people up...
benjamin, Agaric Design Collective
benjamin, agaric
yep, they are for HEAD, which is a Drupal 5 version
Benjamin,
that is correct the patches go onto the HEAD available under "all releases".
http://ftp.drupal.org/files/projects/og2list-HEAD.tar.gz
I am working on the 2 needing backported patches as we speak (CCK rendering and node revisions)... so that should give us all the fixes we need to roll out a 5.x-1.0... then we can figure out what is next re: getting more functionality in there.
Also if someone wanted/needed it for D6 right away someone could start work on that.
I'm hoping to have all my demo stuff and public testing environments ready to rock before DrupalCon Boston in a week. I won't be attending but Mack Hardy (mackh) is going to be there and can demo the post carbon version in all its feature laden glory :)
I'm hoping that in turn generates enough interest to get CVS moving or some of us as co-maintainers and to a release.
How can we test and contribute to get this module out?
I'm interested in using this module for an open site I'm setting up at groupmill.com.
We don't have on staff developers (but may have a test server available) but are interested in helping to get this module out as soon as possible.
Let me know how.
Ditto. Looking for the code!
And if i wasn't clear before, WSF2008.net is in 5, and that's what I would help test/develop this module in.
benjamin, Agaric Design Collective
benjamin, agaric
test instances are ready
in case you didn't get the notification, please see: http://groups.drupal.org/node/9392
The test instances are ready and the first patch is ready for testing!
The og2list demo at Drupalcon
The og2list demo at Drupalcon was unexpected and utterly fantastic. Thank you all for your work on this. I'm looking forward to testing this when I have time to configure our server.
is there video?
is there video?
not yet
There isn't as far as I can tell but keep an eye on the video discussion and the drupalconboston2008 keyword at archive.org.