When trying to import external forums into Drupal, I've found the main difficulty is a single fundamental difference between Drupal forums and the rest of today's forums.
It's the same one difference that for example makes common administrative tasks such as splitting or merging threads (while keeping chronological order), etc., easy in well-known forum systems but so difficult in Drupal forums, core and Advanced; difficult with Comment Mover, and even directly editing the database. Drupal forums are good and usable in most cases, with most normal functions similar to other systems but, for more advanced moderation, they are not yet ready.
Why? What's that basic difference? We can see it in the support forums of the currently most popular forum systems, such as vBulletin, phpBB, SMF, and Invision Power. Or we can install them and look at the tables where posts and threads are stored.
Drupal have modules for threaded and flat forums, so this is not the main difference. Most current forums are linear/flat by default, but part of them admit both linear and threaded display modes.
The basic difference is another one, truly simple but with fundamental effects:
In forum systems, all posts are treated equally as part of a discussion -which can include several or many long posts-, with ID numbers in chronological order (including also the first post in each thread), and this gives the flexibility needed for common forum moderation tasks, to merge/split/move things around while automatically keeping chronological order, etc. On the other hand, for example blog systems use a different content type, with a main long post and brief replies to the main post.
And Drupal forums are... not forums. They are blogs trying to behave as forums, using the wrong tool or content type. Drupal forums are blogs. The first post in each topic is the topic node (stored in the node and node_revisions tables), and the rest are comments to the node (stored in the comments table).
Surely most people in this group know about all this. Even many people from outside. Quoting for example from the phpBB support forums: "The reason I don't like the default drupal forum module is because it handles the first post in a thread as a new node and each post that comes after it as a comment."
So it's a known fact, but I'm pointing at it because in my opinion we should also look at it as a fundamental flaw.
It appears this has happened for historical reasons: nodes and comments were the tools available when the core forum module was created, and they were used for forums and for blogs in similar ways. That is, instead of creating a structure similar to standard forums, taxonomy was powerful enough to disguise blogs as forums.
Therefore, possible solutions to move from that legacy arrangement to a flexible, standard forum structure would be really simple:
- All posts as comments: when creating a new topic/thread -a node without text-, a first comment would be created at the same time for the first post.
- Or all posts as nodes: like in the other solution, topics should have their table and ID numbers, and posts their different table and ID numbers.
This would make things much simpler, having real forums instead of blogs needing convoluted procedures to behave like forums.
At this moment, looking at the bright side, Drupal's Advanced Forum seems to be the best option available until a real Drupal forum is developed. Even when it does not solve the basic problem, Advanced Forum is a great progress to add functionality from modules to the forums, and to style them like common modern systems, duplicating for the node the enhanced look and features of the comments. Sadly, the inner working is still the same blog-like, with the same difficulties for importing into Drupal from standard forums, and for moderation functions common in the other systems. But it works reasonably well, and improving, in many simple cases that don't need much moderation/administration.
Since people like Catch and others are looking into this kind of problems, surely improvements will keep growing into Drupal forums.
Comments
.
You already can have all the posts in a topic be nodes and merlinofchaos will be working on a moderation module as soon as he has a chance which will make things like moving and merging topics possible. We're on the right track already. I'm not sure the point of this post...
Michelle
That would be great
If those projects are in the direction of Drupal forums working like all standard forums, that would be great, naturally. Let's hope so, and thank you very much for your generous efforts. I will try the new node feature in Advanced Forum when it's ready, I think in the upcoming 2.0 version.
I wrote that from the experience I'm having right now importing forums and trying to administer them. Working with the databases, all tasks are clearly simpler with the standard forum structure, not the current of Drupal forums.
Just a couple of examples: If you import a forum with threads created by anonymous users, their data (name, email...) are lost because nodes and comments have different properties. Or, even with Comment Mover or editing the database, it's really difficult to merge threads, because if you move the first post content (a node) to a new comment in another node, a new ID is assigned and chronological order is lost. And so on...
In spite of the constant improvements -for example in Advanced Forum-, all those problems come from that old legacy blog-like structure, good for blogs but not for forums.
Thank you as always for all your great work improving Drupal.
.
If there is a problem on importing, then the import modules need to be smarter. We're not going to rewrite the structure just to make what is normally a 1 time task easier. As for moderation, yes, that is a sticky point with Drupal's forums and it will be addressed just as soon as merlinofchaos is able to work on it. It won't be done by redesigning the data storage, though, but by working with the existing node system.
Michelle
an old chestnut
I agree with the op. Forum comments should be nodes. There are pros and cons to both approaches and I've read through long discussions on this issue since pretty much day 1 that I joined Drupal.org. Many of the problems we have encountered with dealing with the Drupal forums has arisen from this duality of form for discussion related posts. Michelle has done some amazing work on AF but it is clear to an outsider like me that a lot of this is work to pull together modules and add functionality that tries to get around this fundamental discrepancy. For anyone trying to build a site, the Forum Topics/Comments duality just causes headaches. Views 2.0 helps by dealing with comment and nodes equally, but you're still left with moderation issues, themeing, importing your old forum (we gave up on this as it wasn't worth the time) and so on.
Sigh
I don't know how many more times I can say it. You can already use all nodes in your forums. The starting topic is a node, the replies are nodes, it's all nodes. This thread is just pointing out the obvious that is already solved, except for the moderation submodule, which is coming. And I find the OP's comment about using my module until a "real forum" comes along to be insulting. I've worked very hard to make Drupal's forums work as well as a stand alone app and I don't appreciate having my work dismissed as a stopgap until someone else comes along to do it "right".
Michelle
Don't take it so hard,
Don't take it so hard, Michelle. I think the OP has just poorly phrased his comments. You have done amazing work, and I know that I have personally mentioned this to you before. We run a large forum at ChuckPalahniuk.net http://chuckpalahniuk.net/forum that would be next to impossible if it weren't for all of the work you have done on advanced forum.
Frankly, the only portion that we find to be lacking is, as you've mentioned, moderation. We have been slowly learning about proper module development, and hope to have some tools to provide back to the community soon. Though our tools tend to be more on the side of managing users.
At the end of the day, you know that you have done awesome work and there are many appreciative people out here. I gave a demo of AF at my local Drupal meetup group, and everyone was shocked at how much you've been able to accomplish.
kclawes, do your forums use
kclawes, do your forums use replies as comments or nodes? (Looks really good)
We use replies as comments. I
We use replies as comments. I don't like the idea of dumping all replies into nodes because I'm always fearful of an upgrade path. I really don't understand what the problem is with using replies for comments, unless of course, you want to do a bunch of fancy things like having cck fields on replies.
Though if you want to do that, you're obviously not happy with how VB or phpBB handle replies either.
My apologies. As I said, the legacy flaw comes from core forum.
Michelle, my apologies if you feel like that. Very sorry for saying "real forums", I mean "standard forums". I've thanked you in the last months in several threads and issues, because -as I've repeated also here- your work with Advanced Forum is really great and a must for current Drupal forums.
What in my opinion is a flaw, that is to say the difference from the rest of forum systems, comes from the core forum, which Advanced Forum uses as a basis for improvements. Even when the inner problem is still there, in core forum, Advanced Forum is a great progress as I said. It's not perfect just because core forum is not perfect.
What I'm mentioning is perhaps more for special configurations, large forums, etc., and most users don't need most admin features found in standard forums. But for example I know some of those standard forum systems, and I'm now struggling to do imports into Drupal, and administrative tasks. One of the workarounds is to make all possible admin tasks (merging, etc.) before the conversion to Drupal, because in Drupal there are less admin possibilities for forums at this moment.
In a few other threads, I've seen some users don't need much moderation and in fact prefer the blog style of Drupal forums, instead of the style of standard forums such as vBulletin, phpBB, etc. So maybe at least two kind of Drupal forums should exist, the current core+AF, and maybe some kind of CCK+Views forum.
We will see, but for now Advanced Forum is better than other possibilities like bridges, because the mentioned difference with standard forums is compensated with integration into Drupal, which for example makes possible the use of interesting caching possibilities like Boost, etc.
Sorry, and thank you again.
.
Apology accepted and my own apologies for misunderstanding that you were talking about core forum. But you're still missing the point. What you are asking for already exists, except for the moderation module, which is just a matter of merlinofchaos (or someone else who gets the itch) finding time to write it. There is no need to write a "CCK+Views forum" because that's essentially what we already have.
Michelle
So, what do we have to do then
to use all nodes in our forums? I had a look around on the advanced forums project home page http://drupal.org/node/227108 but couldn't find an answer there either.
Jan
Michelle, we are all
Michelle, we are all extremely grateful for the huge effort you've made on Advanced Forum - we depend on it as do many other sites!
I have to admit I hadn't noticed the recent work on the nodecomment module, although it is not without issues: http://drupal.org/node/344617. Is there a page that could be linked to from here http://drupal.org/node/227121 that would give a run through of the issues likely to be encountered when using nodecomments with AF?
Michelle, I see the fact that
Michelle, I see the fact that people complain to you about this "fundamental discrepancy", which is actually in core drupal, not as sign of insult, but as a significant change in the way people think about drupal forums thanks to your efforts.
AF is the best we've got, and I think if anything, the core forum module will be drawing from AF, not the other way around. That said, it is open source, and it may take another few years for AF or other modules to be superior to today's forum software. There are ways to integrate phpbb and other boards into your site, and in fact there are a few bridging modules, if you don't want to wait. I am confident AF will win out, the integration is already so powerful.
Sorry
I guess I was being a bit grumpy. Thanks for the cheering up, folks. :)
What annoyed me was the OP came into my queue about this and I told him that you can make all the posts nodes and he countered with some idea of making the node be empty and having everything be comments instead. When I told him I wasn't interested in going that direction, he came and made this post. That plus the comment about using AF until a "real Drupal forum is developed" just set me off.
@sparq-l: If you do anything in the area of moderation, talk to merlinofchaos. It's on his "to do" list but he is so busy with Views and Ctools and Panels that I don't know when he'll have time for it.
@scedwar: That issue you linked to applies to 1.x. 2.x is completely different. There is one problem in that when you convert comments to nodes it resets the timestamp that needs to be worked around but otherwise it's working pretty good. I'm using it on Coulee Region Online and it's wonderful being able to use imagefield consistently in both the starting node and the replies.
@luxx: I don't think AF will ever be superior to other forum software. While I have had some help, it's mostly me working on this in my spare time. I just can't compete with software that has a team of programmers behind it. I would never expect someone to use AF if all they want to have is a forum. But if you want a forum that is fully integrated into your Drupal site, that's where AF shines. Once that moderation submodule gets in, it will do 90% of what stand alone forum apps do with all the benefits of true integration.
Michelle
Just to offer, I've taken
Just to offer, I've taken roughly what I want to do for moderation from phpBB, which has some pretty nice tools for it.
I want to do it using VBO and actions though I think there needs to be some customization. There needs to be a way to assign moderators to a forum; ideally it will integrate with forum_access if possible but I don't want to require that, because forum_access requires using the node_access table and if you don't need that you certainly do not want it.
It will allow the following operations when using Advanced Forum 2.0 and nodecomments:
It will provide a widget to do this on a per post (starter or followup post) as well as using VBO to allow a search widget to find posts and moderate them en masse.
As Michelle said, I don't have a lot of time, so this has been on my TODO list for months but hasn't actually happened.
Integration with forum_access
I want to contribute my share here. Supporting nodecomments is on my TODO list for forum_access.
I have to agree with Juan_g
I'm new at Drupal and I have battled with it for the last couple of months. The reason why I'm sticking with it is that I understood quite quickly that Drupal allows you to modify and control every little detail of a site... so kudos to Drupal for that. BUT we do have to acknowledge that it's not an easy ride (at first) for most. Drupal has a very steep learning curve and sometimes its behaviour is just not what you would expect. The Drupal forum (core) module is an example of this.
I agree with Juan_G that the one fundamental difference between Drupal's (core) forum module and other forum software is the fact that the first post behaves differently from what people have come to expect due to the widespread use of phpBB and co. No-one expects the first post to pop up again on the second and consecutive pages!
As I understand the Drupal community has set itself a goal to make Drupal more userfriendly and increase its popularity. I also understand that it has been quite succesful at doing so. By changing the behavior (not the system) of the first post in Drupal's (core) forum module Drupal will become just another bit more userfriendly.
Yes, I know that it is possible to change the default behaviour to anything I want to and I do know that I can install the Advanced Forum module to get an experience more akin to other forum software... but I don't want to. I like the fact that Drupal's core forum module has minimal, basic features. It's just a shame that the first thing that needs to be done is to change the default behaviour. Something I've been doing a lot lately.
In my opinion a new user doesn't care whether the first post is a node and the other posts are comments, or that they are all nodes, or all comments... To a (new) user they are forum posts, regardless of the system behind it. Heck, a new user usually doesn't even know what nodes are (I certainly didn't). I therefore don't think this problem should be discussed from an experienced Drupal developer's point of view but rather: "what will a new user expect when they install a Drupal forum"?
Watch this DrupalCon movie to see what I'm talking about. I love this quote: "there's a module for that!" (ffwd to 18:45).
This post is not meant to criticize anyone or anyone's work. Drupal is awesome and so is its community. I just want to give my 'fresh' view as a newbie.
DrupalCon movie requires Password now to access.
I click on the link to watch it, it is asking for login credentials.
Just sayin'.
Same argument
@RedTop: Personally I can't imagine your comment with go far RedTop. From the feeling I'm getting from blog posts such as this: http://www.disambiguity.com/designing-for-the-wrong-target-audience/
It seems like this Drupal 7 focus on an end product has quite a lot of remaining resistence amongst the developers who contribute and make drupal actually work. Drupal is incredibaly good for developers willing to go through the learning curve, in fact its so good its pretty much the only piece of opensource software that does this at all. Wordpress and Joomla are essentially entirely different products. For this reason it attracts awesome people to scratch their own itches and they don't want to spend time on things like the overlay module.
It seems like the feel at the moment is that Drupal should continue in the old direction (or focusing on developers) or (as Dries thinks) seperate focus into Drupal framework and drupal product. It possible that Drupal 7 will see such an influx of new people due to the increased usability that the entire community will change and it will move in the direction you're looking for. But until then, it looks like if you're not prepared to use Drupal modules then frankly drupal isn't aimed at you.
Even for developers there are reasons why some modules should go in core. CCK in core means core drupal stuff (like taxonomy and body) can become a field and I'm sure more awesomeness would result from views in core. But its not like that with the forum module. There is no disadvantage in ignoring core forum and just using AF instead. The only disadvantage with AF as it is, is that people have to put a little extra work finding and installing it, but thats a learning curve that simply comes with Drupal at the moment, and will be way easier once drupal.org redesign happens and everyone start rating the AF module highly.
Again, maybe everything will change before Drupal 8, but until then there isn't much point pushing for tiny changes in the core forum module.
Valid point, but...
Your point about the target audience is a valid one. Worst thing that can happen is for Drupal to aim at two different audiences and inevitably falling short to deliver. However, I'm not saying I don't like modules or that I think Drupal should be more like Wordpress or Drupal. If that is what I'm looking for I would not be posting here and I would not have bothered to get to know Drupal. I certainly wouldn't have put in the time and effort I have. I like the fact that Drupal can be customized any which way one wants to. I'm with you on that one.
My point is not that Drupal should sacrifice possibilities for useability. I 'm not advocating a shift in target audience or philosophy. My point is that this one feature of this one core module (forum) is behaving in a way most people wouldn't expect it to behave. These are people that might be new to Drupal but have experienced lots of other forum software, so their points are not completely invalid. Why not change this behaviour (of the first post) as it seems to be something that comes up quite often. No need to alter the system behind it, just alter the behaviour of the first post and that's it. Developers can still go the extra distance and new users can install a forum and instantly have it behave the way they expect it to behave... simple really.
Drupal developers are rightfully proud of their products. They provide powerful features, momentum and valuable support free of charge and that's just awesome. However, when people are really proud of something they've put a lot of time and effort in people tend to get (overly) defensive. Small points of (constructive) criticism quickly turn into discussions about philosophy or are taken personally, such as with juan_g's remark. This clouds the point being made, even if there are lots of people making the same point. This effect is even stronger if people identify themselves as a group ("we" the developers versus "they" the newbies who want to change what we have made).
I'm an auditor and I come across this behaviour daily. It's human and understandable. However, my professional experience taught me that the people/companies that achieve the best results are also the ones open to criticism. Not to copy everything blindly but to at least really listen, think it over and adapt the best points. I pointed to the DrupalCon movie because it addresses something fundamental: Sometimes you've got to step back and look at your product through a newbie's eyes.
The point made by juan_g is what I, as an auditor, would classify as a "quick win". It probably takes just a few lines of code while the underlying system remains intact, perhaps even as simple as changing the theme templates. It would make people like me shut up... wouldn't that be worth it? ;)
I'm still gonna go with Drupal's core forum module. I'll just struggle and change what needs to be changed. Not because I don't want to install another module, but because the core forum provides enough functionality for me as it is... if only this first post behaviour... oh well. ;)
Not a few lines of code
The core forum module is one of if not the oldest modules in Drupal. It receives very little attention outside of making sure it's functional with each new core release.
What you are talking about is a major change, certainly not a few lines of code. Drupal is very much centered on the node + comments model. The forum module follows this and really isn't all that much different than blog except it uses taxonomy to provide order. To change it over to use all nodes would mean rewriting most of it. It's just not going to happen.
Like most things in Drupal, the solution exists in contrib. With AF + Nodecomments + Views, you have a forum that acts just like you'd expect. If you choose not to use the tools available, that's your choice. But you'll find what you're trying to do is not simple and you are likely to simply re-invent contrib doing it on your own.
Michelle
I think I read that Dries
I think I read that Dries started off Drupal as a forum package and then turned it into a CMS, rather then the other way round?
I'm not talking about changing the system
I'm not talking about changing the system, I'm talking changing the way the first post behaves or at least seems to behave, regardless of whether its a node or a comment. I think that can be done quite easily at the theme layer through templates?
Like I said, new users don't care about nodes or comments. They just want a forum that behaves the way they expect it to. So why not make the first post's appearance exactly the same as the other posts? Next prevent the first post from appearing on any other page but the first one and you're set. I wouldn't have given it any other thought if that had been the case. Only after it didn't I went and figured out about the system behind core forum.
Maybe core forum module is in need for a mayor overhaul but until that time this could be changed by some clever templates? You're right that AF + nodecomments + views gives me a forum which acts as I expect to. I've tried and it works beautifully. I'm just looking for a very, very basic forum.
Let me stress again that I'm not advocating a mayor change to Drupal or Drupal's philosophy here...
Behavior or appearance?
There's a difference between behavior and appearance. Yes, it's easy to make all the forum posts look the same; that's simple theming. Making them behave the same is different. If you add a CCK field to the topic node and want it on replies as well (think: images), you find out quickly that nodes and comments behave quite differently.
At any rate, you can pull the bits you need out of AF. Look at the node & comment preprocess functions.
If you want the change in core, you'll need to file an issue and convince people of how it should work, which is often more politics than code. That's why I stay in contrib. ;)
Michelle
Perhaps I indeed should have
Perhaps I indeed should have used the word appearance instead of behaviour. I'm content with posts simply looking the same and not having the first post appear on every page in a thread. I have no need (this time) for some of the more sophisticated Drupal features (which are not part of any other forum software either). :)
Thanks for the pointers. I'll look at AF to see which parts I can use. Copying is a good way to learn new things and is also the biggest compliment a developer/artist can receive. ;)
Good one on the politics vs code analysis. I think I am going to go with your approach... I hate politics. :P It also provides a really, really strong argument for Drupal to keep moving in the old direction. It prevents us having to bother with core politics. :)
Personally I'm an end-user
Personally I'm an end-user with no coding experience and love the direction of Drupal7, also forums are important to me (come from phpbb as a cms) so I'd love a total core forum overhaul (probably by replacing it with AF :) ). Also I think your point of view fits with what drupal developers believed about a year ago (as you've shown in that video).
However, that blog post was written by one of the 2 main designers who's Job it was to make usability happen, and since then opinions are changing. I think drupal has listened to the kind of criticism you've talked about generally (and Michelle has heard it quite a bit I think regarding the forums) and I think now (and I mean literally right this second and the next couple of weeks/months) drupal is moving on from that criticism. Either to ignore it, embrace it more fully or do something in between.
(Also the more I read of core, there is no such thing as a quick win :( I'm sad about some of the cck features left out)
I'm only saying this because whilst I think your arguments are valid, I don't think they are that new, and I think you'd find your energy is best placed elsewhere. (As I said I don't think they are proved invalid yet either and we'll see where Drupal goes) Either by continuing your direction of modding stuff yourself, or by making AF fit your needs or even just releasing your code as a contrib module that you'd then campaign to get into core, maybe a Simple Advanced Forums module :P (Personally wouldn't recommend the latter)
Drupal Pro Dev book in the mail
The reason why I'm actively participitating in this discussion is because I love Drupal. Just like you guys I'm glad it's incredibly versatile even if it does give me a headache every couple of days. :P
I trying to contribute as much as I can with my limited coding/Drupal experience. Look at this for example:
http://drupal.org/node/509234
and:
http://drupal.org/node/459222
It's not much and I've got a long way to go but I'm trying!
I will try and make the changes I need and get back to you guys to share my experiences (and headaches). :)