Cracking the open-source community code for drive-by developers

Amazon's picture

Cracking the open-source community code for drive-by developers

Matt hits on one of the core driving factors of this redesign. In order to grow the Drupal project, we need to make it easier to contribute. How do we do that? Big prizes await those who crack it.

Kieran

Comments

...

roopletheme's picture

This is just a small part of the puzzle, but...

As long as there is a requirement that Drupal contributors master the details of CVS, we will effectively shut out the vast majority of the designer community. It might be reasonable to expect that module developers understand CVS, but I would argue that module availability is already an area of strength for Drupal. The much more common criticism is that there are not enough quality themes for Drupal, and this criticism is well-founded. Unless the technical barrier to entry is dramatically lowered for designers, this will not change. Pointing potential contributors from the design community to FAQs and how-tos about the technical details of CVS is obviously not the answer.

I recognize that I'm not offering any solutions here... but identifying the problems is part of the solution.

The central code repository

yaph's picture

The central code repository is one of the big strengths of Drupal and it would not work without a version control system. I agree that the technical barrier imposed by CVS can discourage designers and developers to contribute to Drupal.

On the other hand this barrier is one of the reasons why the quality of code is generally better than what you find in many other open source projects. Without a central repository monitoring contributed projects for security vulnerabilities would not be feasible.

Using version control systems is something very beneficial not only for developers and there are good tools with GUIs that facilitate using CVS for people not comfortable with command line interfaces.

Documentation on CVS usage could be improved using more visual elements such as screenshots and screencasts. Still we should expect people who are willing to contribute to also be willing to learn something new.

--
My Drupal Articles

...

roopletheme's picture

With all due respect, you've missed my point entirely. I'm not suggesting that we stop using CVS. Code management and version control is an obvious requirement. I'm simply suggesting that the current requirements for contributing a new project to drupal.org restrict the pool of contributors to those familiar with code management and version control, or those with the technical capability and the desire to learn code management and version control. If it were possible to provide a means of contributing to the existing repository without having to essentially become a software developer, we could expand the community to a whole new world of talented individuals who happen to not be programmers (artists, designers, typographers, etc.)

Easier said than done, I know. But that's why we're having the conversation, isn't it?

gsoc project proposal

catch's picture

I suggested this for a Summer of Code project, but no-one was really interested - the discussion is here though: http://groups.drupal.org/node/9552

Essentially, we'd make a web-based front-end to $rcs which lets you upload themes to be checked in.

Documentation

chrislynch's picture

Maybe a bit lo-fi, but could some decent documentation and some walkthroughs of how to use things like Tortoise (http://www.tortoisecvs.org/) not resolve this situation?

like these?

catch's picture

There's already documentation for Tortoise etc., I'll admit to not having looked at it for a while though (and while I work on core a fair bit, I only maintain one module in cvs so don't have to deal with this too often).

http://drupal.org/handbook/cvs/clients

I'm not sure the problem is actually using the clients etc. - it's learning anything about version control at all which is likely to be a barrier to 'drive-by designers' - any amount of documentation isn't going to make it as easy as uploading a zip or tarball. But then I'm not convinced that having thousands of themes uploaded as zips then left to rot is such a great way to encourage contributions anyway. We already have many dozens of insecure, poorly coded contributed modules which are little good to anyone, and confusing to new users - what's more useful is getting people to contribute to existing projects and make them better, and IMO this is quite a bit harder.

...

roopletheme's picture

I suggested this for a Summer of Code project, but no-one was really interested - the discussion is here though: http://groups.drupal.org/node/9552

Essentially, we'd make a web-based front-end to $rcs which lets you upload themes to be checked in.

Precisely the type of thing I'm suggesting, although I would hope that the scope could extend beyond the range of themes. I think the community would benefit greatly if there were a means for artistic types to be able to contribute things like icon libraries, background images, roll-over images, Flash elements, logos, and a whole range of other really significant contributions that aren't necessarily represented by code.

...

dvessel's picture

This would have been promising. I can imagine an interface with a drop down menu for the folder structure and list of files underneath. To make changes, the theme dev can go to a specific folder and upload a single file or groups of compressed files and set their tags and commit messages.

From there the server can do a diff possibly previewing the affected files with a final "commit" button to apply their changes.

Maybe even develop a module so theme devs can install in the local machine so their CPU's can do the heavy lifting then translate it to cvs commands.

I didn't mean to imply that

yaph's picture

I didn't mean to imply that you want to stop using version control. What I wanted to point out is that having barriers also has its benefits and is not something entirely negative.

--
My Drupal Articles

More screenshots and

joachim's picture

More screenshots and screencasts do not make better documentation!
Clear, well written prose is the key to good documentation.

Software development is

earnie's picture

Software development is technical. If technical people can't follow technical requirements who would want to use their results anyway. A good resource for Drupal's use of CVS is http://drupal.org/handbook/cvs/quickstart. True, it is written for a command line environment but you can find command line cvs clients for Windows OS as well. I've used Tortoise but I prefer the command line over the GUI because I have more control over it. I use MSYS from www.mingw.org and its msysDTK package which contains a few necessary items for development like a cvs client.

Design work is not technical

laura s's picture

—or I should say is technical in completely different ways. I think roopletheme@... makes a good point (especially since I've been making the same point for 3 years now ;) ).... anyway, it's a reality and any way to lower barriers to designers would be beneficial to the community, imho.

OTOH, another reason we're having difficulty attracting strong designers is that universities are still cranking out graduates who think "web design" means knowing how to use Dreamweaver. What a waste of a Bachelor's Degree! I don't know what the answer is there, except to make the community most inviting to designers so they can learn like so many of us did – by doing.


Laura
pingVision, LLC (we're hiring)

Laura Scott
PINGV | Strategy • Design • Drupal Development

By "design" do you mean

earnie's picture

By "design" do you mean providing themes?

Well, what do you

laura s's picture

Well, what do you think?


Laura
pingVision, LLC (we're hiring)

Laura Scott
PINGV | Strategy • Design • Drupal Development

...

roopletheme's picture

Software development is technical. If technical people can't follow technical requirements who would want to use their results anyway.

I agree. As I alluded to in my original post, I think that module developers should use the existing methods of project submission and maintenance. Same goes for 'code-heavy' theme submissions. I'm not advocating a means to dummy-down the process of submitting complex software solutions. I'm more interested in creating a simpler method of submitting CSS-only themes, and perhaps creating a new class of submission that doesn't fall into one of the existing module/theme buckets.

As examples of the latter, how would one go about submitting an icon library along the lines of the lullabot icons? Or the forum icons offered up at http://drupal.org/node/102743? If an artist created a set of replacement icons for the advanced forum module, should he/she be required to master CVS in order to share them with the community? Suppose someone wanted to submit a set of custom header images for use with the Marinelli theme... where and how would they do this?

Bill

AF styles

Michelle's picture

I know this is totally OT so this is a one off post... Contact me if anyone wants to take it farther. But, since you asked, if anyone wants to submit icons or even complete styles for AF, just file an issue and I'll get them added. No CVS required. :) I'm actually hoping the community will start providing styles once this massive redoing of the style system is released in the next alpha.

Michelle


See my Drupal articles and tutorials or come check out the Coulee Region

Boris Mann's picture

For all of your examples, they could put them in a zip, attach them to a project issue, and say "I want to contribute these". That's really more of a matter of the project owner reaching out and saying they need / have that.

For example, Quicktabs has nine styles of tabs. Fivestar has a variety of "star" icons.

The lowest common denominator of contribution are ALL the same -- create a project issue and attach your changes. Don't know how to make a proper patch? Attach your module / whatever, set the status to needs work and ask someone to help you make one.

...

roopletheme's picture

No doubt, a means exists for submissions of any type. But does this means achieve the desired result? IMHO, empirical evidence suggests that it does not.

One of my examples (the replacement forum icons from http://drupal.org/node/102743) actually involved a project issue with an attached submission, exactly as you describe. After 177 posts, a potential contribution died and the contributor decided that he was no longer interested in submitting his work, stating "I'll finish this set and release it on my own site."

How many people in the community even know that this submission exists? How many are likely to run across it buried in the issue queue? Would the community have been better served if there were a place for this person to easily submit the icons and say "Here they are, use them if you want to"? Do they really need to be integrated into an existing project before they are made available to the community at large in a manner that is easy to locate for someone searching something like this? Couldn't they just stand alone as a modest but useful contribution?

Yep

Boris Mann's picture

They absolutely could. As a file attachment. In an issue queue. Which is what works today, and just needs a "contribute here" guideline / workflow that leads straight to a minimal issue queue submission.

As I pointed out elsewhere, for bonus points set up a "Contributions" project issue queue.

I read that forum thread. 1. Everyone has an opinion on graphics, so it generates lots of discussion. 2. Wow, cross browser, cross OS graphics issues really suck. 3. Wow, cross cultural definitions of icons really suck. 4. It's core, which is the ultimate in bike shed discussions and 5. It probably would have been easier to submit an entire new core theme.

Also, all of the designs / icons directly uploaded to that thread OR linked off site accomplish the goal. Those are contributions. Great!

(Also, no one came along and rolled a patch and marked it RTBC at any point ... which is how contributions actually happen, or a commit on the part of a contrib maintainer -- don't know what that says other than quality of contribution).

#5

Michelle's picture

I agree with 1-4 but #5 doesn't make sense. These are replacement forum icons, which live in /misc, and aren't even part of a theme.

As for no one making a patch, good point. I didn't even realize you could make a patch for something that isn't code. Maybe no one else on the issue did, either? I thought it was a matter of the core committer grabbing the icons and dumping them in the folder. Had no idea a patch for that was required.

Michelle


See my Drupal articles and tutorials or come check out the Coulee Region

I think creating a Forum for

deviantintegral's picture

I think creating a Forum for these sorts of contributions is not a good idea. As a module maintainer, I want people to be directed to the issue queues for each project, so I can see what has been done. I've had people upload zip files, or even just a line of code in the body, allowing me to make a patch myself. If these sorts of things are directed to a general forum, then I have to wade through all of those threads just to find the ones relevant to my projects.

I think what needs to be done is to reach out to those who are more comfortable with Forums, and make them more comfortable with the issue queue system. The other thread about having a documentation tab for each project might solve this; simple patches and modifications could start out as issues, and then either be committed to CVS, or posted within the module's documentation handbook.

--Andrew

Huh?

Michelle's picture

Maybe I missed it but I don't recall anyone suggesting using the forums to submit things to projects... If it's something for a project, it belongs in the issue queue. Are you consuing this with the "dump forum" idea? Or am I the one that's confused? LOL

Michelle


See my Drupal articles and tutorials or come check out the Coulee Region

I hope I'm not confusing

deviantintegral's picture

I hope I'm not confusing anything :)

The way I understood it, the idea of the forum was so people could put code snippets up to change in core or contributed modules. I've seen this with phpbb, and it's hell to maintain.

Even if that's not the intent of the forum, and it's for new contrib modules and themes which don't touch anything else, I think users will use the forum for both. It just adds more confusion to Drupal projects - I know one of the first questions I would ask, would be "where does my module / theme go?". As is, we have a single consistent set of instructions.

Here's a new idea: have a list of individuals who are willing to guide new theme contributors. I'd be completely willing to act as a maintainer for a theme or two, and do one-on-one over IRC to eventually teach the individual how to do it themselves. I know it's a lot of work for the volunteers, but it might help "bootstrap" the theming community.

For those who have been to many Drupal conferences - is there often a demand for sessions aimed at themers on how to use CVS and the project module? What's the response like?

--Andrew

Ah, no

Michelle's picture

The dumping ground forum would be for just misc stuff that people want to dump off and not maintain. The junkpile, so to speak. Modules / themes would still be maintained thru the normal process.

Michelle


See my Drupal articles and tutorials or come check out the Coulee Region

Managing the one-liner patches

Evan Wise's picture

Long time watcher, first time poster.

This is my own experience with wanting to contribute to projects (open or otherwise) in the past.

I have a simple one liner patch that fixes KNOWN_BUG. I want to give this functionality over to the community. Let's say I am, on the scale of skills (Drupal or otherwise), a 4 out of 5 stars kinda person and really don't have any intention of becoming a module maintainer, core contributer or the deacon of documentation; I am just scratching the itch of "something was broke, I fixed it, lots of people need this fix, here's what I fixed and it works". There are lots of these types of people.

So, patch in hand I dive into IRC/mailing list/etc and say "I have a patch - anyone want to run with this"? I then stand back and enjoy the glow of various 5 out of 5 stars people telling me to do any of the following:
a) create test cases
b) create an issue in the BUG_SYSTEM and then create a patch of the form XYZ
c) create documentation
d) learn ARCANE_COMMAND used on VERSIONING_SYSTEM
OR
e) you get completely ignored or worse

Wow, that's a lot of work for a one line patch! All these are great things BUT from my perspective of dev-who-fixed-bug - I don't want to get involved; my itch has been scratched. I essentially want to paste my code into an automated queue, give it over to the trivial patch person, ??? and walk away with the warm, fuzzy feeling of being a minor contributor.

Now, I realize that processes are needed to support the development of code - I am CMMI trained/aware/scared - but to make someone transition from drive-by to local resident you need to make those first few one line patches/experiences enjoyable. The process should be revealed in a gradual progression so they become more aware of the "Right Way" as they get more deeply involved and are making bigger contributions. In the end it boils down to all code should be treated equally but all people should not be.

my 10 cents.

E/.

Evan Wise
http://justwerks.com
Skype: evan_wise

--
Evan Wise
http://justwerks.com
Skype: evan_wise

that hard?

catch's picture

If you go to http://drupal.org/node/add/project-issue/bug - write up the bug, upload the patch (or even paste it in), then there's a fair chance someone will run with it. Although what we really need is more 'trivial patch people' to actually look at the patches people post - since there's much less of those than there are people submitting patches.

not hard, just more is needed

Evan Wise's picture

Completely agreed, in the past where I have made minor contributions, in one case Oracle support did not work as it used MySQL only SQL, I got lucky and a core contributor saw my contribution and it was rolled in, so I got the warm and fuzzies. I then dropped off due to "life/work".

You completely nailed the issue though: if there is no-one who is monitoring these minor contributions you might miss out on the next great code guru. It's a double edged sword because if you are managing a queue where just about anything can be contributed you need to have a broad view and contact with many people in the entire project and typically those people are too busy.

Perhaps if there was a maintainer for a particular domain (themes, CCK/Views, core, major use modules, minor use modules, abandonware, etc) then there would be greater retention of the people that provide patches and a greater likelihood that those patches actually get rolled in.

E/.

Evan Wise
http://justwerks.com
Skype: evan_wise

--
Evan Wise
http://justwerks.com
Skype: evan_wise

Patches == issue queue

Boris Mann's picture

Everything starts with the issue queue. None of the rest is "Required". You have done your part.

Next up is easy auto-susbcription (Get notified when someone touches your patch) as well as "attention" of the patch itself. These are all things that help EVERYONE who uses d.o. for project development, so we can focus on helping with that.

I think it's as simple as clear, guided step that teleports you from "I want to contribute a patch!" to a link to the project issue add page. Everything else is followup, which is what is needed anway.

BUT ... until YOU the patch CONTRIBUTOR understand that contributing patches and following up them yourself make YOUR life easier ... then it's fail. Unless the interests of contributors are aligned, this will not work. Why contribute a patch? So you don't have to maintain a bunch of patches as the code gets upgraded.

I've been reading the recent

earnie's picture

I've been reading the recent comments for this posting. One method I can think of is a dumpbin project that gathers bits an pieces from the one off contributor who just wants to leave a piece of code. The maintainer of the dumpbin project would take that code and massage it to something useful for a patch or different project.

.

Michelle's picture

I suggested making a forum for this over a year ago and it got shot down.

Michelle


See my Drupal articles and tutorials or come check out the Coulee Region

Forum, no; Project, yes

earnie's picture

I don't think a forum has the correct methods to control the submissions. A project named dumpbin where code is deposited and left for others to handle does have controls. The project download file would contain one README file that explains the user of the project.

.

Michelle's picture

How do users get code into this dumpbin without CVS? That was the point of using a forum. People with themes or bits of modules or whatnot to contribute could just make a post and attach it. No one I talked to liked the idea of having all that potentially insecure code sitting there with no one checking it out to make sure it was ok. I don't see how putting it in a project helps that any.

Michelle


See my Drupal articles and tutorials or come check out the Coulee Region

snippets

catch's picture

Isn't this the php snippets section in the handbook?

A place for icons etc. would make sense, and that's currently a lot harder to deal with. Along similar lines - the usability group would like some kind of centralised place to post wireframes/screenshots etc. and it's not obvious where that should go either (I think it'll end up with emfield installed on here though, which should be useful for other groups too).

Not exactly

Michelle's picture

The handbook gets weeded and snippets tested and checked. People expect things in the handbook to work. The forum would be a total dumping ground with a big warning on it. Anything could go in there from small bits of code and graphics to full modules and themes. Good snippets could be culled from there and put in the handbook. Good themes and modules could be put into projects if someone decides to adopt them.

Michelle


See my Drupal articles and tutorials or come check out the Coulee Region

well...

catch's picture

There's already a big bold warning at the top of the handbook sections - http://drupal.org/handbook/customization/snippets

The dumpbin would be a

earnie's picture

The dumpbin would be a project where they can paste or upload the code into an issue for the dumpbin project. A responsible party (the maintainers of the dumpbin project) would then need to decide to create a patch based on the dumped code into the appropriate queue or create another module or theme as needed.

Who?

Michelle's picture

And who is going to maintain this? My idea with using the forums is that it would be intended just as a sharing spot and nothing more. Someone with something to share posts it and people can use it at their own risk if it looks useful to them. It would be clearly labeled as "unpoliced" because of the very fact that no one has time to maintain a dumping ground of code.

Michelle


See my Drupal articles and tutorials or come check out the Coulee Region

No one has time -- a trademark of many

earnie's picture

No one particular person but the community of developers could do the maintenance. But yes, it could be unpoliced and if the code makes it into a patch, module or theme the implementor can close the issue. The dumpbin project would be just a holding place for one off code for those who don't want to spend the time making a patch or project.

Another advantage

dwees's picture

The forums wouldn't be unpoliced actually. People would try out the snippet, patch, module and if it worked, they would naturally say so, and if it doesn't work, they would come back and complain. This already happens on an ad hoc basis in the forums all the time, it just isn't very well organized. Having a specific forum dumping ground would be a good idea. What would not happen on the forums is an organizational effort focused on these new ideas.

Coming to an issue queue with 450 new submissions might be a bit demoralizing for anyone whose turn it is to maintain the dumpbin that week. My guess is that many inappropriately placed patches/feature requests would end up here for people who didn't take the time to properly categorize their response (guess how many issues I've had opened up for Book Expand when actually the person wants the Drupal project - Book module).

Dave

How about drive by companies?

Boris Mann's picture

I'm less concerned with drive by individual developers. I'm more concerned with drive by companies -- agencies, smaller dev shops, etc. How can we convert THEM into doing contributions -- from design, to documentation, to patches -- and show them that this is GOOD for their business model.

I bet most people can name half a dozen shops in their local region/city that use Drupal but don't contribute back.

By Easy, we mean easy, not hard

Amazon's picture

Pine commands for email.
Command prompt syntax.
Manual transmission
Natural language query syntax

These are all things that changed the world once they were made easy. For example:
Web based email. Browse, type, send.
Windows, Icons, Menus, Pointers.
Automatic transmission: steer, gas, brake.
Google search box.

Think about it. Still want to make excuses for why things should be hard?

Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

Not excuses

Boris Mann's picture

Currently, professional code development dictates revision control systems. You need to learn it, so suck it up if you want to be professional.

I would argue that configuring a drupal site as desired through the web interface, then pushing a button that says "make Drupal site recipe", and having THAT auto submit to Drupal / some DrupalRecipes.com site, would be the ultimate in easy. Of course, I've only been working towards that goal for about 4 years now :P

I don't see any ways of making good quality PHP code easier. Economic incentives and feedback cycles, hence my question on drive by companies.

Lowest barrier already exists ...

kbahey's picture

I agree with Boris.

All we need is a project called "Miscellaneous modules" or "Module dumping ground" or something, and then people attach a tar.gz or .zip to issues describing what the code does.

This is what Ubercart does on their site and the response is very good.

As modules mature, or people want to contribute more, they can apply for a CVS and go the usual route, at their own pace, as they get more comfortable.

This way, there is no bottleneck by volunteers needed to commit anything.

In the future, we can have five star or some other way to vote up modules attached to issues.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Well...

EclipseGc's picture

We've talked a lot about theming here and I think that is not w/o merit. Boris is absolutely correct, versioning systems are a must corporately... however I'm debating whether such a thing is necessary for a theme. Really versioning is for the developer in many ways, and most themers are NOT devs... don't pretend to be devs, and would rather not have the responsibility of knowledge that the typical dev has.

With that said I like catch's idea that we have a system where theme developers could upload their theme much more easily... this same environment could have CVS backend for those developers savvy enough to make use of it, but I think our contribution of themes would go up drastically if this solution were in place. While we could argue over quality of contribution, I think we still can, so perhaps it's a moot point. Make theme contribution easier and I'd bet the face of our community would change quickly, because frankly, phptemplate is one of the best templating systems out there, and themers will get it.

That's my opinion.

Eclipse

I think the main thing that

mikey_p's picture

I think the main thing that we need to be concerned with, with an uploading system, to keep it working with update module and such, is that everytime a theme maintainer uploads a new theme, they are forced to give it a new version number, and also, it must have a .info file added automatically, for update module to know what to do with it. Other wise, after that is done, it functions just like a project release, that is, that we discourage deleting, so on...

Just do it

Boris Mann's picture

So, before we go off and build something... go find X number of people willing to create projects / commit to CVS on behalf of designers.

Say that you'll do this for any theme.

See how much submissions go up over a 3 month period, if it works, automate it.

I've been trying to nail

EclipseGc's picture

I've been trying to nail down our two biggest competitors (Joomla! and WordPress) to see what they do in this regard (considering they both have a better quality of themes generally speaking than we do and more of them). What I've managed to learn at this point is that WordPress basically does what catch is suggesting.

http://wordpress.org/extend/themes/about/

Additionally it's worth noting that we could generate .info files for themes easily enough with just making the user fill out a few fields when they upload the theme files. I would think that "version" could be controlled COMPLETELY by such a system... i.e. Drupal versions the info file, unzips and tags all the incoming files from the upload, and then checks them in and tags them the same as the created info file. After that it could auto-release the new update, and I think CVS could still be in charge of most of this stuff... I know I'm making this sound easier than it would be, but I think it could work pretty easily.

And after many hours of looking and asking questions I have yet to find out how Joomla does it... many of their themes appear to be "pay for" anyway, so it may be that due to their licensing it's not as prevalent. I dunno... what I do know is that they have an abusive bot in their irc channel which was at least amusing.

Eclipse

brain-fart

dman's picture

x-posting myself from the "badlands" code-bin/trash-can suggestion thread...

... if that means coming up with a magic system that auto-commits peoples uploads into CVS for them, that would be cool. Surely it's been done before. We have web-based repository checkouts, why not commits?

There's no real reason (apart from time and/or money) why someone couldn't just upload a copy of the file they changed into the issue queue and have a system generate the patch that maintainers are so picky about! ... just a random thought. Anyone want a project?

I know it's not as simple as all that, but it's not impossible either.

Could even be extended to multi-file zips. Could be a killer solution to the whole CVS 'problem' for young players.
Issue queues are already tagged with the project version numbers, so a script would know what to apply it to. There are almost no chances of filename conflicts in current Drupal project modules ...

And for bonus points, add a feature that pumps out module+patch_x for all those whiners that want a fast-food lunch.
(Personally I've often been in the 'you gotta know to understand' team in the learn-CVS vs I-want-it-now debates but a button to "download this package with this patch attached" could surely be automated?)

I'm not volunteering :-)

Automated patches is

deviantintegral's picture

Automated patches is certainly a good idea - especially if it was self documenting. If it showed the diff command run each time, it would be useful for those learning. And every time such a file is uploaded, the submission page should link to the CVS docs in the handbooks.

I think auto-patching a module is a bad idea. It gives the idea that such a set of code is supported, when it may totally break a site. When you apply custom patches to a module, you have to have enough knowledge to be able to undo it and preserve your data if the patch is not applied to the main branch.

--Andrew

true

dman's picture

Yeah, I'll agree with everything you say about auto-patching for dummies. I've made similar arguments myself before.
I think I was just imagining what was possible not what was a good idea ;-)
Scratch it.