Managing the Drupal.org redesign: Prelude to a manifesto

Events happening in the community are now at Drupal community events on www.drupal.org.
todd nienkerk's picture

We need to discuss how we want the d.o redesign project to move forward. One- to 5-day sprints every few months is not a sustainable solution. Maintaining momentum and continuity is nearly impossible.

Initial thoughts:

  • Take a cue from the Aegir project and introduce a proper schedule
    • Require weekly time commitments and weekly meetings
    • Initial thoughts: 4 hours per week for at least three out of every four weeks (meaning you can skip one week from every four to catch up on client work)
  • Find a proper Project Manager -- not a designer/themer/developer willing to take the role of PM
    • May require hiring a part-time PM
  • We need a dedicated and managed development environment. Having to download, pass around, and import DB dumps can easily eat up half a day or more, and we need a place where people can publicly view or test our work.

Comments

Thoughts.

ceardach's picture

Weekly meetings may be too often. I have found in other projects that if we meet weekly it starts eating into our productivity -- especially if we're not doing that project full time. But people should be able to touch base in an easy and casual manner, and schedule real group meetings on an as-needed, or bi-monthly basis.

So, we should enable easy and casual communication. I love IRC. Is there a redesign implementers IRC channel? If not, maybe we should have one. We should say, "If you're working on it, sign into IRC. If you have some brain cycles, then sign into IRC." That should help foster better communication, which will result in inspiration, and thusly motivation.

Having a scheduled "virtual code sprint", or otherwise imagined as "virtual office hours", could be helpful. Just a time where everyone who is working on the project is expected to be on IRC and actively doing work during that time period. Being able to talk about your ideas, get questions answered, and casually discuss larger issues could promote productivity.

Another thing to do is to break tasks up into small bite size chunks. Figure out how to divide things up into 1 to 2 hour tasks that people can grab and complete on a Saturday morning. Large tasks that can take all day may never get into someone's schedule. On the other hand, the pride of marking an issue closed can motivate an individual to tackle another small task.

I haven't looked at the issue queue yet, but we could try breaking up tickets into components, site sections, and task types. That can help enable a contributor to create a view that is targeted on the types of tasks that are good for them. We can link to advanced searches based on this categorization and promote those views for the different types of contributors... like themers, designers, module developers, etc. Even more focused than that, people can choose to really dive into a certain section of the site that motivates them.

Potentially, if we provide a view that has "exciting" things for a contributor in the list, it may avoid the chance of large or overwhelming tasks turning them off.

So, you want this done by September 1st? We may need to plan it. Divide it up into chunks and create goals. Create deadlines of reasonable-to-accomplish tasks. If we have two week development phases, we have time for 5 mini deadlines before September. We can break things into tasks, divide into milestones, and assign tasks to individuals. If we all keep communicating openly and participate in IRC, we could potentially initiate enough inspiration and motivation for people to work to complete the milestone.

And lastly... development process. Can you explain the current process in detail, and all of the problems you are encountering? What is your wishlist? How do you want things to work? (This may need to be in a different post, actually)

Re: Thoughts.

todd nienkerk's picture

@ceardach:

Thank you for the thoughtful and well-articulated response. I apologize that we as a team have not replied sooner. Time management is, as we've identified, one of the roadblocks to success.

I'd like to address your post point-by-point. Please bear in mind that I'm voicing my opinion -- I don't speak for the entire team. I hope they'll chime in so we can converge on some solutions.

Weekly meetings may be too often. I have found in other projects that if we meet weekly it starts eating into our productivity -- especially if we're not doing that project full time. [...]

I respectfully disagree. I dislike meetings in general, but regular, coordinated, time-boxed, and focused meetings can be incredibly helpful -- especially when we're working with a distributed team. Getting everyone "in the same room" for as little as 30 minutes a week is beneficial in three ways:

  • It will rapidly bring problems, bugs, conflicts, etc. to light. If we're all working on disjointed schedules throughout the day on IRC, it's likely we'll repeat each other's work, unknowingly break something, or undo someone's changes because they didn't understand why they were implemented a certain way.
  • It reminds us of our accountability to each other -- not just the project.
  • Finally, it's an important team-building exercise. Not only is voice communication more efficient than typing, but speaking to one another as a team helps avoid misunderstandings. (When you know how someone sounds when they talk, you're less inclined to think they're being rude.)

So, we should enable easy and casual communication. I love IRC. Is there a redesign implementers IRC channel? If not, maybe we should have one. [...]

Absolutely agreed. IRC should be home base. I'm working on securing #drupal-redesign.

Having a scheduled "virtual code sprint", or otherwise imagined as "virtual office hours", could be helpful. [...]

Agreed. This is absolutely essential. However, we should recognize the limitations of "office hours." They do not, for example, replace all-hands-on-deck meetings. Scheduling a weekly, half-hour meeting that works for as many people as possible is going to be tough; agreeing on four office hours per week will be impossible. Due to these scheduling restrictions, office hours may overlap, but they will be largely unique for each person. They will be essential for getting work done, but they won't necessarily facilitate communication.

That said, I'd like to try and overlap office hours as much as possible.

Another thing to do is to break tasks up into small bite size chunks. [...]

Yes! Now we just need someone to break apart those chunks for everyone. :)

[... W]e could try breaking up tickets into components, site sections, and task types. That can help enable a contributor to create a view that is targeted on the types of tasks that are good for them. [...]

This is a great idea. We should definitely make more use of tagging. Can anyone volunteer to sift through the issues and come up with 3-4 (or more?) tags to categorize them?

Potentially, if we provide a view that has "exciting" things for a contributor in the list, it may avoid the chance of large or overwhelming tasks turning them off.

Can you define "exciting"? :)

So, you want this done by September 1st? We may need to plan it. Divide it up into chunks and create goals. Create deadlines of reasonable-to-accomplish tasks. If we have two week development phases, we have time for 5 mini deadlines before September. We can break things into tasks, divide into milestones, and assign tasks to individuals. If we all keep communicating openly and participate in IRC, we could potentially initiate enough inspiration and motivation for people to work to complete the milestone.

Yes, yes, yes, and yes. And yes!

And lastly... development process. Can you explain the current process in detail, and all of the problems you are encountering? What is your wishlist? How do you want things to work? (This may need to be in a different post, actually)

The purpose of this post is to do exactly this: detail all of the problems and roadblocks we're encountering. I've identified those that bug me the most above. Others will need to step in and voice their concerns.

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

I'd love to do bug triage

ceardach's picture

I'd love to do bug triage and organizing. My road block for commencing that is a testing instance. http://staging.dosprint.org/ has been down for over a week.

This is my development process wishlist:

  • A continuously available, sanitized, Drupal.org database
    • use a naming scheme that lists the compatible svn revision
    • use rsync to grab the database (on the assumption the db is in the range of gigabytes)
  • An automatically updated integration and testing instance
    • A script that will update SVN once a day during a down time
    • Automatically grab the compatible sanitized-database and restore it

Infrastructure

todd nienkerk's picture

Sadly, I think staging.dosprint.org has been down for several weeks -- perhaps even months.

I this these are all good ideas, but we should consult a developer with regards to automatic SVN updates. They may object simply because some commits may require changes to the site's configuration that could appear to break the site if an update happened while someone was away. Instead, they'd probably prefer manual updates. Luckily, this is easy to do assuming enough people have proper credentials.

Ideally, there will be several theming dev instances -- theme1.dosprint, theme2.dosprint, etc. -- that themers can hop on and off of. I've also heard that especially dedicated contributors will receive their own instances. I like both ideas.

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

Reduce/eliminate db config

ceardach's picture

some commits may require changes to the site's configuration

I recommend that we minimize, towards eliminating, the need to make any configuration changes through the GUI. It is possible to do 100% development through code, and use update hooks for db configuration. Maybe we could pull in Sacha for assistance in this area.

It'll make the database 10x easier to manage, and the upgrade would then, theoretically "just work" when we're ready to deploy.

You're right

todd nienkerk's picture

You're absolutely right. All config changes should be made in an install or update hook. Good call.

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

Need a place to store asset files

todd nienkerk's picture

We need to find a better way to store and share assets (Photoshop/Illustrator comps, icons, etc.). Currently, it's stored in the /assets directory of the SVN repository, which essentially hides it from people who don't have SVN access (or can't figure it out).

Options:

  • Dropbox appears to be free so long as we stay below a certain amount of disk usage. There may be problems allowing multiple users, however.
  • Get a simple SFTP subdomain and tie access to a Drupal.org account. Problem: this requires constant help from the infrastructure team. We'd rather cut through the tape and manage our own permissions.

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

Dropbox works fine for many users

Mark Boulton's picture

Todd - just thought I'd drop by (no pun intended)

Dropbox works fine for many users (so I'm told). The only thing is, you're limited to 2gb for the free version. Even if one person upgrades to pro, this doesn't have the same effect for everyone - they still have 2gb unless they upgrade too.

I've used it on a fair few projects with a max of 12 users (although I'm told it scales to many more than this pretty well). I'm sure I speak for most designers in saying this would certainly be preferable compared to SVN. :)

Cheers,
Mark

Dropbox isn't what we want

aaron stanush's picture

Dropbox is really more for syncing files rather than selective file storage. When a user is given access to a Dropbox account it creates a mirrored copy of that folder on the user's computer. This means that all of the files from the folder will be automatically copied to the user's computer – obviously, this is not the behavior we want.

We really want something that is more similar to a FTP server/network drive. But Todd's right, we need to be able to manage our own system, rather than relying on the infrastructure team.

A few of the requirements we need are:
* Adequate storage (2GB seems plenty, our assets only total 2% of that right now)
* Multi-user support w/ simple registration and management

I started searching for providers and came up with the following. Though I haven't used any of them and I'm not sure if they satisfy our needs. Anybody used these or have other suggestions?


Aaron Stanush
Co-founder, Four Kitchens | IRC: astanush

Wuala FTW?

aaron stanush's picture

Wuala seems to be a pretty good solution for sharing the Photoshop/Illustrator comps with more designers. It allows us to make the files publicly available to download, while requiring group approval to upload files. There is a basic web file-browsing interface, as well as a Javascript window/app for drag-and-drop uploading.

http://www.wuala.com/drupal-redesign

Designers! Try it out and let me know if you think this solution is lame or awesome.


Aaron Stanush
Co-founder and Designer at Four Kitchens

My team is looking for the

nathanraft's picture

My team is looking for the same kind of solution and have tested a bunch of the available options. We have tried jungle disk. It is pretty flexible and the price is right. We have talked to Jungle disk about integration with Drupal a bit but just started the conversation.

Also check out Zumodrive. They are comparable in features to Dropbox but you can choose which files you want to download which is a real benifit. We have also talked to them about their upcoming API.

Drop me an email if you would like to set up a time to chat and get a more complete summary of what we have found out.

In regards to SVN -- why is

ceardach's picture

In regards to SVN -- why is it hidden to anonymous users? Why do they not have read-only access? I think requiring infrastructure assistance for each potential contributor is effecting the amount of contributors we'll have. No one can browse the code. My coworker applied for SVN access awhile ago, and hasn't gotten it. I applied last night, but I now have to wait until someone approves me.

But at least if people can browse the code, they can submit patches to the issue queue...

Dropbox is probably fine. I'd say the largest potential problem is someone deleting everything, or one person overwriting another person's work.

Another idea is to create a quick website we can upload files to.

SVN: My $0.02

todd nienkerk's picture

@ceardach:

Regarding SVN, does Keiran's reply makes sense? I agree with him, but I want to make sure others understand why.

  • Read-only SVN access will result in people ripping off the theme and using it elsewhere, which dilutes the brand and perpetuates the "all Drupal sites look the same" problem.
  • As Kieran mentioned, we have liberally granted out SVN access in the past and have seen little follow-through. All those inactive users with access to the new Drupal.org theme and modules makes the infrastructure folks a bit nervous -- and probably a bit reluctant to continue granting designers/themers access. To solve this, I propose we check access logs and temporarily suspend all SVN accounts that haven't accessed the repository in, say, two months.
  • Kieran is also trying to gather some dedicated members of the infrastructure team who an commit to granting people SVN access and other logistical issues in a timely manner. This is a problem, and we hope that tapping a few dedicated people will solve it.
  • However, as Aaron mentions below, we have identified the need to move the assets -- comps, icons, page layouts, etc. -- out of SVN and into a simple file upload/download utility that allows us to quickly manage access. Since most designers won't need to touch the actual theme, this should solve the "I don't have access and can't work on anything" problem.

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

Yes, Keiran's reply made

ceardach's picture

Yes, Keiran's reply made sense, and the whole situation is a huge bummer. In the end of the day, we're going to likely keep having problems because it's hard to have an open source development team without the open source part. People can't even post patches to the issue queue :(

I was talking about this with my coworker, Ben, last night. We were thinking that if people can't contribute code, they should be able to contribute to testing and bug triage so they can do something while waiting to be approved, or as a means to prove their motivation. With that, we'll need a continuous testing and integration instance.

Depends on what your role is

todd nienkerk's picture

The people most likely to commit large amounts of code -- not to mention 99% of all patches -- are developers, and they are all familiar with version control. I think we have the developer situation handled, as there are plenty of well-known people already on the job. (But then again, I'm not really part of the development side of the redesign.)

Regarding patches, does the redesign issue queue forbid patches entirely? It would surprise me if it did.

What we really need are designers and themers who can fill in the gaps in Mark Boulton's prototype. Once we've chosen a file storage and sharing solution, SVN access will be unnecessary.

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

Why SVN is hidden

Amazon's picture

Drupal.org theme is meant to be unique to Drupal.org. If we open it up to everyone, then we are going to see half done or poorly done implementations of the Drupal.org theme everywhere. We already have a bunch of user groups, and local groups implementing the theme for their local sites.

The idea of adding SVN was to open up the infrastructure to many contributors and make contributing fairly easy. The problem is we have a lot of people asking for SVN access, but few indicating they have time, have the skills, or are likely to keep contributing. So we are getting a lot of requests, and insufficient follow through.

We've given out SVN read access fairly liberally, to some 50 people or more and only a handful have actually participated in the issues. So throwing open the SVN isn't going to solve the problem alone. We need a combination of the theme access, an access to an installed copy of Drupal.org.

I sent out a blast message to try to get some dedicated infrastructure folks for the redesign. I'll try and follow through on giving them the necessary permissions and access.

Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

Assets vs. theme

aaron stanush's picture

To clarify, we're just talking about making the Photoshop/Illustrator assets open to more people, not the actual Drupal theme. I think we can all agree that the theme should be kept in SVN.


Aaron Stanush
Co-founder and Designer at Four Kitchens

looks like good plan with

mortendk's picture

looks like good plan with the weekly meetup and the put in 4 hours / week so we get some work done.
i would higly recommend a time of day that is not US centric (not that i dont like to be up at 4 in the morning, but not each week!)

To have a place where we can share all out psd design mockups etc isnt the easiest just to create an account somewhere or maybe get an account from the new project tool from dev seed?

i would love to se some PM with a whip and some strict deadlines so we can get this ball rolling, cause i know how i work ... postponing everything untill the last minute...

/morten.dk king of rock
morten.dk | geek Royale

/morten.dk king of rock
morten.dk | geek Royale

Agreed

todd nienkerk's picture

I agree that we need weekly meetups/check-ins/insert-your-favorite-meeting-euphemism-here. We'll try to schedule it for the morning in the US and afternoon in Europe.

Regarding assets (PSDs, AIs, etc.), check out Aaron's list of possible file-sharing solutions. In a follow-up, he suggest Wuala may be what we're looking for. He's invited us to try it for ourselves and provide feedback.

http://www.wuala.com/drupal-redesign

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

As one of the

damien tournoud's picture

As one of the "infrastructure folks", the idea of having a separate drop box for "assets" makes me a little nervous. Those files will be out in the wild without neither access nor version control. There are (for now) no limitations on the traffic or on the size of the files stored in the subversion repository, why not using that?

Damien Tournoud
http://drupalfr.org

Damien Tournoud

SVN is a barrier to entry

todd nienkerk's picture

First, anyone who needs access to the assets -- comps, visual diagrams, etc. -- has to apply for an SVN account, wait to be approved, and learn how to check out files. These are all huge barriers to entry for all designers, most of whom have never used version control before. None of us have the time to teach SVN to remote designers, nor do they have the desire to learn.

Second, we don't need version control for these documents. Designers are accustomed to creating multiple iterations of files for major changes, so it matches the typical design workflow. Besides, they're all binaries -- PSDs, AIs, SVGs, PDFs, -- so we can't graft changes.

Third, the drop-box utilities we're looking at will not necessarily make the files publicly accessible. They will likely require credentials of some kind and would be administered by a handful of responsible designers/themers.

Fourth, it doesn't matter if the assets are publicly accessible or "out in the wild." They already are, for the most part, and have been from the start. MB's Visual Style Guide itself is a collection of assets, and it links directly to manuscripts, the wordmark, templates, and page furniture. These and all other assets are harmless. The theme and module files -- the stuff that actually makes Drupal.org work -- should obviously not be publicly accessible.

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

Fair enough. In that case we

damien tournoud's picture

Fair enough. In that case we just need to be sure to keep some regular backups of those files.

Damien Tournoud
http://drupalfr.org

Damien Tournoud

Absolutely

todd nienkerk's picture

Regardless of what we pick, we'll be sure to regularly back up everything -- perhaps to an area of SVN not typically checked out by themers? It'd be annoying to download a huge library of assets each time.

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

Script it

ceardach's picture

I agree we need backups, and also a bit of version control... in addition to ease of use :)

I think that we could set a script that would commit all changes to SVN once or twice a day automatically.

Dedicated IRC Channel?

ceardach's picture

What are your opinions on using a dedicated IRC channel? Something like #drupal-redesign?

I was in #drupal-infrastructure for the day, and although the load was low, there was still significant discussion on system administration, and zero discussion about the actual redesign.

I also think that the cross talk could be distracting if we try to have redesign-specific conversation in the infrastructure channel when other topics of conversation could pop up.

A topic-specific channel would be helpful... and more non-techie friendly.

#drupal-redesign

todd nienkerk's picture

I absolutely agree that we need a dedicated channel. It's always struck me as strange that we were using the infrastructure team's channel to do design and theming work. DamZ in #drupal-infrastructure told me to contact chx regarding a new, officially recognized IRC channel. (By "officially recognized," I mean adding Druplicon bot support and anything else "official" I may not be aware of.)

I'll keep everyone posted.

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

I'm there.

ceardach's picture

I've already put #drupal-redesign on my auto join for IRC. So... I'm already there if anyone wants to talk!

#drupal-redesign is UP

todd nienkerk's picture

Channel #drupal-redesign is now official on irc.freenode.net. Join!

Todd Ross Nienkerk
Co-founder, Four Kitchens | IRC: toddross

Todd Ross Nienkerk
Digital Strategist and Partner
Four Kitchens: Big ideas for the web
IRC: toddross

quid.oblitus's picture

A very good thought - you don't want the mechanics to get in the way when you've got a distributed design effort.

You might consider looking over some of the basecamp project management tools - not free, but available and full-featured - not drupal, but I don't see that that is or should be a total disqualifier.

I'm leaning against using a

ceardach's picture

I'm leaning against using a 3rd party tool, and sticking with g.d.o and d.o for as much as possible.

I would absolutely love to use Trac, since it has awesome flexible abilities to manage tickets (components, blockers and milestones! ::swoon::) and create views on the fly, and views targeted towards the viewing user (amplified by the ability to CC users). However... all communication really has to be in an open an archiveable place.

I am going to do my very best to maximize the tools we have before looking towards another source. And that other source I would hope would allow us to enhance the g.d.o and d.o tools available to us before considering replacements.

Trac is my weapon of choice

lisarex's picture

Trac is my weapon of choice too. It's so powerful and flexible and most importantly, it's efficient. And a built-in wiki....awesome. Glad I'm not the only one with a Trac obsession!

Given that we're in different timezones, have limited time and a deadline, does the efficient solution need to be considered? I honestly don't know how efficient g.d.o and d.o tools are compared to Trac, though.

When you say all communication must be open and archiveable... I don't suppose any of the Trac export options are useful? It'll have ticket summary, description, and all the categorization we specify, but not the comments and attachments (which admittedly is were a lot of value lies). However, when I've managed big Trac-based projects in the past, I've updated the description with final decisions/resolution/ rationale before closing the ticket, so it's available to folks via RSS (with logins) and as CSV to anyone else.

Or we create Trac logins for those that request them. I don't mind doing that.

==================================
http://about.me/lisarex