seeking CVS support

mroswell's picture

I have gone to meetups asking this question, asked on IRC, and tried stuff on my own, without success. Maybe Drupalchix can help.

In essence, I need to be able to routinely grep all Drupal 5 contributed modules. So, naturally, I wanted to download them all (because I haven't found another way to grep them all).

The following command unfortunately gets me a slew of things with info files containing "core = 6.x"

cvs -z6 -d:pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal-contrib checkout -d DRUPAL-5 contributions

I do get a nice DRUPAL-5 subdirectory with the above command, but the contents seem to actually be drupal 6 code.

Can anyone help me with task?

I understand it may be generally frowned upon to download all of contrib, but I do need to be able to grep all of Drupal 5 contributed modules for my current project. Some bandwidth will be saved if I do it right the first time, rather than running a number of commands that get me the wrong results.

Thanks!

Margie

Comments

I think this will do what you want

mlncn's picture

cvs -z6 -d:pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal-contrib checkout -r DRUPAL-5 -d alld5modules contributions/modules

(But I don't have the hard drive space to check!)

In essence you were putting after the "-d" (which is lets you name the directory you want to create for the checkout) what you want after a "-r" which lets you name the version.

benjamin, Agaric Design Collective

benjamin, agaric

Thanks.

mroswell's picture

Thanks very much. That did the trick.

Some notes:

grep -r "core = 6.x" * | wc -l
i.e., basically: "count the number of files (lines) containing 'core = 6.x'"

yielded 936 results with my flawed CVS command,
and, thankfully, only 20, apparently some odd outliers, with Benjamin's much more helpful solution.

Interesting: I did a Google search just now for:
alld5modules

and found zero results.

I appreciate your insight!

Margie

why missing?

mroswell's picture

I just noticed that this project was missed in my CVS download. (I used Benjamin's very helpful command, above.)
http://drupal.org/project/phoneblogz

What would account for that?

It makes me wonder if other projects may also have been missed.

mlncn's picture

It follows the "-d" and can be any name you want. I gave it such an odd name to try to emphasize that fact :-P

It turns out this line only gets the releases of modules with the exact tag DRUPAL-5, which is what any module with a Drupal 5 development branch should call it. It does not get modules, such as phoneblogz, tagged only with a specific release.

cvs -z6 -d:pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal-contrib checkout -r DRUPAL-5 -d alld5modules contributions/modules:

I have no idea what or if one can use instead of "DRUPAL-5" to get, say, all the latest official releases of a module in the Drupal 5 development branch.

So that's why there's a very out of date phone module release downloaded:

phone 5.x-1.x-dev
5.x
thierry_gd - May 2, 2007 - 20:38
Nightly development snapshot from CVS branch: DRUPAL-5
Download: phone-5.x-1.x-dev.tar.gz

But no phoneblogz, which has only this listed for 5 in all releases:

phoneblogz 5.x-1.3
5.x
mattb - April 30, 2007 - 19:04
Official release from CVS tag: DRUPAL-5--1-3
Download: phoneblogz-5.x-1.3.tar.gz

Not sure where to go from here, except to ask a real CVS expert or to see if the Drush module (which communicates with Drupal.org to get latest release information when downloading modules individually) has an option to download all modules.

benjamin, Agaric Design Collective

benjamin, agaric

You can use ftp(1) to

mauror's picture

You can use ftp(1) to download the tarballs:

  • connect to ftp://ftp.osuosl.org anonymously (ftp ftp.osuosl.org)
  • cd to /pub/drupal/files/projects (cd pub/drupal/files/projects)
  • turn glob on, binary on etc... (glob)
  • mget *-5.*

This should do...

Not sure...

mauror's picture

Not sure it can be useful, but I just stumbled on this:
http://drupal.org/node/306451#comment-1004051 drupalpackagelist.sh: full available package list, quick'n'dirty

Sometimes you don't want to throw on a search engine to crawl for a certain package. Or you just want to have a more condensed overview about the cool things you want to install. This script helps in that it downloads the package file listing from the drupal server, converts the html to text, filters it by your preferred drupal-version, removes the first column from the output and writes it to a file. Very suited for grepping for packages.

wget -O - http://ftp.drupal.org/files/projects/|html2text -width 150 -nobs|'grep' "-6.x-"|sed 's/[[   ]] //' >drupalpackagelist.txt

Hard problem, no great solution

dww's picture

There's really no great way to do this, since you're basically at the mercy of contrib maintainers doing sane things (always a risky position to be in). As you've found even if you check out the DRUPAL-5 branch for all projects, some of those actually contain D6 code, since maintainers seem to regularly get confused or otherwise mess up and commit stuff to the wrong branch(es). In spite of tons of effort to the contrary, lots of maintainers don't follow the recommended best practices for how to use branches and tags in CVS, so things are often quite a mess. Furthermore, some projects have DRUPAL-5--2 branches (or higher) where the latest D5 code lives. And, some projects still use HEAD for their latest D5 code, if they haven't been ported to D6 yet. These last two are fully supported approaches, not just regressions from careless maintainers (although the thing about using HEAD is probably going to go away as a recommended practice, see cvs contrib procedure for more.

I don't believe drush can solve this yet, since it currently relies on update_status, and that only queries for releases for your currently installed modules (and in D6, themes). There's an issue open about providing a list of all projects that tools like drush could use, and we rolled it out for a day, but it ended up breaking update_status so we had to revert it. The patch still needs to be reworked. :(