Show me your logs

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
jpetso's picture

Er, right, this update post has taken longer than it should have. I've been active on the code though, and this time I even have a screenshot for you. Without further ado, I present Commit Log:

Screenshot of Commit Log

Commit Log is essentially the same as drupal.org/cvs, only that it's built on the Version Control API instead of direct database queries. It's also more extensive in its code base, quite flexible in themability because the output is split into a good number of theme_*() functions, and most importantly it displays commit logs in a more VCS independent way. That is, it knows about atomic commits, uses either commitlog-internal links or repository viewer URLs where appropriate, and has some other niceties that should make commit log inspections enjoyable and cross-VCS.

As dww rightfully noted, everyone using version control systems is going to need a commit log, but I agreed with AjK not to include non-API functionality in the Version Control API module, so Commit Log is now a separate module but shipped with Version Control API as a subfolder.

Groundwork

The CVS commits that you can see on the screenshot are, of course, real ones from my test CVS setup. And that in turn means that at least the most important of the xcvs-* scripts, xcvs-loginfo.php, has been ported to insert data into Version Control API's database tables. I also made use of aclight's achievements on bootstrapping Drupal from those hook scripts, so it's possible to use all functionality of Drupal and its modules from there. That's actually necessary because of Version Control API's modular delegation scheme.

As a necessity, the important parts of the CVS backend module have been implemented. The CVS backend is not quite capable to act as repository browser backend yet, and all of the admin forms are still missing (actually, the repository was the one thing that I manually entered into the database), but it already works great as data provider for commit logs, and is nearly on par with the analogous parts of cvs.module. Well, some more love is needed still.

Getting Commit Log to work also meant implementing Version Control API's central function, versioncontrol_get_commits(), which takes an array of constraints and returns an array of commits that the you can use for your purposes. For example, displaying them chronologically on a page. To a certain degree, Commit Log is the direct frontend to versioncontrol_get_commits(), and exposes much of its functionality with filters that you can specify in the URL. (It wouldn't be too hard to create a GUI for it, but that's really secondary now, and as commits are not nodes you don't get the Views stuff for free.)

Neatness

So Commit Log can filter by branch. Or by file/directory. Or by version control system. Or by repository. Or by commit id. Or by committer. Or, if the project node integration module is enabled, by project node id and/or "all projects of this maintainer" as well. Or by any combination of those, optionally with multiple values per filter constraint. Of course, cvs.module does most of this as well, but I still find it very cool. It also does paging.

But what's really cool with all this is not how good it works, but how relatively simple it is to write a basic backend. The CVS backend currently has 230 LoC (plus comments and blank lines which blow it up to 523 lines, erm), and that will even decrease a bit in the near future when I move user account associations from the backend to the Version Control API. Sure, the 207 LoC of the commit capturing hook scripts add up to this, but that's enough to have a fully-functional commit log.

SoC is done!

Well, NOT. A good amount of other stuff is still pending:

  • Admin forms for repository and account management. Done.
  • Project node integration is still in its beginnings, and needs lots of work. Done.
  • Account motivation message management and approval. Done.
  • Centralize username/uid associations, and make repository viewer URLs pluggable instead. (Given, the latter one has low priority.) Done. (At least, the database structure and overall architecture for pluggable viewer URLs is there, it can be implemented anytime without large changes.)
  • Tag support for the CVS backend (which has been left out for the moment) and a few more functions in order to reach mandatory backend API completeness. Done.
  • Commit restrictions. Done.
  • Port log parsing from cvs.module to the CVS backend. Done.
  • Lots of API fixes, and some additions. Done.
  • project_release integration, as I think the generic project node integration module won't do for this.
  • More optimization in order to cut down on the number of SQL queries.

Phew, that's quite a bit. Some of those will only take a few hours, but most will need more than a day. Let's see how many I can get done. API fixes and additions will mainly be made while working on the other items, as necessary.

Next up

I think this list can serve as a good todo list, the most important thing for me is to stay focused and actually get stuff done. The schedule from my original proposal doesn't help very much now, as it essentially says something like "move code from cvs.module to the new modules" for the rest of the SoC. That's of course the idea, but I didn't know the tasks as good as I do now, and that code can't always be moved directly but needs partial rewrites. (See Commit Log's code for a good example.) Subsequently, I'm rather going to pick my challenges out of this new list, starting with:

  • Centralize username/uid associations, provide appropriate API functions with implementations. Target date: tomorrow (Saturday, 28th).
  • Admin forms for repository management, and good parts of the account management forms (need to investigate those a bit further). Target date: Tuesday, 31st.

Let's see how this works out.

Comments

Fantastic, keep up the good work!

dww's picture

This is all looking great, thanks! A few ideas/comments:

1) Tag support for the commit log would be great. E.g. "show me all the commits between tag 1 and tag 2 on branch X".

2) Myself and hunmonk have been committing some improvements to the cvs.module, and more are on the way. I hope we can stay in sync on these things. Let me know how you want to try to handle that.

3) I hope your uid/username centralization takes into account that users might need different usernames for different VCS repos and backends.

4) "Port log parsing from cvs.module to the CVS backend." I'd move this to the bottom of your list. It's not needed if you have xcvs-*

5) "project_release integration" This is definitely going to be a multi-day task. However, it's critical, especially for deploying any of this on d.o (which is definitely the goal). That said, it'd probably be wise to get some of the other low-hanging fruit done first, so you don't get bogged down in this. However, it'd be great if you could start to strategize about this a little and see how much you think you could get done.

6) "commit restrictions" This will also be critical for d.o deployment, but it should be a lot easier. The UI for this probably can/should belong directly in the vcs_project.module, and should be very generic for any/all VCS's associated with a given project. However, the implementation is obviously incredibly backend-specific, so it seems you just need a good API for each backend to ask "should user X be allowed to tag/commit to file Y?".

Thanks again, this is incredibly exciting work!
-Derek

Thanks, and some remarks

jpetso's picture

Thanks for the appreciation and the great input!

1) Tag support for the commit log would be great. E.g. "show me all the commits between tag 1 and tag 2 on branch X".

That's a terribly nice idea. It's not completely possible with the current state of the API (we don't know about the time of a tag yet) but it will be possible with a bit of tweaking of the tag/branch retrieval functions' return value. Might not be just as easy to filter the commits as they can have the same tag at different times in different directories, but it should be doable.

2) Myself and hunmonk have been committing some improvements to the cvs.module, and more are on the way. I hope we can stay in sync on these things. Let me know how you want to try to handle that.

Agreed, we want to do that. I've already been monitoring the cvslog commits, and I'll be incorporating your fixes to my VCS modules where applicable. Until now, that was only the .htaccess file, but there may be more in the future. I'm taking care that we stay in sync.

3) I hope your uid/username centralization takes into account that users might need different usernames for different VCS repos and backends.

Sure, it already did so before I centralized it, only that it was a callback for the backend instead of real code. It's much nicer now, and the generic account info consists of uid, username and repository. (The latter one includes the backend, of course.) Backends can add info for their private usage (user passwords, that is) but in this case I think the additional info won't be needed outside of the backend, so for now there's no way for outside API users to get the password. Let me know if you can think of a use case where that would be needed.

4) "Port log parsing from cvs.module to the CVS backend." I'd move this to the bottom of your list. It's not needed if you have xcvs-*

Yes, agreed, that's very low priority. I also agree with point 5, which is very high priority, and I plan to attack project node integration as soon as I've got the generic repository/account admin screens done.

6) "commit restrictions" This will also be critical for d.o deployment, but it should be a lot easier. The UI for this probably can/should belong directly in the vcs_project.module, and should be very generic for any/all VCS's associated with a given project. However, the implementation is obviously incredibly backend-specific, so it seems you just need a good API for each backend to ask "should user X be allowed to tag/commit to file Y?".

I'm not sure the backend should be bothered with this. I thought of an API function versioncontrol_has_permission($op={'commit','tag','branch'}) that provides two hooks - something like 'permit' for "always allowed" overrides, and 'deny' for "not allowed" ones. The xcvs-* scripts would call the appropriate function to determine if the commit/tag/branch may be created, and versioncontrol_project or any drupal.org specific modules or whatever can hook into those hooks and determine commit policy for the specific site.

The backends only are specifically involved because branches or tags may follow different conventions for different repositories, but even then I'd say that's not backend specific and can be done without requiring each backend to write their own restriction policy.

more on commit restrictitions

dww's picture

I consider the xcvs-* scripts part of the backend. That's what I meant -- the backend is where you actually implement the restrictions. All the API has to do is provide the has_permission() function. Looks like we're in fact on the same page, even if we're using different terminology. ;)

So, if the xcvs-* scripts aren't part of the backend, what are they? ;)

Thanks,
-Derek

xcvs-* belong to...

jpetso's picture

Yes, xcvs-* is part of the backend, of course. I only interpreted "implementing commit restrictions" as figuring out which ones to deny, rather than calling a simple API function - but yeah, it all comes down to the same solution :)

terminology

dww's picture

Ok, good. ;) By "implement commit restrictions" I meant actually preventing the commit. In CVS, that's done by having the commitinfo hook return a non-0 result. In SVN, there's some other hook or something, etc, etc. The main API module just needs to manage the restrictions (mapping drupal users to project ids, etc), and the backends have to implement them. Make sense? ;)

Thanks,
-Derek

Issue tracking and software releases

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds:

Hot content this week