Posted by haxney on June 22, 2009 at 12:06am
A good talk this week, with a lot of stuff to say.
-
DamZ dropped by to discuss some of the issues and strategies surrounding the eventual migration of drupal.org to the versioncontrol api. Maintaining the existing links could be a challenge, since they are dependent on the particular entries in cvs.module's cid field.
-
chrono325 and corni talked about the difficulties of authorization in Git, since it does not provide the repository hooks with a username like SVN does. The solution chrono325 came up with is detailed in his weekly update.
-
corni and jpetso talked about what would be needed for a comprehensive set of tests for versioncontrol. The idea would be to have a testing backend (called "testvcs") which would be a mock object and allow for testing the general versioncontrol API. Additionally, chrono325 volunteered to help write a standard battery of test commits to run against the different backends.
-
jpetso, corni, and chrono325 talked about adding additional capabilities to the backends to specify things which would be useful for the test commits, such as "cloneable", "merges", and "n-way merges". A test which tested the backend's handling of a merge would be skipped for VCSs which did not support merging (like CVS or Subversion).
[Sun Jun 21 2009]
*** You have joined channel #drupal-vcs [14:16]
*** Topic for #drupal-vcs:
http://github.com/sdboyer/drupal-git-scripts/tree/master
*** #drupal-vcs: topic set by sirkitree, 12:40:28 2009/05/13
*** Users on #drupal-vcs: chrono325 corni marvil07 skiquel stodge Druplicon
neclimdul psynaptic
*** #drupal-vcs modes: +sn
*** #drupal-vcs was created on Friday 2009/05/08 01:36:39 AM
*** #drupal-vcs: [freenode-info] why register and identify? your IRC nick is
how people know you. http://freenode.net/faq.shtml#nicksetup
<corni> hello chrono325
<chrono325> hey
*** DamZ (n=damz@drupal.org/user/22211/view) has joined channel #drupal-vcs
[14:17]
<DamZ> and another #drupal-xx channel ;)
<corni> ;)
<chrono325> sorry, I lot track of time a bit, did I miss anything?
<corni> we haven't started yet
[14:18]
<corni> still 12 minutes to go
<corni> and jpetso is missing
<chrono325> ah
<marvil07> hi chrono325
[14:21]
<chrono325> marvil07: hey
<marvil07> corni: jpetso was here some time ago, and I suppose he would be
here on time :D
*** benjamin-agaric (n=I3IVIIVI@drupal.org/user/64383/view) has joined channel
#drupal-vcs
[14:23]
<corni> marvil07: I guess so too
[14:25]
<corni> oh, a drupal consultant is joining us?
[14:28]
<corni> :D
<corni> Druplicon: summon jpetso
[14:30]
*** jpetso (n=jpetso@212-183-44-97.adsl.highway.telekom.at) has joined channel
#drupal-vcs
<corni> thanks Druplicon!
<Druplicon> Awww, shucks!
<jpetso> yess!! i made it
<corni> jpetso: I like Druplicon
[14:32]
<corni> <corni> Druplicon: summon jpetso
<corni> * jpetso (n=jpetso@212-183-44-97.adsl.highway.telekom.at) has joined
#drupal-vcs
<corni> ;)
<jpetso> :D
<marvil07> yep, that was good :D
[14:33]
<jpetso> just finished with a training session at the local fire fighters
<marvil07> :o
<corni> do you participate there?
<jpetso> in fact, i'm still at our fire fighters' house :]
[14:34]
<jpetso> corni: yes, occasionally
<jpetso> anyways, it's :35 here, let's go
[14:35]
<marvil07> ok
<marvil07> jpetso: I tried to make the list with times, but I notice "that
list" is not completely clear, so I put it on the
WU(http://groups.drupal.org/node/23458) but unfinished, for
feedback ;)
<Druplicon> http://groups.drupal.org/node/23458 => WU4: Version Control API
and family changes => 2 IRC mentions
<corni> jpetso: DamZ pinged me 15min ago that he'd like to help, I just
forwarded him to #drupal-vcs and told him about our meeting :)
<marvil07> welcome DamZ
[14:36]
<DamZ> hey everyone
<chrono325> hi
<Druplicon> hey
<jpetso> hi DamZ, here we go again :]
<DamZ> I'm here for two reasons: (1) keeping an eye on a project that might be
one day deployed on d.o, and (2) understanding the project itself
better to maybe using it one day on one of our projects
[14:37]
<DamZ> full disclosure ;)
<jpetso> DamZ: in fact, i'm nearly done preparing it for d.o deployment
[14:38]
<jpetso> DamZ: it has been a long-running task, by now we're only left with
migration scripts and then we're good to go
<DamZ> jpetso: that's a really good news
<DamZ> jpetso: whenever you are ready, we could test run that on real data ;)
[14:39]
<jpetso> DamZ: btw, how does the process of testing this stuff work?
<jpetso> DamZ: do i get db access for, say, project.drupal.org, or will i stay
local with my own test server
[14:40]
<DamZ> jpetso: we have a full test environment
<DamZ> jpetso: we will need to get you access to d6.drupal.org
<jpetso> DamZ: well, i'm looking forward to that :)
[14:41]
<DamZ> SELECT COUNT(*) FROM cvs_messages;
<DamZ> >> 199771
<corni> jpetso: can you sum up what you've done wrt d.o deployment (just for
me ;))
<jpetso> DamZ: i need to get it done until July 13, because i can't tell how
much time i can put into it afterwards
<DamZ> ^ I hope you have fast migration scripts ;)
<corni> ?
<jpetso> DamZ: hunmonk started scripts based on update.php, with automatic
chunking of db tasks
[14:43]
<jpetso> i also hope the scripts are fast enough, but i haven't yet tried
running them
[14:44]
<DamZ> jpetso: our staging VM is not really fast, so I hope that it will not
take several days
<corni> why do you want to import the old cvs-messages?
[14:45]
<corni> is re-parsing them not an option?
<chrono325> corni: probably to preserve links
[14:46]
<jpetso> corni: no, because the cids (afterwards: vc_op_ids) would get lost
<corni> ah okay
<chrono325> it would suck to break all of the links to cvs messages.
<corni> except one would parse them and update them afterwards
[14:47]
<corni> but probably not easier to do than importing the old data...
<jpetso> let's see how fast they run, but it should be bearable - if not, we
can modify the scripts to convert a range of commits and perform the
bulk of the migration while d.o is running productive
[14:49]
<jpetso> but let's see.
<DamZ> we also have some constraints on d.o, like autoincrement = 2, I hope
nothing in the version control API is assuming that IDs are separated
by 1
[14:50]
<jpetso> corni: work until now has been focused on making Version Control API
do all the things that cvs.module does, with the same level of
scalability
[14:51]
<corni> :)
<jpetso> DamZ: nope, it's just serials.
<jpetso> DamZ: like Schema API array('type' => 'serial')
[14:52]
<DamZ> jpetso: perfect, then
<jpetso> anyways, I believe marvil07 is under time pressure
[14:53]
<jpetso> let's finish that part, maybe
[14:54]
* jpetso tries hard to concentrate while a movie is running at the same time
[14:55]
<corni> lol
<marvil07> (bis) jpetso: I tried to make the list with times, but I notice
"that list" is not completely clear, so I put it on the
WU(http://groups.drupal.org/node/23458) but unfinished, for
feedback ;)
[14:56]
<Druplicon> http://groups.drupal.org/node/23458 => WU4: Version Control API
and family changes => 3 IRC mentions
<jpetso> marvil07: so, how to test Commit Restrictions and Account Status?
[14:58]
<jpetso> marvil07: good question. at times like these, i wish we got
simpletests already
*** benjamin-agaric (n=I3IVIIVI@drupal.org/user/64383/view) has quit:
[14:59]
<marvil07> jpetso: so, maybe it's a good idea to do that this week
[15:00]
<jpetso> marvil07: chrono325 also suggested that he might write simpletests
instead of porting more backends
[15:01]
<jpetso> i thought about that, and i find that to be a good idea
<chrono325> plus, making the little bar fill up is SO MUCH FUN :)
<jpetso> totally.
<jpetso> so yeah, let's go testing. who wants to be first? :P
[15:02]
<jpetso> chrono325: how far have you come with the Git scripts already?
[15:03]
<DamZ> does it make sense to test the main module or not?
<jpetso> totally, the question is how to test the main module
[15:04]
<marvil07> about testing, for versioncontrol core, I tought using
<chrono325> jpetso: delayed. Git is more complicated than I had anticipated,
since each push can contain a number of commits. I wrote a php
interface for git fast-import to help with testing, and the first
pass of the update hook is done
<DamZ> versioncontrol_fakevcs comes to mind
<marvil07> http://groups.drupal.org/node/22332
<Druplicon> http://groups.drupal.org/node/22332 => Mock Web Service for
SimpleTest => 1 IRC mention
<marvil07> it started on http://groups.drupal.org/node/22176
[15:05]
<Druplicon> http://groups.drupal.org/node/22176 => sending a cookie in the
simpletest browser => 1 IRC mention
<marvil07> well.. the same idea
[15:06]
<jpetso> marvil07: why would it be necessary to refer to web services?
<marvil07> like DamZ said, use fakevcs
<marvil07> jpetso: sorry, I refer only the idea
[15:07]
<DamZ> the first step in testing the engine is to define a common dataset,
that could be expressed in all the backends
<DamZ> a set of commits, branches, tags, files, etc. that can be standardized
across backends
<DamZ> does that sound doable?
<marvil07> we should :D
[15:08]
<jpetso> DamZ: sounds good, yes
<DamZ> the idea being that you should be able to run the exact same test suite
on every backend
<jpetso> the FakeVCS backend might be an idea, but in order to get that going,
we'd need to make FakeVCS work as a real module rather than a series
of example implementations
<DamZ> on fakevcs to test the core modules itself first
<DamZ> then on each real backend, with a real repository implementing this
test dataset
<jpetso> the SVN backend now has working tests (thanks, chrono325) so
extending that would be the faster way
[15:09]
<DamZ> jpetso: it basically depends on what you want to test
[15:10]
<DamZ> jpetso: do you want to test importing data from the backends?
<jpetso> DamZ: essentially, the backends implement most functionality that can
be tested, so in addition to tests for data import they could also
test other modules' functionality
[15:12]
<jpetso> the SVN backend can do commit restrictions, so it could also test the
Commit Restrictions module
<jpetso> CVS likewise, only that no tests are written for that one yet
<DamZ> jpetso: k, so you are essentially testing the backends, in the hope
that it will also test the central module too
[15:14]
<DamZ> jpetso: that can work
<jpetso> i'm not sure if making the FakeVCS work like a proper backend is
worth the effort
<jpetso> better port the backend tests to other backends and make sure they
work on all of them
[15:15]
<jpetso> or stuff.
<marvil07> I see jpetso, but we also want to test if core itself is doing its
work right
<DamZ> jpetso: there are two issues with this approach: first you need to
duplicate all tests in each backend, second you have no standardized
set of test to help you when you are writing a new backend
[15:16]
<jpetso> yes, that's an issue
[15:17]
<jpetso> suggestions?
<Druplicon> bot-factoid-template-template-suggestons.tpl.php is
http://drupal.org/node/190815 => Core templates and suggestions
and http://drupal.org/node/223440 => Working with template
suggestions
<jpetso> dude, Druplicon
<DamZ> jpetso: if you can define a standard set of actions (ie. create a
repository, add/modify a file, commit, branch, tag), and a standard set
of tests (ie. test that after a commit, the file is there, etc.), you
could write the tests in an abstract way, and run that into the
different backends
[15:18]
<jpetso> DamZ: yes, that'll make sense
[15:19]
<jpetso> also, we can whip up a backend that, with little effort, implements
just creating a repository
<jpetso> with that, we could test repository creation and account management
tasks
[15:20]
<jpetso> and defer to the real backends for everything involving operations
themselves
<marvil07> gtg now, but I would read this later and implement tests on the
week for versioncontrol core :D
[15:21]
<marvil07> s/would/will/
<jpetso> marvil07: ok, then let's say you create a new fake backend
("TestVCS"?) with a minimal implementation so that creating/editing a
TestVCS repository will work
[15:22]
<jpetso> marvil07: for 6.x-1.x
<jpetso> marvil07: and based on that, you can write tests for the Account
Status module
[15:23]
<DamZ> unrelated question: did someone worked on an integration between
versioncontrol and activity (or some other kind of activity stream)?
[15:26]
<jpetso> DamZ: i'm not aware of anything in that direction
[15:27]
<jpetso> DamZ: how do you imagine such an integration?
<DamZ> jpetso: I don't know yet, but the basic idea is to have commits
available in an activity stream, as in the trac "timeline" and the
"activity" tab of redmine
[15:28]
<jpetso> DamZ: people can implement hook_versioncontrol_operation() which is
invoked every time a commit (or branch/tag creation) is recorded
[15:30]
<DamZ> jpetso: should be simple enough then
[15:31]
<DamZ> jpetso: thanks for the pointer
[15:32]
<jpetso> DamZ: other than that, versioncontrol_get_operation() can retrieve
commits with a set of filters (in reverse chronological order)
[15:33]
<jpetso> er, ..._operationS()
<jpetso> (damn, i can't concentrate)
[15:34]
<corni> how's that movie called?
[15:35]
<corni> :P
<jpetso> In drei Tagen bist du tot (horror movie)
<jpetso> chrono325: any news or stuff where you need assistance?
[15:36]
<chrono325> jpetso: well, right now I am working on testing the git
hook. There isn't really any particular part where I am stuck,
though any help in testing it would be great
[15:37]
<jpetso> help in testing, like in manual testing or with writing simpletests?
<corni> jpetso:i know the film. i always laughed about the accent, and that
youths still speak in such a way :D
[15:38]
<chrono325> The hardest thing at this point is that I have to figure out which
commits should be access checked
<chrono325> jpetso: simpletests
<jpetso> corni: ts, Germans
[15:39]
<corni> jpetso: jaja, aber euer dialekt ist nunmal zum wegschmeißen :P
* corni is afk, after i've fought enough for today with git rebase.
[15:41]
<chrono325> warum sprechen wir Deutsch?
<jpetso> dude! this is a serious gathering! ;P
[15:42]
<corni> I'll try another time to rebase the git repo...
<corni> :D
<corni> then i'll go so you can be serious if you want :P
<jpetso> corni: neh, don't take me serious, because i'm not :]
<jpetso> chrono325: i think the backend side of access checking should be
similar to centralized backends
[15:43]
<chrono325> jpetso: yes, except that with the centralized ones, the pre-commit
hook triggers on each commit, whereas with distributed, it only
happens on a push
<jpetso> so the task at hand is defining the set of commits, like DamZ
proposed?
<corni> chrono325: you know german?
<chrono325> yup, 5 years in school and a half year in Karlsruhe :)
[15:44]
<corni> chrono325: from which country do you come?
<chrono325> jpetso: and you have to be careful not to re-check commits that
someone else pushed
<chrono325> jpetso: US. But I took German in high school and spent a half year
in an exchange program in Germany
<jpetso> ah, right
[15:45]
<chrono325> jpetso: anyway, the problem is that for each push, you could be
pushing commits with a different author than the person pushing
<chrono325> so you can't just look at all the commits by the same author,
since I could push a commit authored by someone else, but since I
am pushing it, the push should still succeed.
[15:46]
<corni> chrono325: nice :) we could visit jpetso in austria and held a vcs
meeting entirely in german ;) just marvil07 would have a problem...
<jpetso> chrono325: yeah... in order to solve that issue, we would need the
author/committer distinction that marvil07 included in his branch
<corni> (no, i'm not serious anymore today, sorry ;))
<jpetso> :D
<chrono325> corni: unfortunately, my German isn't great (haven't really spoken
it in a few years) an it's even worse for technical stuff :)
[15:47]
<chrono325> jpetso: I didn't know that marvil07 had done that, nice
<corni> chrono325: we'll teach you ;)
<jpetso> chrono325: how about we defer author checks until that branch is
merged?
<corni> chrono325: have a look at http://groups.drupal.org/node/23458
<Druplicon> http://groups.drupal.org/node/23458 => WU4: Version Control API
and family changes => 4 IRC mentions
<chrono325> still, it is not just author/committer, but
author/committer/pusher
[15:48]
<jpetso> aargh
<jpetso> no, wait
<chrono325> since I might push a commit authored by one person and committed
by another.
<corni> chrono325: yeah but access checking is done for the pusher?
<chrono325> corni: it ought to be.
<chrono325> but git doesn't store any info in the tree about who the pusher is
[15:49]
<jpetso> author, in our terms, would be the person who initially commits the
commit
<chrono325> as it should, since that can change
<corni> chrono325: not even in the hooks?
<corni> aka hook invocations
<chrono325> what we could do is something like having an ssh authorized-keys
entry where it calls a wrapper and passes it the name of the
pusher
<jpetso> i mean, we can't go back to information that is outside the
information that the VCS provides (like "patch was not written by the
original committer")
[15:50]
<chrono325> corni: correct, since the hook script doesn't know where the push
is coming from (local filesystem, ssh, http, etc)
<corni> chrono325: anpther avriant is, that pushing is always allowed, when
you have an account for the repo
[15:51]
<corni> chrono325: but this is too easy, isn't it?
<corni> chrono325: normally, to be able to push ssh-keys etc have to be set up
already
<chrono325> corni: exactly
[15:52]
<chrono325> the problem is that there is no way to keep someone from claiming
a commit is authored by anyone they want
<chrono325> since "author" and "committer" are just strings you pass to git
config
[15:53]
<corni> I don't see the problem, as that's the way intended, isn't it?
<corni> eg webchick says that jpetso authored a patch, and commits it
<chrono325> I could make a commit and claim that it was authored by "admin
<admin@example.com>" and git would just accept that
<corni> later dries pushes this to the d.o master
<chrono325> yes, that is how it works, and that is how it should be
[15:54]
<chrono325> but it makes access checking harder
<jpetso> chrono325: so everything identifying your is your SSH key, right?
<corni> but how do you want to sanity-check the author?
<chrono325> compare this to subversion, where the only way to get SVN to say a
commit is by "chrono325" is to transmit the correct
username/password pair
<chrono325> hold on, afk
[15:55]
#drupal-vcs> /away brb
*** You have been marked as being away
<jpetso> that makes for a good occasion
*** jpetso (n=jpetso@212-183-44-97.adsl.highway.telekom.at) is now known as
jpetso|toilet
#drupal-vcs> /away
[15:56]
*** You are no longer marked as being away
<chrono325> jpetso|toilet: lol@toilet
<corni> I'd just allow arbitrary author entrys. When they are known, you can
map them to drupal users, when not, they're just displayed
<corni> you#re responsible enough to set the author field in a good way if
you've commit access
[15:57]
<chrono325> corni: I agree, but the problem is that you can't rely on them for
access checking
<corni> yep sure
<corni> but *what* do you want to protect?
<corni> branches, paths, files?
<chrono325> I man push access
<chrono325> all of the above, potentially
[15:58]
<corni> chrono325: push access is already granted if you're able to push to a
repo
<corni> chrono325: by doing some ssh-keys magic, for example
<corni> chrono325: or giving out an acc on a box
<chrono325> so would we just say that you need a public key, which grants you
full access to a particular repository?
*** jpetso|toilet (n=jpetso@212-183-44-97.adsl.highway.telekom.at) is now
known as jpetso
<chrono325> and not involve versioncontrol_api at all?
[15:59]
<corni> chrono325: you have to enter a public key which can be unlocked to
allow access to a repo for pushing.
<corni> chrono325: if you want to check for paths, branches etc i'd just go
with the committer
<corni> it's not 100%, but well...
<chrono325> I mean, we could always just say that the pusher has full write
access
[16:00]
<corni> chrono325: imho we should state some goals for what we would need
access checking with git repos
[16:01]
<corni> chrono325: except can push/cannot
<chrono325> corni: agreed.
[16:02]
<chrono325> I have the framework in place to check every commit being updated
for *something*, but I don't yet know exactly what we want to
check
[16:03]
<chrono325> right now, I think it checks that the author corresponds to a
username within the system, but that is not really ideal
<chrono325> in fact, it's probably worse than nothing, since it prevents
pushes of merges (where the author is different from the committer
or pusher)
[16:04]
<chrono325> a separate problem (which I think I've solved, but would need to
test to be sure) is figuring out which commits to check in a push
[16:05]
<chrono325> since a push can be a fast-forward (with a linear set of commits);
a non-fast-forward push, such as a rebase; or a push with merges.
[16:06]
<chrono325> we want to make sure to check only the commits which are being
added to the repository.
[16:07]
<corni> huh, that stuff is really complicated...
[16:08]
<jpetso> aww.
<jpetso> chrono325: what's your "solution" to the problem of which commits to
check?
<chrono325> jpetso: well, I use the old_ref..new_ref notation to get the list
of commits between the old and new ref
[16:09]
<chrono325> where the definition of "between" is kind of complicated, such as
for a rebase
<chrono325> also, in the case where a branch is changed, but does not
introduce commits, I need to make sure not to check any commits
[16:10]
*** benjamin-agaric (n=I3IVIIVI@drupal.org/user/64383/view) has joined channel
#drupal-vcs
<chrono325> so if you have branches "master", "next", and "testing", where
"next" and "testing" start out the same
<corni> I just vote for not access-checking individual commits, sorry
<corni> but i don't think we can solve the problems arising from this
[16:11]
<chrono325> and you have a push which changes "testing" from "next" to
"master", the revlist "next..master" would give you a bunch of
commits
<chrono325> but none of them are being added to the repository by the push, so
you don't want to check them
<corni> and in a DVCS access checking is imho not really needed
<chrono325> corni: that is true
[16:12]
<chrono325> I think I have a system which works, but I can't be sure until I
test it
<chrono325> hence the fast-import interface
<chrono325> create a bunch of repositories with different conditions and test
them all
<corni> chrono325: or can you propose a system of reliable access checking
which works for paths?
<chrono325> corni: well, right now all the backend is doing is figuring out
which commits to send to versioncontrol_has_write_access(), which
does the path checking itself
[16:13]
<corni> chrono325: yeah, but what do you want is there's a commit in the
middle of a push violating the access constraints?
[16:14]
<corni> chrono325: abort the whole push?
<chrono325> corni: yes.
<chrono325> corni: or rather, abort the updating of that ref
<jpetso> corni: yeah, but how does the pusher cope with it?
<chrono325> the 'update' hook of git does this automatically
[16:15]
<chrono325> so the hook triggers once for each ref being updated (label, in
versioncontrol_api lingo)
<corni> chrono325, jpetso: imho the best would be having the pusher for a ref,
and then having acvess checks based on branchs. this make imho the
most sense for dvcs
<chrono325> and if it returns a non-zero exit code, then it aborts the
updating of just that ref
<chrono325> corni: that is what I am doing (minus checking the pusher) if I am
understanding you correctly
[16:16]
<corni> I#ve gtg soon, if i just exit, i'll maybe bring up the topic tomorrow
again, and read in the log what you've talked about
<corni> chrono325: no, i mean just checking if a pusher is allowed to push to
branch x
[16:17]
<corni> chrono325: nothing further
<jpetso> the more i learn about DVCS commit access checks, the more I think
it's way too complex for the outcome
<corni> :D
<chrono325> corni: ah. I see. That may be the way to go
<chrono325> I think that in general, fine-grained access controls for DVCSs
don't really make much sense
<corni> chrono325: this leaves us without the commit logic which is way to
complicated to be 100% bullet-proof imho
[16:18]
<jpetso> yeah
<corni> yep
<corni> btw, what do the hooks tell you about the person who created a new
branch/deleted an old?
<chrono325> now if only I had known that before spending hours looking at the
documentation for git rev-list and rev-parse :S
[16:19]
<chrono325> the ref name, the old id, and the new id
<corni> cool...
<chrono325> for a deleted branch, the new id is 00000...
* marvil07 with the long read
<corni> :P
<chrono325> for a created branche, the old id is 0000...
<chrono325> oops /branche/branch/
[16:20]
<jpetso> so for a sensible system to work, we'd need to modify/extend the
write_access() hook to check labels without referring to commits
<corni> wait, under which user do the hooks execute?
<corni> push hooks
<chrono325> whichever user initiates the push
<chrono325> on the server
<corni> gotcha
[16:21]
<chrono325> so if you're doing an ssh-key thing, it would be the unix user
"git" (or whatever you use)
<corni> really?
<chrono325> what we could do is set an environment variable to the email of
the pusher
<corni> not the unix user of the one who logged in?
<corni> aren't this restricted unix accounts?
[16:22]
<corni> or just different logins to the git account?
<chrono325> it is the unix user who runs git receive-pack
<jpetso> chrono325: does that also apply to the non-Unix-user-based
authentication thing... what was it called, Gitosis?
<corni> jpetso: it is based on ssh-keys iirc
<chrono325> yeah, so what gitosis does is run ALL pushes as the unix user
"gitosis"
<jpetso> k
[16:23]
<jpetso> do we have access to the SSH key?
<corni> that'sworse, because with normal unix users we have the pusher
<chrono325> and when a remote user pushes, they execute the ssh-key command of
(something like) "gitosis shell -u <username>"
<corni> but do they still have an entry in /etc/passwd?
[16:24]
<chrono325> corni: no, there is only one entry
<chrono325> so the program "inDefero" (which is written in PHP) has the
following in its ".ssh/authorized_keys" file:
<chrono325> command="python /home/git/indefero/scripts/gitserve.py
<username>",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
ssh-rsa AAAAB3Nza... user@computer
[16:25]
<chrono325> where "/home/git/indefero" is the root of the installation
<corni> does gitosis sets an environment variable with the name of the pushing
user?
[16:26]
<chrono325> gitserve.py runs a gitserve.php and then "git shell"
[16:27]
<corni>
http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secu...
may help
<chrono325> check out the man page for git-shell(1)
<chrono325> corni: yeah, I used gitosis for a while, but found the
configuration to be annoying to maintain, so I switched to
indefero
[16:29]
<chrono325> but the auth part is the same
<corni> i don't know indefero, but does gitosis allow access checked based on
branches?
<corni> chrono325: and iirc your or marvil07's soc includes adding support for
one hosting system so you don't have to do that stuff manually
[16:30]
<jpetso> corni: marvil's
<chrono325> yay! :)
[16:31]
<chrono325> but yeah, getting the push access checking to work requires
knowing how user auth is done
<jpetso> :(
[16:32]
<chrono325> as opposed to svn, where the user auth is included in the commit
info
<chrono325> in svn, the author and the pusher are the same, so checking the
author automatically means checking the pusher
<chrono325> in git, they are different, and git doesn't record or know
anything about the pusher, so it is a bit more difficult
[16:33]
<corni> chrono325: can you put infrastructure in place which does branch
access checking when it get's a username, but just don't enable that
code until marvil is done with his integration ?
<chrono325> corni: sure. It will require some interaction with
.ssh/authorized-keys, and the requires writing user info to a file
[16:34]
<chrono325> well, at least, making it actually work would require that
<chrono325> I could just write it to assume that the script has been invoked
with the appropriate params, and then let marvil07 deal with
actually implementing that
[16:35]
<chrono325> luckily, there are a lot of examples of how to do it right,
gitosis, indefero, etc
<corni> yep, we'd just need the actual git hosting backend to do that
[16:36]
<corni> for unix users accs, it's easy
<corni> and eg for gitosis as second backend it should be still doable
<corni> marvil07: we probably want to allow different hosting backends in
vcs_git :P
<chrono325> well, I think we would re-implement the account access stuff that
gitosis does rather than using its actual code
[16:37]
<jpetso> thanks, i was just about to ask that
[16:38]
<jpetso> chrono325: but we can still track Git repos without relying on them
being managed by Drupal, right?
[16:39]
<corni> without access checking, jpetso
<jpetso> that's what i meant
<corni> you're never forced to use the hooks
<jpetso> sounds like an acceptable solution for me
<marvil07> corni: so, I think I miss something .. what do you mean with
hosting backend?
<chrono325> jpetso: oh yeah. The post-receive hook should be much more
straightforward
<corni> and you should be able to use the update hook without access checking
<chrono325> and if it checks a bit too much, then that's not a problem
<corni> marvil07: ways in which user accs are managed for repositories, aka
unix users, gitosis like solution, ...
[16:40]
<marvil07> I get it, ssh auth you mean or any other ways..
[16:41]
<jpetso> marvil07: essentially, chrono325 suggests we reimplement gitosis-like
functionality for versioncontrol_git
<jpetso> so that we can do the SSH key <-> Drupal user association by
ourselves
[16:42]
<corni> and i suggest to support one unix user<->one drupal user, too
<corni> for smaller projects
<corni> maybe in a seperate module
<jpetso> corni: i doubt you'll need access checks for smaller projects
<marvil07> yep jpetso, like github
[16:43]
<corni> jpetso: good point, though having a drag&drop-solution for an user
backend would be imho nice anyways
<chrono325> corni: one unix user per drupal user would make it possible to do
access control simply by chmod and groups
<corni> jpetso: how many of them we provide is another question :D
<corni> chrono325: :)
[16:44]
<corni> I've not yet thought of that possibility
<chrono325> as in, one group per project, one unix user/drupal user, and
drupal users with write access to a project are part of that
project's group
<chrono325> the problem is that it creates tons of unix users
[16:45]
<jpetso> on the other hand, doing all of that by ourselves would mean that the
webserver user needs write access to the repositories, no?
<corni> no?
<Druplicon> maybe?
<chrono325> lol@ druplicon
<corni> lol
<corni> the php scrip is executed by one git user which has access to the git
repos
<corni> not by the webserver
<corni> +t
<jpetso> k
<jpetso> but that script bootstraps Drupal
[16:46]
<chrono325> I think git supports pushing through http (though it is less
efficient, I think) so if we wanted, we could possibly insert
drupal in between there
<corni> it needs read access to the web server's file/the drupal installation
;)
<corni> nah, please
<corni> ssh+invoking a php script mimicking gitosis should be fine
<corni> or just using gitosis, this would save us some work
[16:47]
<jpetso> HTTP maybe as an additional option, but we mustn't rely on that
<chrono325> for the "unix permissions handle everything" idea, the webserver
would need useradd and chown access to the repositories
<corni> we'd just have to spit out gitosis config files
<corni> yep
<corni> oh wait
<corni> and for gitosis like it would need a way to modify the user list from
gitosis
<chrono325> corni: Gitosis isn't that big of a program, so we could just clone
its functionality
<corni> so for *that* repo it probably would need write access
[16:48]
<jpetso> d.o and cvs.module currently do that by syncing the CVS passwd file
externally via cron or something
[16:50]
<jpetso> so that the webserver writes a file, but not the original file
<corni> this can be used, too
<corni> maybe even from a pre-push-hook :D
<corni> *can be done, too
<chrono325> I don't think we need to write any extra files at all (except as a
way to avoid a bootstrap, possibly)
[16:51]
<chrono325> indefero doesn't do that, except for writing to
.ssh/authorized_keys
<chrono325> and it does a bootstrap on each push as well
[16:52]
<chrono325> yeah, I'm taking a look at how indefero does it right now
[16:53]
<chrono325> looks like ssh uses the environment variable SSH_ORIGINAL_COMMAND
to pass the original command if a forced command is used
<chrono325> oh yeah, this will be doable!
[16:55]
* jpetso is all ears
[16:56]
<chrono325> take a look at http://www.indefero.net/
<chrono325> specifically, src/IDF/Plugin/SyncGit/Serve.php
<chrono325> that's the part that has the handling of push (or pull) commands
[16:57]
<chrono325> all that git needs to execute (on the server side) to do pushes or
pulls is git-receive-pack and git-upload-pack
<chrono325> git-shell(1) is used for allowing access only to those commands
[16:58]
<corni> i'm going to bed now, please post a log later :)
[17:00]
<chrono325> corni: sure, I think I'm figuring it out :)
<jpetso> where does it get $username? i mean, eh, $argv[1]?
[17:01]
<corni> $argv is set by php
<corni> if not turned off by php.ini
<corni> same as in c int main(int argc, char* argv[])
[17:02]
<jpetso> yeah, i know that
<chrono325> it gets the username from the command= param in authorized_keys
<jpetso> ah
<chrono325> it would look something like "command='git-serve.php chrono325'"
[17:03]
<chrono325> git-serve.php would get the value of SSH_ORIGINAL_COMMAND and
check that the user had access to perform the operation in
SSH_ORIGINAL_COMMAND
[17:04]
<jpetso> and we'd write that part by ourselves
<corni> and from there we can do access checks, which are, at least for
branches, reliable, can't we?
<chrono325> jpetso: yup
<chrono325> jpetso: borrowing from indefero or gitosis
<chrono325> corni: yeah, we would have the username and the refs being updated
[17:05]
<chrono325> well, actually it may be a bit more difficult than that
<chrono325> :)
<chrono325> the hooks are called by git receive-pack, so we can't insert
anything between receive-pack and the execution of the hooks
[17:06]
<chrono325> but, if before running receive-pack, we set an environment
variable, we might be able to pass the username to the hook
<chrono325> and that is a complicated control flow
[17:07]
<chrono325> so the whole thing would look like this:
<chrono325> 1) user runs "git push"
<chrono325> 2) user connects to server via ssh
<chrono325> 3) ssh server looks up user's key in ".ssh/authorized_keys" and
sees that there is a "command=" property on that user's key
[17:08]
<chrono325> 4) the value of "command=" is run, which would be something like
"git-serve.php <username>"
<corni> under one common user like 'gitosis'
[17:09]
<corni> ?
<chrono325> 5) git-serve.php checks whether there is a drupal user with
<username> (or who has a vcs account username) and if so, sets an
env variable GIT_DRUPAL_USER_NAME
[17:10]
<chrono325> corni: correct
<chrono325> 6) git-serve.php grabs the value of the env var (which was set by
ssh) SSH_ORIGINAL_COMMAND, which will be "git receive-pack
<repo_path>" and runs that (if step 5 passed)
[17:11]
<DamZ> I'm wondering about hook_versioncontrol_operation()
[17:12]
<chrono325> 7) git receive-pack runs the "update" hook once for each branch or
tag being updated. It gets the user name from GIT_DRUPAL_USER_NAME
<DamZ> its current signature is hook_versioncontrol_operation($op, $operation,
$operation_items)
<DamZ> where $op is 'insert' and 'delete'
<corni> lol
<DamZ> if $op is 'insert', $operation contains a sub operation ('commit',
'branch', 'tag')
[17:13]
<DamZ> why not moving those sub operations to $op?
<corni> because $operation is an object of the vcs_api, and $op specifies what
happens to that object
<corni> $operation is just one action you can do in a vcs
<chrono325> 8) the update hook builds $operation, $operation_items for each
commit being added (using the steps I described earlier) and sets
the author of $operation to GIT_DRUPAL_USER_NAME.
<jpetso> DamZ: think of it as with hook_nodeapi($op, $node, ...)
<corni> and $op is what you can do with that action in php (add and delete)
[17:14]
<DamZ> well, except that it makes writing a trigger based on that less obvious
<corni> chrono325: author should be set by the commiter
<chrono325> 9) if any of has_write_access($operation, $operation_items) fails,
then that ref update is refused.
<corni> chrono325: before the push time
<DamZ> after all, $op is generally 'insert'
<DamZ> 'delete' is only called in a corner case
<jpetso> DamZ: we'll get rid of $op sooner or later, and have
hook_versioncontrol_operation_insert() and
hook_versioncontrol_operation_delete() like core did in D7
[17:15]
<DamZ> (I remove a repository from the system)
<chrono325> corni: but this is the 'author' which versioncontrol_api checks
for permission reasons
<jpetso> DamZ: i thought i'd wait until the D7 version to be consistent to
core
<corni> chrono325: then can you check with the pusher set as author, but later
swap them and use the *real* author for iniserting?
<chrono325> corni: nothing in the "update" hook actually gets saved to the
db. "author" in this case is the uid of the drupal user who is
doing the push
<corni> chrono325: or let amrvil07 introduce a pusher, too
[17:16]
<corni> *marvil07
<chrono325> corni: yes, this is only for access checking. On "post-receive",
the real author would be used
<corni> chrono325: then it's fine :)
<jpetso> DamZ: anyways, $operation is going to be an object
(VersioncontrolOperation $operation) soon, it wouldn't make sense to
merge $op into that object really
<chrono325> so that's it, the nine easy steps to distributed access checking
with git and Drupal!
[17:17]
<jpetso> chrono325: you're brilliant
[17:18]
<corni> can you add them to g.d.o?
<DamZ> jpetso: it's kind of independent, I'm not asking to change $operation,
but $op
<chrono325> jpetso: aww, shucks :D
<corni> chrono325: maybe refined with the stuff we discussed, so that's clear
<marvil07> chrono325: see, like you had it relly clear :D, yep like jpetso I
thinks it's a good idea to have this on gdo
<corni> chrono325: but great work, yeah!
<corni> marvil07: me != jpetso :P
[17:19]
<jpetso> well, i can't be all things to all people
<corni> lol
<chrono325> The only question is whether the env for git receive-pack gets
passed to the update hook
<chrono325> if not, then we are boned
<chrono325> :)
[17:20]
<marvil07> :p
<corni> it should be
<corni> as it's all in the same session, isn't it?
<chrono325> I think it would, doesn't seem like there is any reason it
shouldn't
<corni> that's what the *environment* is for
<jpetso> DamZ: what action would you suggest for $op? get rid of it the D7 way
already?
<chrono325> let me test it
<jpetso> as long as it's a sub-shell, all *exported* variables are passed to
it
[17:21]
<chrono325> I assumed as much
[17:23]
<DamZ> jpetso: thinking about it, I guess it will be ok
<chrono325> woot! it most certainly does! I just tested it on my server, and
the env does get passed. Always nice to double-check
[17:26]
<corni> :)
<chrono325> if we wanted to get really fancy, the ssh connection info is also
passed along, so you get to see the IP of the pusher
[17:27]
<chrono325> we could blacklist certain IPs from pushing, if we wanted :)
<chrono325> alright, I'll include all of this in my weekly update on
Drupal.org
<jpetso> awesome, I like that approach
[17:28]
<jpetso> looking at it again, i'm not sure i get the difference between
"committer" and "pusher", assuming that the original committer is the
"author" (as on Github)
[17:32]
<chrono325> so the author is the person who writes the patch and first commits
it to git
[17:33]
<chrono325> jpetso: they could email their change to someone else who commits
it to their local repository. This second person would be the
committer
<chrono325> btw, this is the way that the Linux kernel does its development.
<jpetso> hm, yeah, makes sense
[17:34]
<chrono325> jpetso: git saves both the author and committer as part of the
commit
<corni> jpetso: and a third person could have the repo of the committer and
push it into mainline ;)
<chrono325> corni: exactly.
<chrono325> let me put names on these, so it is more clear
<chrono325> I write a patch, email it to corni, who commits it to his local
repository
[17:35]
<jpetso> ok, i get it
<chrono325> jpetso then pulls from corni and pushes that branch to d.o
<chrono325> in this situation, the author would be chrono325, the committer
would be corni, and the pusher would be jpetso.
<corni> and access checking should be done for jpetso
[17:36]
<jpetso> according to http://github.com/marvil07/versioncontrol/tree/master
the importing person also counts as "committer" for direct pulls
without email
<corni> this could be github-specific
<chrono325> the problem is that if you look at the information that Git
provides you, jpetso doesn't appear anywhere within the repository
<chrono325> jpetso: no, the committer does not change if you do a pull
<corni> which is solved by 1)-9) though :)
<corni> shouldn't
[17:37]
<corni> chrono325: github does
<jpetso> then Github labels the pusher as committer
<chrono325> jpetso: since changing the committer would change the SHA-1 of the
commit
<corni> chrono325: if you use their fork queue
<chrono325> jpetso: it does? that seems wrong.
<corni> chrono325: which is done, too
<jpetso> or does it?
<corni> chrono325: marvil07 noticed when i merged a change from him into
vcs_git
<chrono325> corni: The merge would be attributed to you, but the committer of
the individual commits should still be his.
[17:38]
<chrono325> corni: I think, I haven't done much multi-person git development
yet
<corni> chrono325: i used the fork queue feature of github and i'm committer
[17:39]
<corni> chrono325: if i pull the commit/cherry pick from amrvil07s repo, he's
the committer and author
<corni> chrono325: see here
<corni> http://github.com/CorniI/versioncontrol_git/commits/master
<chrono325> corni: that may be something that github does
<corni> 1st commit by me, 2nd by push/pull on command line, 3rd by github fork
queue
<jpetso> ah. i know. it's rebasing.
[17:40]
<chrono325> yeah, that's what it looks like
<corni> could be
[17:41]
<jpetso> might be treated just like email imports, in that it can't keep the
original commit
<chrono325> corni: the "username" field of the commits you did on the command
line is your full name, but for the 3rd commit
(1975215d314027987ceb7897ef6e688500ae8b3c), it is "CorniI", which
is your github name
[17:42]
<chrono325> so it looks like github is doing its own thing there
<corni> github also adds a signed-off by line
[17:43]
<chrono325> jpetso: exactly. Since the SHA-1 depends on all of the information
of the commit and is obviously very sensitive to any change,
methods which don't strictly preserve the content of the commit
change the id
<chrono325> and so git goes ahead and changes the committer as well
<corni> but else i don't see a difference in namimng
<corni> though i'm nbot looking at git log atm
<chrono325> anyway, that isn't really relevant to what we need
<corni> bottom line is imho that cli>github
[17:44]
<corni> ;)
<chrono325> the point is that we can't rely on git's notion of "author" and
"committer" for authorization, since neither are authenticated
<marvil07> yep
[17:45]
<corni> imho the design is sorted, or are there any questions left?
<jpetso> i'm happy, way to go
<corni> (for access checking for git, that's it)
<chrono325> actually, this brings up a good point. We should probably ignore
the username part of the author for determining git user-drupal
user correspondence
<marvil07> that's the reason why linux use mails signed gpg :D
<jpetso> chrono325: totally, we should just use the mail for that
[17:46]
<chrono325> since the user name is subject to things like difference in accent
chars and such, but the email address doesn't change
<chrono325> I'll submit an issue
<jpetso> corni: how does the Git backend currently store the username?
<corni> jpetso: if you mean the author, iirc without email
<chrono325> jpetso: I think it is "username <email>"
<corni> jpetso: the email is stored speratly
<corni> jpetso: and later merged to username <email>
[17:47]
<corni> jpetso: but it could be stored that way, too, dunno exactly anymore :D
<jpetso> :]
<jpetso> k. if it's done that way, it should probably be the other way round -
email as author, username for later display
<chrono325> at any rate, we should make sure to use only the email for
figuring out if a particular commit belongs to a drupal user
[17:48]
<chrono325> jpetso: exactly
<corni> jpetso: yep, and this should be mentioned in the issue, too :)
<jpetso> Version Control API does the comparison by itself, so it needs the
account username to match to the one passed in the $operation
[17:49]
<corni> i'm going to bed now
[17:50]
<corni> it's ways too late already
<corni> cu guys
*** corni (n=corni@p54A4C327.dip.t-dialin.net) has quit: "Verlassend"
<chrono325> http://drupal.org/node/498086
[17:51]
<Druplicon> http://drupal.org/node/498086 => Use only the email address for
git user-drupal user relation => Version Control API -- Git
backend, Code, normal, active, 1 IRC mention
<chrono325> jpetso: The description is relatively brief, so if there is more
relevant info that should be included, I can do that as well, but
I wanted to get something in there before I forgot :)
[17:52]
<jpetso> ok, so what's up next on your tables?
<jpetso> yeah, i think it's fine
<chrono325> well, we will have to restructure things a bit, since I did not
get any work done towards rules integration
<jpetso> that's how it goes, i guess, i'm fine with that
[17:53]
<chrono325> jpetso: the current thing on the schedule is to start work on
multiple project branches
<jpetso> let me have a short look at the current schedule
<chrono325> http://groups.drupal.org/node/22301
[17:54]
<Druplicon> http://groups.drupal.org/node/22301 => Completion of Version
Control Integration and Deployment to Drupal.org => 2 IRC
mentions
<jpetso> wah, that's now already?
<jpetso> time does fly
<chrono325> jpetso: I know :(
[17:55]
<jpetso> nothing to do against that, i know you're (both) busy working on
stuff, so that's ok
<chrono325> jpetso: so what do you think should happen going forward? I would
really like to get the git stuff done, since I've already invested
so much time in it
[17:56]
<chrono325> jpetso: and I now have a clear idea of how to move forward
<jpetso> yeah.
[17:57]
<jpetso> i absolutely want multiple project branches to get done. and proper
Git backend workings is important too, so let's make sure to get
those two right
<chrono325> jpetso: agreed. I think that part of "getting it right" will have
to include good simpletests (at least for the hooks, writing tests
for the rest of the backend is a separate proposition)
[17:58]
<chrono325> jpetso: especially for all of the complex use cases of git
<chrono325> jpetso: as I said, I wrote a git fast-import library for PHP so
you can easily create a bunch of commits in php and add them to a
repository
<jpetso> :)
[17:59]
<chrono325> jpetso: my plan was to create a bunch of wildly intertangled
repositories, push bits of them to the repository, and make sure
the right commits are being checked.
<chrono325> jpetso: I'm currently in the process of testing the fast-import
module
[18:00]
* chrono325 tests the tester, ack!
<jpetso> chrono325: would you be interested in defining that set of example
commits that DamZ mentioned previously?
<chrono325> jpetso: which ones?
<jpetso> chrono325: like, having a set of commits that all backends will
eventually test their functionality with
[18:01]
<jpetso> chrono325: although that might not be directly related to the hook
scripts
[18:02]
<chrono325> ah, I se
<chrono325> s/se/see
<chrono325> jpetso: yeah, I could do that.
<jpetso> is it a good idea to do that now? feedback, please
[18:03]
<chrono325> jpetso: The issue is trying to come up with a way of being able to
specify the commits once and then have them tested for each repo
<chrono325> also, I don't think svn and CVS have a native idea of a merge
<chrono325> and I don't know if hg and bzr support n-way merges
<jpetso> chrono325: sure, we need to drop some of those for centralized VCS
<chrono325> they might, I just don't have any experience with them
[18:04]
<jpetso> hg doesn't do n-way, don't know about bzr
<chrono325> jpetso: maybe we could have an interface for converting
$operations to commits within a backend
<chrono325> jpetso: so we would also need a way to specify which capabilities
a test needed
[18:05]
<chrono325> jpetso: and then only run test if the currently tested backend
supported all of the needed caps
<jpetso> chrono325: split into functions, and let the backend call the
appropriate ones?
<jpetso> we could of course reuse the $backend array from
hook_versioncontrol_backends() (it already has capabilities), but i'm
not sure if that's necessary
[18:06]
<jpetso> do we already have a "distributed" capability?
[18:07]
* jpetso looks up
<jpetso> oh noes! no such thing yet
<chrono325> jpetso: I don't think we need "distributed", just "merges", "n-way
merges", etc
<chrono325> since not all DVCSs support n-way merges
[18:08]
<jpetso> for tests, yes
<chrono325> I think putting it in hook_versioncontrol_backends() would be the
way to go
<jpetso> is there any usage for these capabilities outside of tests?
[18:09]
<chrono325> and rather than having a "distributed" cap, we would just
enumerate the caps we are interested in
<chrono325> jpetso: not that I can think of off the top of my head
<chrono325> jpetso: well, support for merges might be useful
<jpetso> (re: chrono325~2) yeah, you're right, i think that's also the reason
why i haven't yet added a "distributed" cap
[18:10]
<chrono325> jpetso: plus, if we already have a place where we are defining
caps of a backend, why not put any caps we are interested in
there, regardless of whether they only apply to tests?
<jpetso> yeah, agreed
<jpetso> i want a "cloneable" capability sooner or later, too
[18:11]
<chrono325> ah, good idea
<jpetso> although we could also define that capability by having the backend
implement a function
[18:12]
<chrono325> I think that the "distributed" cap is really just the combination
of "cloneable", "branches" and a few others
<chrono325> jpetso: hmm, I feel like having a consistent, unified way of
representing a cap would be best.
[18:13]
<marvil07> maybe want binary like in chmod :p
<jpetso> or in svnlib, see sdboyer code
<chrono325> Is there any overhead in computation or complexity of having a
whole bunch of things in the capability array?
[18:14]
<marvil07> I'm not entirely sure, but I thinks caps are gonna desappear with
interfaces ... not really sure untii I do it
<chrono325> marvil07: ah, good point
<chrono325> marvil07: really good point. Interfaces are probably the best way
to do this
<jpetso> +1
[18:15]
<chrono325> so then the question becomes: do we want to add capabilities which
will exist only until marvil07 finishes the OO-ification of the
module?
* marvil07 really write BAD english :p
* marvil07 thinks OO-ification sound really funny, lol
[18:16]
<jpetso> the question is, do we want the test improvements for the 6.x-1.x
branch, or do we wait until 2.x (= marvil07 OOPification)
<chrono325> marvil07: try pronouncing it :D
<jpetso> otoh, 6.x-1.0 is nearly out, it'll probably all go into 2.x anyways
[18:17]
<chrono325> jpetso: well, the comprehensive tests likely won't be done for a
bit anyways, and it could be a good hook to get people to upgrade
to our new and improved 2.x branch! :)
<jpetso> :D
<chrono325> jpetso: The tests are probably too big of a change to add after an
rc1
[18:18]
<jpetso> well they don't hurt, assuming they don't need additional
infrastructure
<marvil07> well, then marvil07 finish oo-ification as soon as you can
<chrono325> we could possibly do a 1.1 release with the tests
<jpetso> marvil07: so are you ok with the approach that i suggested at the
beginning of the session?
[18:19]
<marvil07> jpetso: about tests?
[18:20]
<jpetso> yes
<jpetso> (writing a minimal backend for repository management, and testing
repository and account stuff)
<chrono325> so a mock backend for testing?
[18:21]
<jpetso> yeah
<marvil07> exactly what all are thinking :D
[18:22]
<jpetso> cool. and when that's done, going back to more object orientation
[18:23]
<chrono325> I haven't written mock objects before, but I could probably
learn. FakeVCS is probably a good place to start, but it likely
needs to be changed a bit to make it able to handle actual usage
(rather than just being an example)
<jpetso> i'm not sure if it's a good idea to have FakeVCS do the mock
[18:24]
<jpetso> the issue that i have with this is that FakeVCS is supposed to
provide easily readable example implementations for all possible
callbacks
[18:25]
<jpetso> while a mock backend needs to manage data by itself (� more
complicated code) and probably doesn't want to implement all
callbacks as it's then expected to provide all that functionality as
well
[18:26]
<marvil07> jpetso: good point
<marvil07> so, TestVCS module would be the first target this week
[18:28]
<jpetso> yep, nice
[18:29]
<marvil07> jpetso: but like I saidon the WU, and still applies, I need help
with listing "all things we can do" with versioncontrol core
<marvil07> jpetso: so I could write all needed tests
<jpetso> marvil07: although i think it's just a hook_versioncontrol_backend()
implementation, i don't remember anything more that's needed
<jpetso> marvil07: let me have a look if i can find anything more
<chrono325> I can help generate a set of test commits, since I will do that
for my hooks anyway
[18:31]
<jpetso> marvil07: don't find more requirements, it's really just that one
function apparently
[18:32]
<chrono325> we may also want to make a "convert operation to native commit"
function for each backend, so we can write the tests once, in a
backend-agnostic way.
[18:33]
<jpetso> yes, i like that
<jpetso> can we have chrono325 doing commit tests and marvil07 doing
repository/account tests?
[18:34]
<chrono325> Git, for example, has fast-import, which is designed for exactly
this sort of thing.
<chrono325> well, really for importing from a different VCS, but same idea
<jpetso> chrono325: you're mentioning it the third time already today, now i
need to look up its manpage
<chrono325> jpetso: sounds good to me. From working on the hooks, I've been
close to the commits anyway
<chrono325> :)
[18:35]
<jpetso> right
<chrono325> jpetso: it is a command which accepts input on stdin and converts
it to git commits and writes them to the repository
<chrono325> jpetso: the idea is that you write a program for svn (for example)
which spits out the author, date, files changed, etc. to std out
and then pipe that to fast-import
[18:36]
<jpetso> heh, nice
<chrono325> jpetso: This lets you write importers to git very quickly
<chrono325> also, fast-import is supposed to be REALLY FAST, since it says so
quite often in the manpage :)
[18:37]
<chrono325> but it is ideal for this sort of thing
<jpetso> git help fast-import: "import times for projects holding 10+ years of
history and containing 100,000+ individual commits are generally
completed in just 1-2 hours on quite modest (~$2,000 USD) hardware"
[18:38]
<chrono325> exactly.
<jpetso> i hope they need to do more than d.o needs to do for the Version
Control API migration :P
<chrono325> jpetso: well, it is what we would use when doing the
import. Writing a frontend to it is really not that bad
[18:39]
<chrono325> I did one in PHP in about 3 hours
[18:40]
<jpetso> yeah, i was just wondering if their speed might be any indicative of
how much time we can expect for our own migration scripts from
cvs.module
[18:41]
<jpetso> off-topic, admittedly
<chrono325> I think so. The manpage says that the limiting factor is either
the platter speed of the hard drive or the speed at which the
frontend can keep fast-import supplied with info.
[18:42]
<jpetso> ok guys, i'm going home, and going to bed
[18:47]
<jpetso> you're doing well, i trust you to rock the upcoming week :]
[18:48]
<marvil07> jpetso: thanks for the time
<jpetso> marvil07: thanks to you, i thought you wanted to spend it with your
father instead :P
[18:49]
<marvil07> :p
<chrono325> jpetso: yes, this has been extremely useful
<marvil07> oh, one more question jpetso
<jpetso> sure
[18:50]
<marvil07> simpletest for vc core on versioncontrol_testvcs backend module or
at core itself?
<chrono325> marvil07: I imagine there is not all that much that could be
tested without a backend of some sort, so it would have to live in
testvcs
[18:51]
<jpetso> bah, faster than i could
<chrono325> marvil07: I wouldn't make testvcs a separate project, though.
<chrono325> jpetso: :D
<marvil07> chrono325: yep, another embed module there
[18:52]
<jpetso> exactly that, yes. let's have a versioncontrol_test module that
(also) implements the backend for a VCS named "test"
<jpetso> and the tests, too.
[18:53]
<chrono325> marvil07: careful about file paths, though, since simpletest only
looks in the "tests" dir of a project
<jpetso> oh, that brings me to a question
<chrono325> for example, I have a "gitlib" subdir in versioncontrol_git and a
"gitlib/tests" directory. SimpleTest does not automatically find
tests in "gitlib/tests"
[18:54]
<jpetso> chrono325: did you use the 6.x-2.x branch of the Simpletest module,
or 6.x-1.x?
<chrono325> um
<chrono325> jpetso: lemme check
<marvil07> jpetso: actually following versioncontrol_fakevcs, it'd be
versioncontrol_testvcs :p
<chrono325> 2.8
<jpetso> k, thanks
<chrono325> jpetso: why, should I have used 1.x?
[18:55]
<jpetso> dunno, maybe because the 2.x branch doesn't have as much
functionality ported or whatever
<jpetso> i just want to make sure we're using the same version
<chrono325> jpetso: It has worked well enough for me. If we need to switch, I
can always port my tests.
[18:56]
<jpetso> marvil07: but as that module will contain all test related stuff, it
should be named versioncontrol_test without "vcs"
<jpetso> chrono325: no, 2.x is fine
<chrono325> jpetso: okay, cool.
<marvil07> jpetso: ok
[18:58]
<marvil07> chrono325: thanks for the warning, so where should I put them?
[18:59]
<chrono325> marvil07: you can put them in "versioncontrol/tests/filename.test"
<marvil07> yep
[19:00]
<jpetso> and the accompanying module is
versioncontrol/tests/versioncontrol_test.module, right?
<jpetso> i think that'll work
<marvil07> ok
<chrono325> marvil07: or, you could put them in
"versioncontrol/versioncontrol_testvcs/tests/filename.test" and
create a file "versioncontrol/tests/testvcs_loader.test" which
require_once()d the tests
<chrono325> it's a little hackish, but it works
[19:01]
<marvil07> :D
* chrono325 is doing it that hackish way for testing gitlib
<jpetso> i prefer a single "tests" directory, if that's possible
<marvil07> :p
[19:02]
<chrono325> jpetso: certainly. Then you would just put everything in
"versioncontrol/tests"
<chrono325> you can have multiple files in there, they just all have to end in
".test" for SimpleTest to pick them up
<marvil07> I think it's solved now :D
[19:03]
<marvil07> before leaving jpetso.. do you have plans to be at dcon paris? and
you chrono325 ?
<jpetso> perfect, then we can have other non-.test files without further
directory nesting.
<chrono325> jpetso: exactly.
<jpetso> marvil07: i'd like to come but i can't, gonna be working hard in
canada
<chrono325> I have a "svn_helper.inc" in versioncontrol_svn/tests which has
stuff common to both hook tests
[19:04]
<chrono325> marvil07: It would be fun, but I don't know about the cost
<chrono325> marvil07: Most of my GSoC money will be going towards rent, food,
and biding my time until I can find a job in the fall.
[19:05]
<marvil07> chrono325: actually I was thinking on scholarship :p
<chrono325> marvil07: They have scholarships for that? Hm...
<jpetso> my (current) coworkers will be there, though
<marvil07> chrono325: http://paris2009.drupalcon.org/community/scholarships
<jpetso> marvil07: when you meet fago, criz, klausi or my boss, say hello to
them :D
[19:06]
<marvil07> jpetso: what's the company's name?
[19:07]
<jpetso> Pro.Karriere
<jpetso> oh, and mh86, i forgot
<chrono325> ah, I didn't know about that.
[19:08]
<chrono325> Drupal rocks! *tear*
<jpetso> :D
<jpetso> anyways, i'm leaving now. see you soon!
[19:09]
*** jpetso (n=jpetso@212-183-44-97.adsl.highway.telekom.at) has quit:
[19:10]