Distributed development with SVK

As mentioned yesterday, I’ve been looking for a good way to do revision-control my changes to Typo. Typo is maintained in a publicly-visible Subversion tree, so it’s easy to see when changes occur, but I don’t have the ability to write to the Typo tree, so I can’t use it to track my own changes. Since I currently have 4 or 5 different patches for Subversion floating around, plus a pile of local modifications (like adding Adsense banners and changing the ‘Link’ section in the sidebar), I really need some form of revision control.

I briefly tried to use Submaster yesterday, but it hasn’t been updated for Subversion 1.2, and I was unable to commit any changes. So I tried using SVK. Unlike Submaster, SVK is available in Debian’s sid tree, so it was trivial to install. After running apt-get install svk, I followed the tutorial and was up and running in a few minutes. It took a few minutes to import all 272 revisions from the Typo Subversion tree, but once that was complete I was able to make my own branch and start making changes within a couple minutes.

Here are the important steps to using SVK:

Initialize your local repository

The most recent SVK docs suggest using a single repository for everything, instead of managing a repository per project, so I’ll document that path. The command needed is:

$ svk depotmap --init

This will create a new depot in ~/.svk/local. This only needs to happen once, when you first install SVK.

Import your external Subversion repository

Since I want to track Typo’s SVN tree, I needed to tell SVK about it. That’s pretty easy, too:

$ svk mirror svn://leetsoft.com/typo/trunk //typo/trunk
$ svk sync --all

Once those were done, I had a full copy of the Typo SVN tree in my SVK repository’s //typo/trunk directory.

Build a local branch

It’s a lot easier to keep track of changes if you make a local branch and then apply all of your development directly to the branch. This works just like it does in Subversion:

$ svk cp //typo/trunk //typo/local

Check out the local branch

A local working branch isn’t much good if you can’t edit it, so check it out somewhere:

$ svk checkout //typo/local local-typo-changes

This will produce a directory called local-typo-changes in your current working directory, and will fill it will all of the files from the local Typo branch. You can now edit files in this tree and use all of the usual version control commands (svk commit, svk add, svk diff, etc–all of the usual Subversion or CVS commands will work with SVK as well).

Producing patches for external consumption

So, let’s say that you’ve hacked a couple nice new features into your local SVK tree. In my case, I added several different features, committing the changes after each change, but I sometimes I found bugs in features after I committed changes to other features. For instance, my threaded-comment patch was modified in revisions 229, 232, and 246. I wanted to roll up all three of those changes into a single testable patch, but not include any of the other changes that were in my tree, like per-article RSS comment feeds.

This turns out to be very easy. First, I created a subdirectory for patchsets within SVK. This isn’t necessary, but I felt like a bit of organization would be useful:

$ svk mkdir //typo/patchsets

Then I created a new threaded-comments branch, starting with the official Typo SVN tree:

$ svk cp //typo/trunk //typo/patchsets/commentthread

Next, I merged the comment thread changesets from my local tree into the commentthread tree:

$ svk merge -c 229 //typo/local //typo/patchsets/commentthread -l
$ svk merge -c 232 //typo/local //typo/patchsets/commentthread -l
$ svk merge -c 246 //typo/local //typo/patchsets/commentthread -l

The -l uses the previous commit message as part of the new merge commit message.

Finally, I checked out the commentthread tree and tested it:

$ svk checkout //typo/patchsets/commentthread commentthread
$ cd commentthread
$ ruby script/server

This way, I had a completely clean tree, using only the comment thread changes, and I was able to verify that everything worked correctly. As it turned out, I’d missed a couple small changes that had snuck into my main tree as part of other, non-commentthread commits, so if I’d submitted this patch without testing, it wouldn’t have worked for people. So, I made the changes, tested things, checked the changes in, and then produced a patch:

$ xemacs <files>
$ ruby script/server
$ svk commit -m 'Fixed stuff'
$ svk diff //typo/trunk //typo/patchsets/commentthread

Now I can go back to working on my main branch. When I fix bugs in the comment thread code in my local branch, I just need to merge the new commit into the commentthread branch and re-diff. Assuming that revision 270 is comment-thread related:

$ cd commentthread
$ svk merge -c 270 -m 'Merge commit 270 from local tree'
$ svk update
$ ruby script/server
$ svk diff //typo/trunk //typo/patchsets/commentthread

One new patch, tested and ready for submission.

Merging with upstream changes

So, what happens when the upstream SVN tree changes? Well, that’s easy enough. First, re-sync your local copy of the trunk:

$ svk sync //typo/trunk
$ svk smerge //typo/trunk //typo/patchsets/articlerss
<follow the directions>
$ cd <working directory>
$ svk update
<verify and test>
$ svk diff //typo/trunk //typo/patchsets/articlerss

I my case, one of my patches was accepted upstream with minor changes. Once I accepted the merge into my patchset tree, I wanted to promote the changes into my main working tree. I did it this way because it should be easier to merge in this order. Probably. Anyway:

$ svk smerge //typo/patchsets/articlerss //typo/local
<follow the directions>
$ ruby script/server
<test>

At this point, my working tree is synced back up with the mainline.


All in all, SVK isn’t much harder to use then plain Subversion, and the ability to import and modify complete trees is fantastic. My next couple goals are:

  1. Figure out how to export my local changes to a publically-visible SVN server, like the one at svn.scottstuff.net.

  2. Figure out how to manage a SVK share on my development web server and on my laptop. This shouldn’t be hard, but I’m still not fully up to speed with SVK.

Posted by Scott Laird Thu, 07 Jul 2005 22:31:00 GMT


Comments

  1. Kevin Ballard 11 days later:

    These instructions worked great, although I think the svk merge lines need to have the depotpaths added to the end.

  2. Scott Laird 11 days later:

    Thanks. I updated things a bit.

  3. lanfeust 2 months later:

    i was facing the same problem and din’t know that such solutions ever existed. :)

  4. Joe Hosteny 3 months later:

    Scottt, did you ever figure out to manage a share for your laptop and server?

    I’d like to do something similar for Typo, but I want the local copy of the trunk to be mapped to an SVN repo under a Textdrive account, so that I can use SwitchTower to easily deploy my local edits to that account.

    Essentially, I want the functional equivalent of being able to do “svk copy //typo/trunk http://textdrive.com/svn/typo.”;

    Any ideas?

  5. Stefan Scholl 4 months later:

    Wouldn’t it be easier to use darcs?

  6. Scott Laird 4 months later:

    Not really, because I have to interact with multiple Subversion servers that I don’t control. SVK makes working with external Subversion servers trivial.

  7. Charles M. Gerungan 6 months later:

    This the typo of guide I’m looking for when doing the first step in evaluating software. Thanks!

  8. Esad Hajdarevic 6 months later:

    Huh, maybe you can have the “main” checkout on the server and then use something like Unison file syncing to synchronize local and remote copies (laptop and server)

  9. Lars Pind 7 months later:

    This is very useful as I’m setting up my own svk installation.

    How did you end up deploying local changes to your web server? I assume you’re using switchtower to manage the deployment?

    I’ve setup the //typo/production path to be a mirror of the svn on my webserver, so when I merge to production, it’ll automatically go to my webserver.

    I haven’t yet figured out how that plays with switchtower.

    Am I on the right track? How have you solved this?

  10. Trejkaz 8 months later:

    How did you get this to work?

    I just did exactly the same thing but it gives me a “Different sources” error when I try to merge anything into the production branch.

  11. François Beausoleil 10 months later:

    Wouldn’t it be easier to just create a new repository from which Capistrano would deploy ? Then, mirror that deployment repos in your depot, and push the changes back when you are ready to deploy.

    Mind you, I have about five minutes of experience with SVK, but that’s how I’ll use it for deployment.