Posted by Scott Laird
Fri, 15 Jul 2005 07:55:38 GMT
As mentioned before, I’ve been using SVK to manage the changes that I’ve been making to Typo.
One of the items on my SVK to-do list was to figure out how to mirror a local SVK repository onto a public Subversion server. This is useful for a number of reasons:
- It provides a backup if the local SVK tree is lost.
- It allows me to use SVK on my laptop and work against the same SVK tree that I’m managing on my server.
- It allows others to see what I have in my SVK tree.
The SVK documentation doesn’t provide an exact recipe for doing this, but it turns out to be pretty simple. First, make sure that you have a publicly-accessible Subversion server where you have write privileges. Follow the Subversion Book if you need help with this.
Next, tell SVK to mirror a chunk of the Subversion share into your local SVK depot:
$ svk mkdir //mirror
$ svk ls http://svn.scottstuff.net/public/
<follow the prompts and have it mirror it onto //mirror/public>
$ svk sync --all
$ svk mkdir //mirror/public/typo
$ svk smerge --baseless //typo/ //mirror/public/typo -m 'Initial publication'
The final mkdir operation should prompt you for your Subversion username and password; once this is done, you should have write access to the Subversion share for all future operations.
One thing that I probably should have done was use the -Il option to smerge instead of -m. With -Il, SVK commits each local change to the remote server individually, using the local commit message. Without it, all changes get bundled up in one big change, and this isn’t really ideal.
At this point the public Subversion server should have a complete copy of your local Typo tree. To keep it up to date, you’ll need to run something like svk smerge -Il //typo/ //mirror/public/typo from time to time.
Posted in Typo, Blog stuff, Computer Programming | Tags mirror, public, smerge, subversion, svk | 5 comments
Posted by Scott Laird
Fri, 08 Jul 2005 14:59:33 GMT
I spent an hour or so this morning making sure that everything for this blog is checked into SVK and then created a couple new trees–one as a testing/staging area for the blog, and another for actual production use.
That gives me 4 branches that I use on a daily basis:
//typo/trunk–a copy of Typo’s public Subversion tree.//typo/local–where I do development.//typo/staging–where I stage changes to scottstuff.net.//typo/production–the actual production website.
There are several site-specific changes that live in //typo/staging. This includes changes to the CSS for this site, local graphics, Adsense blocks, links, and eventually things like Flickr integration. They aren’t appropriate for //typo/local, because everything in the local tree is eventually supposed to be merged into the main Typo tree. The production tree has a couple additional changes that only make sense in the production tree–there are a couple small changes to Rails’s public/.htaccess file that need to be made every time I push a change from staging to production. Now that it’s all in SVK, I don’t need to worry about merging this sort of thing anymore–I just run svk smerge //typo/local //typo/staging to push local diffs to the staging area, and then svk smerge //typo/staging //typo/production once I’ve tested things in staging.
Posted in Typo, Blog stuff | Tags distributeddevelopment, subversion, svk, svn, typo | no comments
Posted by Scott Laird
Thu, 07 Jul 2005 22:31:00 GMT
As mentioned yesterday, I’ve been looking for a good way to do revision-control my changes to Typo. Typo is maintained in a publicly-visible Subversion tree, so it’s easy to see when changes occur, but I don’t have the ability to write to the Typo tree, so I can’t use it to track my own changes. Since I currently have 4 or 5 different patches for Subversion floating around, plus a pile of local modifications (like adding Adsense banners and changing the ‘Link’ section in the sidebar), I really need some form of revision control.
I briefly tried to use Submaster yesterday, but it hasn’t been updated for Subversion 1.2, and I was unable to commit any changes. So I tried using SVK. Unlike Submaster, SVK is available in Debian’s sid tree, so it was trivial to install. After running apt-get install svk, I followed the tutorial and was up and running in a few minutes. It took a few minutes to import all 272 revisions from the Typo Subversion tree, but once that was complete I was able to make my own branch and start making changes within a couple minutes.
Here are the important steps to using SVK:
Initialize your local repository
The most recent SVK docs suggest using a single repository for everything, instead of managing a repository per project, so I’ll document that path. The command needed is:
This will create a new depot in ~/.svk/local. This only needs to happen once, when you first install SVK.
Import your external Subversion repository
Since I want to track Typo’s SVN tree, I needed to tell SVK about it. That’s pretty easy, too:
$ svk mirror svn://leetsoft.com/typo/trunk //typo/trunk
$ svk sync --all
Once those were done, I had a full copy of the Typo SVN tree in my SVK repository’s //typo/trunk directory.
Build a local branch
It’s a lot easier to keep track of changes if you make a local branch and then apply all of your development directly to the branch. This works just like it does in Subversion:
$ svk cp //typo/trunk //typo/local
Check out the local branch
A local working branch isn’t much good if you can’t edit it, so check it out somewhere:
$ svk checkout //typo/local local-typo-changes
This will produce a directory called local-typo-changes in your current working directory, and will fill it will all of the files from the local Typo branch. You can now edit files in this tree and use all of the usual version control commands (svk commit, svk add, svk diff, etc–all of the usual Subversion or CVS commands will work with SVK as well).
Producing patches for external consumption
So, let’s say that you’ve hacked a couple nice new features into your local SVK tree. In my case, I added several different features, committing the changes after each change, but I sometimes I found bugs in features after I committed changes to other features. For instance, my threaded-comment patch was modified in revisions 229, 232, and 246. I wanted to roll up all three of those changes into a single testable patch, but not include any of the other changes that were in my tree, like per-article RSS comment feeds.
This turns out to be very easy. First, I created a subdirectory for patchsets within SVK. This isn’t necessary, but I felt like a bit of organization would be useful:
$ svk mkdir //typo/patchsets
Then I created a new threaded-comments branch, starting with the official Typo SVN tree:
$ svk cp //typo/trunk //typo/patchsets/commentthread
Next, I merged the comment thread changesets from my local tree into the commentthread tree:
$ svk merge -c 229 //typo/local //typo/patchsets/commentthread -l
$ svk merge -c 232 //typo/local //typo/patchsets/commentthread -l
$ svk merge -c 246 //typo/local //typo/patchsets/commentthread -l
The -l uses the previous commit message as part of the new merge commit message.
Finally, I checked out the commentthread tree and tested it:
$ svk checkout //typo/patchsets/commentthread commentthread
$ cd commentthread
$ ruby script/server
This way, I had a completely clean tree, using only the comment thread changes, and I was able to verify that everything worked correctly. As it turned out, I’d missed a couple small changes that had snuck into my main tree as part of other, non-commentthread commits, so if I’d submitted this patch without testing, it wouldn’t have worked for people. So, I made the changes, tested things, checked the changes in, and then produced a patch:
$ xemacs <files>
$ ruby script/server
$ svk commit -m 'Fixed stuff'
$ svk diff //typo/trunk //typo/patchsets/commentthread
Now I can go back to working on my main branch. When I fix bugs in the comment thread code in my local branch, I just need to merge the new commit into the commentthread branch and re-diff. Assuming that revision 270 is comment-thread related:
$ cd commentthread
$ svk merge -c 270 -m 'Merge commit 270 from local tree'
$ svk update
$ ruby script/server
$ svk diff //typo/trunk //typo/patchsets/commentthread
One new patch, tested and ready for submission.
Merging with upstream changes
So, what happens when the upstream SVN tree changes? Well, that’s easy enough. First, re-sync your local copy of the trunk:
$ svk sync //typo/trunk
$ svk smerge //typo/trunk //typo/patchsets/articlerss
<follow the directions>
$ cd <working directory>
$ svk update
<verify and test>
$ svk diff //typo/trunk //typo/patchsets/articlerss
I my case, one of my patches was accepted upstream with minor changes. Once I accepted the merge into my patchset tree, I wanted to promote the changes into my main working tree. I did it this way because it should be easier to merge in this order. Probably. Anyway:
$ svk smerge //typo/patchsets/articlerss //typo/local
<follow the directions>
$ ruby script/server
<test>
At this point, my working tree is synced back up with the mainline.
All in all, SVK isn’t much harder to use then plain Subversion, and the ability to import and modify complete trees is fantastic. My next couple goals are:
Figure out how to export my local changes to a publically-visible SVN server, like the one at svn.scottstuff.net.
Figure out how to manage a SVK share on my development web server and on my laptop. This shouldn’t be hard, but I’m still not fully up to speed with SVK.
Posted in Typo, Blog stuff, Computer Programming | Tags distributeddevelopment, subversion, svk, svn, typo | 11 comments
Posted by Scott Laird
Wed, 06 Jul 2005 14:31:00 GMT
My recent spate of typo patches has left me with a bit of a dilemma. Typo is managed using the Subversion version control system. That’s great–Subversion is quite nice–except I don’t have (and don’t really want) commit privileges to the master Typo SVN repository. I can check things out and edit them, and use Subversion to make patches, but once my patches start piling up, I’m left without any form of local version control. I can’t tell my patches apart, and there’s no easy way for me to merge my patches with other changes without totally losing control of which change goes with which patch.
This has to be a fairly common problem–a lot of open-source software is developed using Subversion these days, and CVS has similar problems. I could just set up my own Subversion repository for Typo changes, but syncing it with the upstream tree will be a pain.
Does anyone have any “best practices” suggestions?
The only thing that I see online that might be helpful is Submaster, but it’s full of comments like this:
WARNING: This is just a proof-of-concept implementation. Don’t use it in production environments! You have been warned!
NOTE: This version of SubMaster is written for Subversion 0.35.0. It might not work with any other version of subversion as it is now.
That doesn’t instill confidence.
Update: I gave Submaster a brief spin this morning. I was able to create a Submaster repository, but I was unable to check changes into the local repository without getting errors from svn. I suspect that it has problems with Subversion 1.2.
Since that doesn’t look like it’ll work, I’m going to give SVK a try. The short tutorial on the SVK wiki makes it look pretty simple. It’s currently cloning 272 revisions out of the Typo SVN tree; hopefully it’ll actually work right once it’s done.
Update 2: This is continued in a new article.
Posted in Typo, Computer Programming | Tags distributeddevelopment, subversion, svk, svn, typo | 3 comments
Posted by Scott Laird
Tue, 08 Mar 2005 18:47:02 GMT
I’ve been playing with Trac a bit recently for a hobby project, and I’m really impressed. Trac is an open-source project management application that combines bug tracking, a wiki, and a Subversion browser into a nicely integrated little package. You can think of it as a sort of light-weight do-it-yourself SourceForge, minus the mailing lists and hosted downloads.
One of its cooler features is the ability to provide a single integrated RSS feed that lists SVN checkins, bug tracker progress, and wiki changes all in a single feed.
Unlike certain other bug trackers, Trac’s bug database is actually usable by mere mortals. It does a nice job balancing complexity and simplicity while still being reasonably configurable. For instance, it’s really easy to change Trac’s bug priority and severity options–just run trac-admin on the command line and use its priority and severity commands. Like Joe, I agree that 5 priority levels are at least 2 too many.
Posted in Computer Programming | Tags programming, projectmanagement, subversion, svn, trac | 2 comments
Posted by Scott Laird
Tue, 30 Mar 2004 13:32:22 GMT
I tend to write lots of little scripts and tools for my own personal use, but I very rarely share them. The main reason for this is that it’s a pain to share things–I need to set up infrastructure for storing the source, and some way of generating web pages, and then I need to tell people about the software. It’s a pain.
On the other hand, I have a bunch of Asterisk stuff sitting around that I strongly suspect people would find interesting. I’d like to share it, but the barrier to sharing tends to make it more of a pain then it’s worth for small stuff.
So, I broke down and wrote some infrastructure. It’s not perfect, but it’s easy to use. I have a publicly readable Subversion server at http://svn.scottstuff.net. The tree itself is available in the public directory; there’s also a ViewCVS interface lurking there; see the front page for details. Each project in the Subversion tree will get a page in the /project directory automatically, with the README file auto-converted to HTML. Parts of this are still a bit spotty, and I need to add some menus, but the basic framework is there, it works, and it’s basically trivial to use.
Posted in Computer Software, Personal | Tags subversion | no comments
Posted by Scott Laird
Tue, 09 Mar 2004 10:59:17 GMT
Continuing the theme of backing up my PIM data via Subversion, I’m now exporting the contents of the OS X address book into a subversion tree, and then syncing that from my laptop onto a box at home.
This was a bigger pain then I anticipated; I couldn’t find a tutorial that showed how to use AppleScript to export data from the Address Book, but I could find an Objective C program that extracts data from it. I’ve never used AppleScript or Objective C, so either choice involves learning a new language. At least with Objective C, I have working source to start from. It took most of an hour to rip out the text printing code from ‘contacts’ and substitute in code to save vCards to individual files, but the result seems to work quite nicely. I’ll clean it up and release it eventually; if anyone is interested, send me mail and I’ll speed it up.
Update: I’ve released this as ab2vcard. Let me know if you have any problems.
Posted in Mac stuff, ab2vcard | Tags ab2vcard, addressbook, macosx, subversion | 2 comments
Posted by Scott Laird
Tue, 09 Mar 2004 03:47:26 GMT
My palm is now amazingly well backed up. Every couple hours a script on my PowerBook copies the Palm hotsync backup files into a subversion tree and commits the changes. So, I have a running history of all of the packages that I’ve installed onto my palm as well as a record of what I deleted and when it happened.
Getting the deletes right was a bit of a pain–I’m using rsync and parsing the output of ‘rsync –delete’, and then feeding that into ‘svn delete’.
All of this is then synced over the net to one of my systems at home. So, even if I lost my Palm and my laptop, I’d still be able to recover the contents of my palm. Yes, that’s total overkill, but it’s worth it just for the record of what is changing over time on the palm.
Posted in Handheld and PDA | Tags backup, palm, rsync, subversion | no comments
Posted by Scott Laird
Tue, 24 Feb 2004 11:42:21 GMT
Somewhere in the middle of my MPx200 adventure, something ate all of my repeating annual calendar entries, including birthdays and anniversaries. Personally, I blame PocketMac, but it’s really my fault for not keeping a good backup of my calendar.
That’s not to say that I don’t have a backup–I sync iCal with my home web server every few minutes–but there’s no way to track changes or dig up old versions of my calendar. Once entries are deleted, they’re gone for good. What I really want is revision control for my calendar.
And now I have it. Through a feat of astounding geekiness, I’ve finished installing the subversion version control system on my home web server, and my calendar WebDAV share is actually a subversion repository. What that means is that every time iCal exports a new calendar to my home server, it’s transparently checked into subversion in the background. I can get subversion to give me a list of all of the changes that have been made over time, generate diffs between specific versions, and even revert to older versions.
It’s actually pretty easy. Just install Apache 2, and then install Subversion with the ‘mod_dav_svn’ Apache module. Configure up a virtual server and then add these lines to your virtual host config files:
<Location /xxx>
Dav svn
AuthType Basic
AuthName "Auth Required"
AuthUserFile /etc/apache/htpasswd
Require valid-user
SVNPath /path/to/svn/repository
SVNAutoversioning on
AuthzSVNAccessFile /etc/apache2/svn_authz
</Location>
Now create /path/to/svn/repository with svnadmin, reload apache, and you should have a nice, revision-controlled web-based file server. Most modern desktop OSes can mount WebDAV-enabled web servers just like a more-traditional file server. Both OS X and XP can do it natively, and there should be at least a dozen ways to accomplish it in Linux. This is almost certainly the easiest (cheap) way to get reliable version-controlled storage.
Thanks to Sterling Hughes and Justin Erenkrantz for providing inspiration and documentation.
Posted in Computer System Administration | Tags apache, backups, ical, subversion, webdav | 2 comments
