Interview: David Roundy of Darcs Revision Control

comforteagle writes "In the aftermath of our last interview with Tom Lord, regardless of personalities, it became apparent that the idea of decentralizing CVS is a big deal. Many mentioned darcs as an alternative to Arch. Mark Stosberg has interviewed project head-hancho David Roundy about darcs, his 'theory of patches,' what's next, and on using Haskell for the project."

CVS-style development with darcs

[Ashish Kulkarni]

There was a good post [abridgegame.org] on this on the mailing list a while ago.

Re:Haskell just won't cut it

[cduffy]

Some people will learn a language because they want to know a language that has that specific set of features, regardless of what applications have already been written in that language.

It's a small group, but if you've the only game in town (in terms of OSS projects for them to work with)... well, that works out pretty well for you!

No, if Darcs has any major issues, it's the RAM and CPU time requirements, some of which the design makes inherently unresolvable.

Darcs is KISS

[Earlybird]

Among the plethora of emerging version control systems [zooko.com] -- Subversion, Arch, Monotone and so on -- Darcs stands out for its simplicity and thoughtful design.

Like CVS, you can get productive within minutes; the same cannot be said for Arch or even Subversion. Let's see:

john@somewhere$ cd ~/myproject
john@somewhere$ darcs init

You now have a Darcs repository! Let's do something with it:

john@somewhere$ darcs add -r *
john@somewhere$ darcs record -am "Initial import."
Finished recording patch 'Initial import.'

Now your repository contains all your files. Let's look at the changelog:

john@somewhere$ darcs changes
Thu Nov 25 06:26:19 CET 2004 [e-mail]
* Initial import.

Now, where's the server? You need a server to share your repository, right? Nearly -- every repository is a potential server, as long as it's accessible either through the file system, through SSH/SFTP, HTTP or email. Let's go to another machine and check out the repository we just made:

jane@elsewhere$ darcs get john@somewhere:~/myproject
Copying patches...
.
Finished getting.

We now have a repository on Jane's box. Let's make a modification:

jane@elsewhere$ echo "#include <foo.h>" >>foo.c
jane@elsewhere$ darcs whatsnew --summary
M ./foo.c +1
jane@elsewhere$ darcs whatsnew
{
hunk ./foo.c 2
+#include <foo.h>
}

This last output, by the way, is Darcs' patch format. A "hunk" is a line-based diff. Other types of changes that may be contained in a changeset include renames, moves and binary changes. (Yes, you can also get a GNU-patch-compatible output similar to "cvs diff".)

Now let's commit and push the changes back to John's repository:

jane@elsewhere$ darcs record -am "Added a missing include."
jane@elsewhere$ darcs push -a
[...]
Finished applying...

Now we can go back to John's machine and look:

john@somewhere$ darcs changes
Thu Nov 25 06:26:10 CET 2004 [e-mail]
* Added missing include.

Thu Nov 25 06:26:19 CET 2004 [e-mail]
* Initial import.

(Note how Darcs generates a GNU-style changelog for you automatically.)

Where are the revision numbers, you ask? Well, they don't exist, because they're not needed. Darcs is changeset-oriented, not file-oriented. You can refer to a changeset by name, date, or a special hash identity.

Darcs changesets aren't just GNU patches; they have context, which means, for example, that someone can check out a repository, move a file "foo.c" into the directory "bar" and commit; meanwhile, another person, working on an older copy of the same repository, edits foo.c (which is still in its old location) and commits that. Darcs know that this edit should apply to foo.c in the new location -- and unlike CVS, you don't need to do anything similar to "cvs update" if you're committing files that have been changed on the server. In other words, people can freely commit changes, and the only kind of visible "conflict" will occur when you actually edit the exact same line.

Unlike CVS and Subversion, but like Arch and Monotone, Darcs is a distributed version control system. Repositories are islands which are constantly out of sync with each other, and Darcs' patch commutation system takes care of integration the changes that flow between them.

This system has several extremely useful effects:

• Offline mode [sourcefrog.net]. You can commit changes even if you're on the road with no access to the server. That's because your own working directory is a repository in its own righ

Self-hosting

[Earlybird]

I believe the term you're looking for is self-hosting. Subversion, for example, was originally maintained in CVS, and has a CVS gateway for maintaining redundant systems.

For pilot-testing a migration to Darcs, there are scripts available that convert other repository formats (Subversion, CVS, possibly others) into Darcs (and back, actually), so you avoid losing history when making the transition.

Its the clients and API that matter

[monkeyboy87]

no matter what the change set "theory" are implemented into the product if it's not easy to do things with it, it will languish. VSS and CVS are still used widely becuase there are lots of clients and tools that make it useful. the draw of SVN (loved or hated) is that it has a good client and the command line client is easy to drive with scripting tools

Re:Darcs is KISS

[Earlybird]

• Subversion runs through Apache/2, so if you've got ssl setup in Apache, you have SSL on your repository ...

Sorry, but we were talking about simplicity here. Apache 2.0, SSL, authentication, WebDAV... Setting up a repository with what you suggest is more than one step -- it's usually a lot of steps, especially considering Subversion's external dependencies.

Setting up a Darcs repository consists of doing this:

$ darcs init

And Darcs works over SSH:

$ darcs get [e-mail]:/some/project

This gives you security (leveraging the public-key encryption system of OpenSSH) without having to set up Apache, SSL, virtual hosts, WebDAV or authentication.

You can also use Darcs read-only over HTTP (the following will check out the Darcs sources):

$ darcs get http://www.abridgegame.org/repos/darcs/

All that's required is a web server mapping the URL to your repository -- any one will do.

I'm sure Subversion works similarly, though.

Here:

[warrax_666]

I think you may find this page [berlios.de] interesting.

Major differences:

1. Darcs is must less strict: For example, it doesn't have a built-in concept of branches/versions (which is necessary in Arch because of the patch ordering/application constraints).
   
2. All working directories are repositories themselves. This can be very useful (for example, it makes it trivial to manage /etc using darcs), but also somewhat dangerous.
   
3. Interactive approval of every recorded diff chunk (change). This may not sound like a big deal, but if you have spurious changes in a tree which you'd rather not record(*) in a given changeset, interactive prompting makes it a a lot easier to achieve clean changesets which only touch one "aspect" of the source at a time.
   

(*) Such as typo fixes, comment fixes, etc. which you just happened to notice while fixing a particular bug.

Our experience CVS vs. DARCs

[ites]

We are looking at something to replace our ageing CVS system. We have large OSS projects, worked on by teams of 3-10 people. CVS is very good for what it does but we are feeling its limitations. The biggest problems are that forks are too delicate to use, so we don't use them, and that in order to work you need access to the central archive.

Darcs looked like the best choice. We converted and imported some of our archives. Then we tried checking them out. With CVS, 2-3 minutes. With darcs, 30 minutes.

Our conclusion: darcs is not scalable. Admittedly our code base is large and has a huge history, but in order to use darcs we would have had to break our projects into many small pieces, each with their own repository.

Darcs looks good. But it needs to be made much, much faster if it's to work with large projects.

Re:Mature RCS?

[David Roundy]

Actually, whether or not an RCS is self-hosting is a pretty useless test of the maturity of that RCS. It is incredibly easy to self-host an RCS, since the users are the developers. Darcs was self-hosting way back before I would even have considered suggesting that anyone else consider using it (as I'm sure was the case with Arch).

The hard part of writing an RCS is dealing with all those users who do things you never would have thought of, uncovering all sorts of interesting bugs.

Re:Our experience CVS vs. DARCs

[David Roundy]

Darcs get (equivalent to CVS checkout) is the single least efficient command in darcs. People keep telling me I need to fix this, since it's the first thing users see, but it's really not an important command to optimize (apart from first impressions issues). When run locally (to create a new branch) it's fast.

And comparing darcs get with cvs checkout really isn't fair, since darcs gives you a copy of the full history of the repository, a separate branch on which to record changes before committing them to the centralized repository, and the ability to browse the history offline.

If you want a fast get, just run optimize --checkpoint on the parent repository (assuming you've tagged recently--if not, then tag the current state first), and then use the --partial flag when running darcs get. It'll still give you more flexibility than a cvs checkout, and will be much faster.

Re:Interesting app. non-troll questions

[David Roundy]

1. It's actually hard to use the patch commutation code to do any good outside the concept of a darcs repository.

1.5 I've thought about creating a C library for manipulating/querying darcs repositories, but haven't gotten around to it. The hard part would be of course designing the API. Ideally I'd like the interface to be such that programs using the library couldn't accidentally corrupt the repository.

2. Darcs requires ghc, since it uses some library code only available in ghc to do more efficient IO, string manipulation and to access zlib. It turns out to be a pain on many systems to link with the necesary libaries when using the interpereted version of ghc. So probably accessing darcs from perl will have to go through the executable until a C library is written (which could of course have perl bindings).

3. Rewriting darcs in perl (or parts of it) would be possible, but would be a pain. In particular, the commutation of patches which have conflicts is pretty complicated.

Re:Mostly agreed, but

[David Roundy]

No, I'm afraid the footnote is dead on. When patches are commuted their content changes, although their meaning remains the same. This happens when people pull (or push) patches from one repository to another.

There is an idea, a repository of patch bundles, which could allow signed patches to be kept, at the cost of either duplication of information or inefficiency. It would "just need to be implemented." It would be a bit tedious, but is simple enough that it could be done (terribly inefficiently) using external scripts.

Re:Darcs is KISS

[o1d5ch001]

You can run Subversion using the svnserve tool as well. To start the server:

svnserve -d -c /repo

To access it:

svn co svn://somehostname

or with ssh

svn co svn+ssh://somehostname

Re:why not improve CVS?

[Alioth]

They have. It's called Subversion.
http://subversion.tigris.org/ [tigris.org]

Distributed vs. centralized repositories

[Ian Bicking]

I played around with several source control systems a while back; I like Darcs, it was the easiest of the distributed systems to understand, with a concise and easy-to-understand command-line interface.

But in the end I think a centralized repository is right for many (most?) projects: Why Bitkeeper Isn't Right For Free Software [mit.edu] (by Greg Hudson) was the most convincing argument to me.

Of course, you can simulate a centralized model with a distributed system (just don't use the distributed bits), but it seems like you are on your own at that point. Subversion is behind when you compare distributed features (of course), but when you compare the other features (usability, access via various method, programmatic access, etc), it's way ahead. As it would be; centralized systems are easier to think about, so they are easier to develop with and on top of.