March 6, 2009 in Twitter
had epiphany of sorts abt difference between free media/culture and free software. Free Software activists started earlier. blog to come
Based on its declared goal of replacing CVS, it is not surprising that the command set of Subversion will be very familiar to CVS users. This document tries to emphasize that similarity, as well as key differences. This document is not intended to cover every command in either version-control tool, but rather to talk to the similarities and differences of key commands in both systems. In general, CVS commands can commonly be executed within Subversion merely by changing “cvs” to “svn.” There are deeper syntactical differences for the non-default uses that are best learned by accessing either the online help or one of the published reference books.
|add -||Used as the way to identify local working copy files to be added to the repository with a commit. The key difference is that the operation is performed offline.|
|annotate -||Used to show the who changed what line in what revision for each line in a specific ASCII file revision. Subversion generally calls this command “blame,” but “annotate” is an alternate name.|
|checkout -||Used as a way to create your local working copy from the repository. Subversion’s command is basically a straight match, with the differences being in the structure and composition of the working copy itself.|
|commit -||Used as the way to publish local working copy changes to the repository. The key difference is again in the execution rather than the command itself. Subversion guarantees an atomic commit of all the modifications associated with the execution of this command, thus creating a changeset. Subversion also assigns each commit/changeset a global revision number rather than applying revision numbers on a per-file per-branch basis. That means each revision number is a unique identifier of a snapshot of the repository after the change set was applied.|
|diff -||Used to examine changes between two revisions of the same file. The biggest difference here with Subversion is the operation can be used offline to compare changes made in the working copy to the revision originally checked out. Additionally, CVS offers some of these commands as prefixed with an ‘r’ (as in “remote”), such as: cvs rlog, cvs rdiff, or cvs rtag. These are used to perform commands directly on CVS repository paths when no working copy is available. Subversion provides similar functionality, but through the use of remote repository URLs instead of working copy paths with its subcommands, such as: svn log URL or svn diff URL1 URL2.|
|export -||Used to create a project tree that does not have the CVS administration directories and files. Subversion’s implementation is basically equivalent, with the exception that a local working copy can also be used as the source of an export. In this case, export is an offline operation.|
|import -||Used to add and commit new files and directories en masse to the repository, CVS’ implementation required two tags to mark the work and supported a particular workflow it related to “vendor branches.” Subversion’s import functions in like manner, but the target can be the trunk and no tags are used. Subversion can support the “vendor branches” workflow, but allows for many other uses of import. Neither tool updates a local working copy, but instead writes directly to the repository.|
|log -||Used as the most common way to access the change history for an object (for example, the log messages associated with each commit/revision). Subversion’s implementation is the same, with the obvious exception that revisions reflect atomic commits so the log message has a larger context.|
|merge -||Used to apply the differences in two sources to a working copy path, marking conflicts that require human resolution, and implemented in CVS as an option to the update command. At this point, Subversion’s implementation is much the same, though merge tracking is on the horizon. The key difference is that Subversion tries to help the user avoid committing files where the conflicts have not been resolved. This is accomplished by adding the ‘svn resolved’ command. That command is required notification to Subversion that conflicts have been resolved. At that point, the file can be committed.|
|remove -||Used to delete files moving forward in history, but not remove all historical references to them. In Subversion, this operation is done offline and also can be used on directories. Additionally, CVS provided a way for administrators to permanently delete files, but Subversion currently takes a strict stance on version control and provides no ability to do such deletions short of a dump, modify, and load of the repository itself.|
|revert -||A command found only in Subversion used to remove all local changes, thereby returning the local working copy to the revision last retrieved from the repository.|
|status -||Used to show any local modifications in the working copy and to show what files are out of date with the repository. Many found the output to be difficult to read so they tended to use ‘cvs update’ to see their changes. Subversion has tried to make the output of this command easier to read and parse. It also defaults to being executed offline, thus only showing working copy changes. However, it can be invoked with a switch (–show-updates) to see what files in the working copy are out of date with the repository.|
|update -||Used as the way to merge changes committed to the repository by others into your local working copy, this command also reported on local working copy changes. Subversion has changed the command to be totally focused on merging the repository committed changes to the working copy leaving the reporting on local modifications to the status command.|
|edit/watch -||Used as a way to notify users that a file was being edited by another and to attempt to guide singular write access to the file. Subversion has implemented the more rigid lock/unlock approach, which guarantees that a single user can change a file that has locking enabled.|
|history -||Used as a way to report operational (for example: checkouts, commits, exports, tags, tags, updates, and releases) history for files matching substrings as executed by you, a specific user(s), or everyone, in a fairly difficult format. With Subversion, using Apache, such information is logged in the Apache logs and operational logging (a feature added in Subversion 1.3) must be enabled to get good information in those logs.|
|tag -||Used to put a human name on a distinct snapshot in time of the full repository or some portion of it, it is a distinct type of meta data in CVS. Subversion implements this concept as just a user’s specific and purposeful execution of the ‘svn copy’ command, which means it is not a type of meta data, but rather a unique path/directory in the repository. Subversion’s implementation provides a cheap and consistent operation that will not slow as the repository grows. Community best practices suggest a top level directory in the repository or in each project contained in the repository for tags (for example, /tags).|
|branch -||Used to establish a distinct line of development (for example, bugfix vs. development vs. developer), it is a specific type of tag in CVS. Subversion implements this concept as just a user’s specific and purposeful execution of the ‘svn copy’ command, which means a branch is just a unique path/directory in the repository. Subversion’s implementation provides a cheap and consistent operation that will not slow as the repository grows. Community best practices suggest a top-level directory in the repository or in each project contained in the repository for branches (for example, /branches).|
|keyword substitution -||Keyword substitution (the ability to substitute dynamic information about a file into the file itself) was controlled by parameters related to the file or working copy and carried an extensive list of keywords reflective of the underlying tool, RCS. Subversion implements keyword substitution through a Subversion-defined property, svn-keywords, (drops support of some RCS-specific keywords) and gives you more control over enabling specific ones versus all.|
|binary file handling -||CVS determined that a file was a binary by either the cvswrappers file (defaulting by file extensions) or when the file was added by a command parameter. It was operationally used to suppress line-end conversions and determine storage. Binary files were stored as whole files, so transmissions from client to server or vice versa always required the full file. The Subversion server does not concern itself with whether a file is a binary or a text/ASCII file, but rather uses the same binary delta differencing algorithm for both storage and transmissions from client to server or server to client. It leaves any concern with whether the file is binary to the client for purposes of ignoring line ending conversions.|
This comparison is not intended to cover all the different nuances of using Subversion commands, but rather to give you a high-level overview of the similarities CVS commands have with their Subversion counterparts. It is not a substitute for Subversion training and command usage should be determined using the online help, as well as a good online or printed reference book.
The import command is the easiest way to migrate snapshots into a Subversion repository. This document clarifies the particular syntax and usage of this command.
Whether the goal is to bring in particular snapshots from other version control tools or to add a local directory structure to your existing Subversion repository, the ‘svn import’ command is a useful and easy to execute tool.
svn import – Add and commit an unversioned file or directory structure to an existing Subversion repository.
svn import [PATH] URL
Recursively add and commit a copy of PATH to URL. If PATH is not present, then the current directory is assumed. Parent directories are created in the repository as necessary.
Directly modifies the repository itself, but not a working copy. Working copies must use the ‘svn update’ command to reflect the changes made to the repository by this command in the local working copy(ies).
This command requires online access to the repository server and cannot be executed in offline mode.
|–message (-m)||TEXT Indicates the commit log message you want associated with this command’s execution|
|–file (-F) FILE||The file listed should be used for the arguments in the execution of this command|
|–quiet (-q)||Only essential information should be output during the execution of this command|
|–non-recursive (-N)||Do not recurse beyond the directory that is the target of this command (default is recursive)|
|–username USER||User’s login account|
|–password PASS||User’s login password|
|–no-auth-cache||Tells Subversion not to cache the username and password in its administrative directories|
|–non-interactive||Do not prompt for username and/or password|
|–force-log||Tells Subversion to accept a suspicious parameter passed to the message or file switches as valid rather than to produce an error|
|–editor-cmd||EDITOR Tells Subversion to use an external editor to enter the commit log message|
|–encoding ENC||Tells Subversion that your commit log message is encoded in the characterset indicated|
|–config-dir DIR||Informs Subversion to read configuration from the specified directory rather than the default location|
|–auto-props||Enables auto-props (overriding what is set in config file)|
|–no-auto-props||Disables auto-props (overriding what is set in config file)|
|–ignore-externals||Tells Subversion to ignore any defined externals when executing this command|
Import the local myproj directory into the root of the repository:
$ svn import -m “New import” myproj http://svn.open.collab.net/repos/projects/trunk
Transmitting file data …………
Committed revision 16.
Import the local myproj directory into the trunk/misc directory (which does not need to exist in the repository prior to executing this command as the command will create the directory).
$ svn import -m “New import” myproj http://svn.open.collab.net/repos/projects/trunk/misc
Transmitting file data …………
Committed revision 19.
Keep in mind that while these examples added directories and files to the repository, they did not create or update a working copy to reflect those changes. You need to either update an existing working copy or do a checkout to create a new one.
CollabNet, through its seven years of interacting with customers, has defined a best practices approach to migrating projects from their current version control (VC) and source code management (SCM) systems. This document outlines CollabNet’s generic approach to migration. As each project contemplates using Subversion moving forward, it is important that the right steps be taken to determine what and how data may be migrated.
The first step in a project’s migration process is to evaluate the costs and benefits of migrating existing project data to Subversion. It is important that project stakeholders:
CollabNet has found that the majority of projects do not experience value that is significant enough to justify the associated costs:
During the needs analysis, it is also important to identify the scope of the data (revisions and associated metadata) to be migrated. It is not realistic to assume that a project needs full migration of all revisions and metadata. A change in version-control systems provides a perfect opportunity to review the contents of the project repository, and this clean up should be a significant aspect of identifying the scope of the data to be migrated.
In conjunction with the Needs Analysis, the potential technical limitations of a migration between two SCM systems need to be investigated. For pure version-control tools, the migration may not have any significant technical limitations, but for more advanced SCM tools, significant technical issues and limitations will arise and must be considered.
For example, several tools have implemented approaches to merge tracking, while the current version of Subversion does not support this feature. Should this data be ignored and result in lost “history” or should it be stored somewhere within Subversion, possibly with limited accessibility?
The difference in implementations of functionality may have important consequences on the scope and method of data migration. For example, many tools and project teams implement tagging or labeling liberally. Subversion implements tags as directories, so while tag creation is a “cheap” operation, it could result in a huge conglomeration of branches that add more confusion than value.
Every SCM tool has a unique set of benefits and features that a good configuration plan needs to utilize appropriately. Migration plans that fail to address changes to the project’s ongoing configuration management needs will provide limited value. Significant value and project satisfaction can be achieved with Subversion by modifying the configuration management plan to take advantage of Subversion’s unique benefits. For example, many configuration management plans limit tagging operations because the legacy SCM tool had performance limitations as the repository grew. Since Subversion’s tagging operation is cheap and consistent, there will likely be further uses for tagging that can provide value to the project. Alternatively, Subversion’s use of a single revision number across the repository to mark a snapshot in time could be used instead of tagging.
CollabNet can provide invaluable insight during the configuration management process, due to our unique and extensive expertise gained from developing, using, and supporting Subversion. CollabNet provides various consulting services to help teams recalibrate their development and configuration management processes before adopting Subversion (rather than transplanting current concrete approaches that worked well on the legacy system into Subversion).
Once the project lead makes the initial project migration strategy assessment, CollabNet has found that an investment in proper upfront planning will result in a faster and easier migration. Strategies such as whether development might continue along in conjunction with some of the migration process versus a full shutdown of development while migration is executed may vary based on numerous factors.
It is also important that several practice migrations are executed, which use a snapshot (or a representative portion) of the project repository in an effort to anticipate any data migration issues prior to the execution of the real migration. Practice migrations significantly reduce potential final migration issues.
Concise information regarding the migration should be communicated to the entire project team. Team members should:
Finally, project members should be notified when the migration is complete so they can begin to interact with Subversion on a full-time basis.
Success with Subversion is not based solely on migrating project artifacts, but more importantly, on enabling users to be successful using Subversion. To that end, it is important that administrators and users receive the appropriate training and coaching that help them become productive in a short period of time. CollabNet offers deep Subversion training for both developers and administrators, which can be customized to meet the needs of specific projects and teams. We can also “train the trainer” when that approach is deemed more appropriate.
The migration team must understand the impact of not having access to the project’s repository, and work with the project lead to limit that downtime, as well as to clearly communicate when the migration starts and finishes. CollabNet has executed numerous strategies around migration and has strong experience in a wide range of migration execution options.
When managing a Subversion server for a number of related projects, in an Open Source community or an enterprise, one needs to strike a useful balance between standardizing the development environment to the extent needed for effective collaboration while leaving enough flexibility to individual teams to work in a variety of ways. Individuals and projects will request particular features or customizations’ with some regularity. This article discusses when to customize, how to customize, and suggests a recommended approach to such requests.
The first and most important question is when to customize. Many software engineers and projects have a natural tendency to adapt, or customize, their tools and environment to their likes, dislikes, and needs. They often feel strongly about particular customizations and push for them, not necessarily properly considering or quantifying the costs for themselves, their project, or the organization or community at large. As long as a customization only impacts an individual, you could arguably assert that it is only one person’s problem and responsibility. However, when it impacts multiple people, a project, or the organization at large, you must balance the benefits against the costs.
The obvious cost is the amount of effort needed to create and maintain the customization. It has to be written, tested, and maintained as the environment evolves. In addition, a customization increases the threshold to accessing data: customizations change the behavior of tools and practices and use of the tool, effectively making it more difficult for an outsider to understand, participate in, and contribute to a project. Also, each customization needs to be supported if users have issues with it. In other words, every customization makes it more expensive to operate and maintain the server, increases the threshold for users to collaborate, and adds to the variety of functionality that needs to be supported.
Therefore, there must be either a tangible benefit to the vast majority of projects or a significant benefit to a few projects. Note that the former is arguably not so much a customization but more a request for (and early prototyping of) a feature users want. The latter is arguably a customization that is too specific for the majority of users.
There is a variety of ways to extend the functionality that Subversion already provides. The options can be classified into two flavors: Client-side or server-side customizations.
Client-side customizations are solutions in which the server remains unchanged, and the customization is done on the client side by wrapping either the command-line client or client-side API calls. Client-side customizations keep the burden of customization on the individual or project requesting it, and forces the requestors to do a proper, honest, cost-benefit analysis as to whether the customization is really wanted or needed. Also, wrappers scale very well in terms of the number of customizations that can be handled: the number of people maintaining the wrappers scales with the customizations.
An example of a client-side customization is svnmerge.py, a Python script on top of the standard Subversion command-line client that allows users to easily merge changes from and to a branch, automatically recording which change sets have already been merged. It can display an always updated list of changes yet to be merged and prevents merge mistakes, such as merging the same change twice. The svnmerge.py script is essentially an early prototype of the merge tracking functionality that is currently being discussed, designed, and implemented for a future release of Subversion.
Server-side customizations are solutions where the server configuration is changed. Server-side customizations scale in terms of rolling out a customization to all projects on the server. They touch the day-to-day operation of the site and increase the effort and cost of operating the service. Also, they potentially impact the security, availability, and performance of the service.
The primary example of server-side customizations are hook scripts. A hook, or hook script, is a program triggered by some repository event, such as creating a new revision or the modifying of an unversioned property. Each hook is handed enough information to tell what that event is, what target(s) it operates on, and the username of the person who triggered the event. Depending on the hook’s output or return status, the hook program may continue the action, stop it, or suspend it in some way. The Version Control with Subversion book describes this in more detail.
Subversion currently defines nine hooks:
Hooks are typically used for three kinds of functionality:
Note that, at present, Subversion does not support a hook performing pre- or post-processing functionality, such as automatically ensuring the code complies with coding guidelines, because the server does not have a means to communicate such changes back to the client. In other words, whatever a hook does, it does not modify the transaction itself. Instead, it can check a condition and accept or reject the action.
Hooks are essentially a way of running arbitrary code on the server in response to actions by the version-control client. Moreover, a hook runs with the same permissions as the web server in general and, with that, has the ability to affect other repositories on the same server. This mechanism is very powerful but has potential implications on the security, availability, and performance of the server. A hook can easily slow down or bring down your server or, even worse, corrupt the data in the repository.
When managing a Subversion server for a number of projects, you need to strike a useful balance between standardizing the environment to enable effective collaboration and efficient operation, while leaving enough flexibility to projects to work in a variety of ways. Standardization can bring many benefits, such as a reduced time to learn the environment when switching projects and enabling more effective collaboration between teams. However, a one-size-fits-all is neither feasible nor desirable with today’s heterogeneity (in local culture, departmental culture, processes, and so on) in individuals and project teams.
From a technical perspective, client- and server-side customizations differ in what they can and can not do:
From a cost-benefit perspective, try to keep customizations that are specific to only a limited set of users on the client side. This puts the burden of customization on the project, stimulating them to make a proper and honest cost-benefit analysis, as well as preventing it from impacting others. Generic customizations that are relevant to and requested by a large percentage of the projects fit better on the server side. Server-side customizations typically have a substantially higher cost, mainly because they potentially impact performance, availability, and security of the entire server. They need rigorous testing, both upon creation and with each upgrade of the server.
Especially when deploying hooks, we strongly recommend using only very commonly used hooks, both to mitigate the risks (the more a hook is used, the more it is tested) and to strike a reasonable balance between standardization and customization: hooks are popular if and only if they contribute value to many people and, with that, are worth the effort. Requests for esoteric customizations are likely not worth the effort of creating, testing, and maintaining.
This article details how to setup svnserve to run as a Windows service. Running the server as a service makes it easy to monitor and manage the server from a remote Windows client and also allows the server to start automatically when the system boots.
This article details how to setup svnserve to run as a Windows service. Running the server as a service makes it easy to monitor and manage the server from a remote Windows client and also allows the server to start automatically when the system boots.
To run svnserve as a Windows service you need
For this article I used the version of svnserve that you can obtain by downloading the CollabNet Subversion server installer from openCollabNet, but these instructions should also work with versions of the executable that you build yourself or have obtained from other sources.
Versions of Subversion prior to 1.4.0 required that you use a third party program that acts as a service-aware wrapper to the svnserve executable. As of 1.4.0, however, the Subversion svnserve executable now has a new –service option that allows the server itself to respond to all of the required events in the Windows service lifecycle. Going forward, this is the preferred way to run svnserve as a service, and is the technique that I describe in this article.
Finally, before continuing, I would like to credit the source of information I used to write this article. As always, the book about Subversion that is maintained by the Subversion developers, Version Control with Subversion, has been invaluable in its role as the canonical reference for all information pertaining to Subversion. If you view the book online, be sure to reference the version that is labeled as containing content for 1.4 of Subversion so that you are viewing the text with the latest information.
The first step is to visit the download section on openCollabNet and grab the installer for the Windows server. The client installer does not contain the svnserve executable, so be sure to get the server installer. Run the installer, which installs the executables, as well as adds the installation folder to your PATH.
The next step is to open a Windows command prompt and create one or more repositories. Technically you can do this step anytime in the process, but the article makes more sense if you do it now, as shown in the following screen shot.
So what did I just do?
I created multiple repositories only to demonstrate that you can have multiple repositories that are all served from the single svnserve server instance: you to decide how many repositories you need. Obviously, you need at least one, and consider that the name you give the repository is exposed in the URL that your users use to access it. Generally, it is a very good idea to avoid using spaces in the repository name. Subversion works with spaces, but they need to be escaped as %20 when used in a URL.
Now that we have some repositories created, the next step is to install svnserve into Windows as a service. Windows includes a program named sc.exe that allows us to do this. On my Windows XP system, this program was already installed and in my PATH. To my knowledge it should be installed automatically on all Windows systems, as it is just a component of Windows. To install svnserve as a service, open a command prompt and run the command as shown:
If you want to just cut and paste that command, here it is:
sc create svnserve binpath= "\"C:\Program Files\CollabNet Subversion Server\svnserve.exe\" --service -r C:\repositories" displayname= "Subversion Server" depend= Tcpip start= auto
Let me break down the elements in the command and explain them in more detail.
sc create svnserve
The first part of the command says that you are creating a service and that you want to give it an identifier of svnserve. You could use a different identifier if you prefer. The only time this name matters is if you want to manage the service from the command line. For example the command “net start svnserve” can be used to start the service. In that example, the service identified by the name “svnserve” would be started.
binpath= "\"C:\Program Files\CollabNet Subversion Server\svnserve.exe\" --service -r C:\repositories"
This is the most important, and complicated, part of the command. It tells Windows the name of the program to run when it starts the service, as well as the arguments. Notice that there is some funky usage of double quotes and escaping. Because the parameter value includes spaces, the whole thing needs to be enclosed in double quotes. In addition, because the path to the svnserve executable contains spaces, it also needs to be enclosed with double quotes. However, since those double quotes are inside of another set, they need to be escaped. So be sure to enter the command exactly as shown.
You must include the –service option as an argument, as that tells svnserve to respond to the Windows service life cycle events. You must not include any conflicting options such as –daemon (-d), –tunnel, or –inetd (-i). You can include additional options such as the -r option shown above. If you want to change the default port, you include the –listen-port NNNN option as an argument.
displayname= "Subversion Server"
This value allows you to specify the name you will see for the service when working with the Services GUI in Windows. If you have embedded spaces in the name, be sure to enclose it all in double quotes as shown.
This value configures the service so that Windows does not attempt to start it until the TCP/IP service has started.
This value configures the service to start automatically when Windows starts.
Now that you added the service, you could start it by running the command:
net start svnserve
Instead, let’s view the service in the Windows GUI and start it from there. The location of the Services application has moved around in various versions of Windows. It is generally available in the Administration Tools folder with the name Services.
This shows our service and that it is currently not started. First, double-click the service so that we can see its details:
One of the aspects of the service you might want to change is on the Log On tab, which I am not going to show but will explain. By default, services run as a restricted system account. This is fine for svnserve and works well. The exception might be if you want to use Subversion hook scripts in your repository. The hook scripts run as the same user as the service, and the default user is fairly limited. For example, it has no authority to access other systems on your network. If your scripts need those capabilities, you can use the Log On tab to change the service so that it runs as a user with the privileges you need to access your domain resources.
Click the Start button to start the service. If it fails to start, the most likely problem is that you got the quotes escaping wrong on the command line. Use the sc.exe program to remove and read the service or update the command line.
If you have a firewall running on your server, you might also need to open the port for svnserve so that it can listen on the port. The default port is tcp/3690.
Open the Windows command prompt and run the svn info command to verify that you can connect to the server.
In this example, I used the svn info command and gave it the URL to the repository I created and named dev. Note that the repository name is exposed in the URL, but not the on disk location (C:\repositories). I can access different repositories just by using a slightly different URL. The three repositories I created at the beginning would be accessed via these URLs:
If you ran these commands from a different machine, you would need to specify the IP address or name of the server in place of localhost. If you used a different port than the default, by using the –listen-port option on the svnserve command, then you would include the port number in the URL:
Finally, the svnserve server is fairly dynamic. You should be able to create additional repositories, as well as make changes to the configurations of existing repositories, without needing to restart the service. It should recognize the changes immediately. In addition, now that the server is running as a service, besides ensuring that the server starts automatically when the system starts, it is also easy to script the stop and start of the server (using the net stop and net start commands) to accommodate your repository backup routines.
An area I did not touch on in this article is repository configuration. There are still things you need to do, such as configure your users and passwords, as well as the access rules. A good source of information about this is the Version Control with Subversion book mentioned at the beginning of the article. The readme file included in the download of CollabNet Subversion also contains a quick walk though of the steps needed to configure your repository.
Finally, if you run into any problems or need help in getting your repository configured correctly, visit the discussion forums that are available on openCollabNet. There are plenty of users willing to provide you with assistance. Be sure to search the forums for answers as well. It is likely that someone else will have asked similar questions in the past, and it is good etiquette to look for those questions and answers before posting a new one. Of course there are also a number of services that are offered by CollabNet to help get you started and train you and your users on the best usage of Subversion.