You are browsing the archive for Articles.

Profile photo of admin

by admin

cvs2svn Command Line Help

March 8, 2009 in Articles, subversion

Syntax

cvs2svn [-a svn-repos-path | –dumpfile=PATH] cvs-repos-path

or   (when using an options file use the syntax below)

cvs2svn –options=PATH

Switches (switches that can be used with the options file are in italics, the full list can be used with the other syntax)

--help | -h		Print the usage message and exit with success
--help-passes		Print the numbers and names of the conversion passes and exit with success
--version		Print the version number
--verbose | -v		Tell cvs2svn to print lots of information about what it is doing to STDOUT
--quiet | -q		Tell cvs2svn to operate in quiet mode, printing little more than pass starts and
			stops to STDOUT. This option may be specified twice to suppress all non-error output
--s PATH		Load CVS repository into the Subversion repository located at PATH. If there is no
			Subversion repository at PATH create a new one.
-p PASS			Execute only pass PASS of the conversion. PASS can be specified by name or by number
			(see --help-passes)
-p [START]:[END]	Execute passes START through END of the conversion (inclusive). START and END can be
			specified by name or by number (see --help-passes). If START or END is missing, it
			defaults to the first or last pass, respectively
--existing-svnrepos	Load the converted CVS repository into an existing Subversion repository, instead of
			creating a new one
--dumpfile=PATH		Output the converted CVS repository into a Subversion dumpfile instead of a Subversion
			repository (useful for importing a CVS repository into an existing Subversion repository.
			PATH is the filename in which to store the dumpfile
--dry-run		Do not create a repository or a dumpfile, just print the details of what cvs2svn would do if
			it were really converting your repository.
--use-cvs		If RCS co is having trouble extracting certain revisions, you may need to pass this flag which
			causes cvs2svn to use CVS instead of RCS to read the repository. RCS is much faster, so it is
			the default,but in certain rare cases it has problems with data that CVS does not have
			problems with. Specifically:
			*  RCS can't handle spaces in author names
      			*  "Unterminated keyword" misread by RCS
      			*  RCS handles the "$Log$" keyword differently from CVS
--trunk-only		Convert only the main line of development from the CVS repository (commonly
			referred to in Subversion parlance as "trunk"), ignoring all tags and branches
--trunk=PATH		The top-level path to use for trunk in the Subversion repository. The default
			value is "trunk"
--branches=PATH		The top-level path to use for branches in the Subversion repository. The default
			value is "branches"
--tags=PATH		The top-level path to use for tags in the Subversion repository. The default
			value is "tags"
--no-prune		When all files are deleted from a directory in the Subversion repository, don't
			delete the empty directory (the default is to delete any empty directories)
--encoding=ENC		Use ENC as the encoding for filenames, log messages, and author names in the CVS
			repos. This option may be specified multiple times, in which case the encodings
			are tried in order until one succeeds. Default: ascii
--fallback-encoding=ENC  If all of the encodings specified with --encoding fail, then fall back to using
			 ENC in local 'replace' mode. Use of this option may cause information to be lost,
			 but at least it allows the conversion to run to completion. Default: disabled
--force-branch=REGEXP	Force symbols whose names match REGEXP (a regular expression) to be branches
--force-tag=REGEXP	Force symbols whose names match REGEXP to be tags. This will cause an error if such a
			symbol has commits on it
--exclude=REGEXP	Exclude branches and tags whose names match REGEXP from the conversion.
--symbol-default=OPT	Specify how to convert ambiguous symbols (i.e., those that appear in the CVS archives for
			both branches and tags). OPT is one of the following:
			*  "branch": Treat every ambiguous symbol as a branch
     			*  "tag": Treat every ambiguous symbol as a tag
      			*  "heuristic": Decide how to treat each ambiguous symbol based on whether it was used more
			   often as a branch or tag in CVS.
      			*  "strict": No default; every ambiguous symbol has to be resolved manually using
			   --force-branch, --force-tag, or --exclude. (This is the default behavior.)
--no-cross-branch-commits  Prevent the creation of SVN commits that affect multiple branches or trunk and a branch.
			   Instead, break such changesets into multiple commits, one per branch.
--symbol-transform=PAT:SUB  Transform RCS/CVS symbol names before entering them into Subversion. PAT is a Python
			    regexp pattern and SUB is a replacement pattern using Python's reference syntax. You may
			    specify any number of these options; They will be applied in the order given on the
			    command line. This option can be useful if you're converting a repository in which the
			    developer uses directory-wide symbol names like 1_0, 1_1 and 2_0 as a kludgy form of
			    release tagging (the C-x v s command in Emacs VC mode encourages this practice).
				A command like:  cvs2svn --symbol-transform=' ([0-9])-(.*):release-\1.\2' -q -s SVN
			    will transform a local CVS repository into a local Subversion repository, performing the
			    following sort of mappings
 			    of RCS symbolic names to Subversion tags:
				1_0 -> release-1.0
  				1_1 -> release-1.1
  				2_0 -> release-2.0
--username=NAME		Use NAME as the author for cvs2svn-synthesized commits (the default value is no author at all).
--fs-type=TYPE		Pass the --fs-type=TYPE option to "svnadmin create" when creating a new repository
--bdb-txn-nosync	Pass the -bdb-txn-nosync switch to "svnadmin create" when creating a new repository
--cvs-revnum		Record CVS revision numbers as file properties in the Subversion repository. (note that unless
			it is removed explicitly, the last CVS revision number will remain associated with the
			file even after the file is changed within Subversion
--mime-types=FILE	Specify an apache-style mime.types file for setting svn:mime-type properties on files in the
			Subversion repository.
--auto-props=FILE	Specify a file in the format of Subversion's config file, whose [auto-props] section can be
			used to set arbitrary properties on files in the Subversion repository based on their
			filenames. (the [auto-props] section header must be present; other sections of the config file,
			including the enable-auto-props setting, are ignored.) Filenames are matched to the filename
			 patterns case-sensitively unless the --autoprops-ignore-case option is specified.
--eol-from-mime-type	For files that don't have the kb expansion mode but have a known mime type, set their style
			based on the mime type. For such files, set the svn:eol-style property to "native" if the
 			mime type begins with "text/", and leave it unset (i.e., no EOL translation) otherwise. Files
			with unknown mime types are not affected by this option. This option has no effect unless the
		    	--mime-types option is also specified
--no-default-eol	Files that don't have the kb expansion mode and (if --eol-from-mime-type is set) unknown
			mime type usually have their svn:eol-style property set to "native" If this option is
			specified, such files are left with no eol-style (i.e., no EOL translations)
--keywords-off		By default, cvs2svn sets svn:keywords on CVS files to "author id date" if the mode of the
			RCS file in question is either kv, kvi or NOT kb. If you use the --keywords-off switch,
			cvs2svn will not set svn:keywords for any file. While this will not touch the keywords in
			the contents of your files, Subversion will not expand them
--tmpdir=PATH		Use the directory PATH for all of cvs2svn's temporary data (which can be a lot of data).
			The default value is the current working directory
--skip-cleanup		Prevent the deletion of the temporary files that cvs2svn creates in the process of conversion
--profile		Dump Python Hotshot profiling data to the file cvs2svn.hotshot
--svnadmin=PATH		If the svnadmin program is not in your $PATH you should specify its absolute path with this

			switch
--co=PATH		If the co program (a part of RCS) is not in your $PATH you should specify its absolute path
			with this switch. (co is needed if --use-cvs is not specified.)
--cvs=PATH		If the cvs program is not in your $PATH you should specify its absolute path with this switch.
 			(CVS is needed if --use-cvs is specified)
--sort=PATH		If the GNU sort program is not in your $PATH you should specify its absolute path with this
 			switch cvs2svn requires GNU sort; Windows sort.exe is not adequate
Profile photo of admin

by admin

CVS Migrations Overview

March 5, 2009 in Articles, subversion

Download PDF document on migrating from CVS to Subversion

Profile photo of admin

by admin

CVS to Subversion Command Migration

March 5, 2009 in Articles, subversion

Product:
Subversion
Component:
Migration
Summary:

Based on its declared goal of replacing CVS, it is not surprising that the command set of Subversion will be very familiar to CVS users. This document tries to emphasize that similarity, as well as key differences. This document is not intended to cover every command in either version-control tool, but rather to talk to the similarities and differences of key commands in both systems. In general, CVS commands can commonly be executed within Subversion merely by changing “cvs” to “svn.” There are deeper syntactical differences for the non-default uses that are best learned by accessing either the online help or one of the published reference books.

Matching Commands

add - Used as the way to identify local working copy files to be added to the repository with a commit. The key difference is that the operation is performed offline.
annotate - Used to show the who changed what line in what revision for each line in a specific ASCII file revision. Subversion generally calls this command “blame,” but “annotate” is an alternate name.
checkout - Used as a way to create your local working copy from the repository. Subversion’s command is basically a straight match, with the differences being in the structure and composition of the working copy itself.
commit - Used as the way to publish local working copy changes to the repository. The key difference is again in the execution rather than the command itself. Subversion guarantees an atomic commit of all the modifications associated with the execution of this command, thus creating a changeset. Subversion also assigns each commit/changeset a global revision number rather than applying revision numbers on a per-file per-branch basis. That means each revision number is a unique identifier of a snapshot of the repository after the change set was applied.
diff - Used to examine changes between two revisions of the same file. The biggest difference here with Subversion is the operation can be used offline to compare changes made in the working copy to the revision originally checked out. Additionally, CVS offers some of these commands as prefixed with an ‘r’ (as in “remote”), such as: cvs rlog, cvs rdiff, or cvs rtag. These are used to perform commands directly on CVS repository paths when no working copy is available. Subversion provides similar functionality, but through the use of remote repository URLs instead of working copy paths with its subcommands, such as: svn log URL or svn diff URL1 URL2.
export - Used to create a project tree that does not have the CVS administration directories and files. Subversion’s implementation is basically equivalent, with the exception that a local working copy can also be used as the source of an export. In this case, export is an offline operation.
import - Used to add and commit new files and directories en masse to the repository, CVS’ implementation required two tags to mark the work and supported a particular workflow it related to “vendor branches.” Subversion’s import functions in like manner, but the target can be the trunk and no tags are used. Subversion can support the “vendor branches” workflow, but allows for many other uses of import. Neither tool updates a local working copy, but instead writes directly to the repository.
log - Used as the most common way to access the change history for an object (for example, the log messages associated with each commit/revision). Subversion’s implementation is the same, with the obvious exception that revisions reflect atomic commits so the log message has a larger context.
merge - Used to apply the differences in two sources to a working copy path, marking conflicts that require human resolution, and implemented in CVS as an option to the update command. At this point, Subversion’s implementation is much the same, though merge tracking is on the horizon. The key difference is that Subversion tries to help the user avoid committing files where the conflicts have not been resolved. This is accomplished by adding the ‘svn resolved’ command. That command is required notification to Subversion that conflicts have been resolved. At that point, the file can be committed.
remove - Used to delete files moving forward in history, but not remove all historical references to them. In Subversion, this operation is done offline and also can be used on directories. Additionally, CVS provided a way for administrators to permanently delete files, but Subversion currently takes a strict stance on version control and provides no ability to do such deletions short of a dump, modify, and load of the repository itself.
revert - A command found only in Subversion used to remove all local changes, thereby returning the local working copy to the revision last retrieved from the repository.
status - Used to show any local modifications in the working copy and to show what files are out of date with the repository. Many found the output to be difficult to read so they tended to use ‘cvs update’ to see their changes. Subversion has tried to make the output of this command easier to read and parse. It also defaults to being executed offline, thus only showing working copy changes. However, it can be invoked with a switch (–show-updates) to see what files in the working copy are out of date with the repository.
update - Used as the way to merge changes committed to the repository by others into your local working copy, this command also reported on local working copy changes. Subversion has changed the command to be totally focused on merging the repository committed changes to the working copy leaving the reporting on local modifications to the status command.

Indirectly Matched Commands

edit/watch - Used as a way to notify users that a file was being edited by another and to attempt to guide singular write access to the file. Subversion has implemented the more rigid lock/unlock approach, which guarantees that a single user can change a file that has locking enabled.
history - Used as a way to report operational (for example: checkouts, commits, exports, tags, tags, updates, and releases) history for files matching substrings as executed by you, a specific user(s), or everyone, in a fairly difficult format. With Subversion, using Apache, such information is logged in the Apache logs and operational logging (a feature added in Subversion 1.3) must be enabled to get good information in those logs.
tag - Used to put a human name on a distinct snapshot in time of the full repository or some portion of it, it is a distinct type of meta data in CVS. Subversion implements this concept as just a user’s specific and purposeful execution of the ‘svn copy’ command, which means it is not a type of meta data, but rather a unique path/directory in the repository. Subversion’s implementation provides a cheap and consistent operation that will not slow as the repository grows. Community best practices suggest a top level directory in the repository or in each project contained in the repository for tags (for example, /tags).

Alternatively Implemented Functionality

branch - Used to establish a distinct line of development (for example, bugfix vs. development vs. developer), it is a specific type of tag in CVS. Subversion implements this concept as just a user’s specific and purposeful execution of the ‘svn copy’ command, which means a branch is just a unique path/directory in the repository. Subversion’s implementation provides a cheap and consistent operation that will not slow as the repository grows. Community best practices suggest a top-level directory in the repository or in each project contained in the repository for branches (for example, /branches).
keyword substitution - Keyword substitution (the ability to substitute dynamic information about a file into the file itself) was controlled by parameters related to the file or working copy and carried an extensive list of keywords reflective of the underlying tool, RCS. Subversion implements keyword substitution through a Subversion-defined property, svn-keywords, (drops support of some RCS-specific keywords) and gives you more control over enabling specific ones versus all.
binary file handling - CVS determined that a file was a binary by either the cvswrappers file (defaulting by file extensions) or when the file was added by a command parameter. It was operationally used to suppress line-end conversions and determine storage. Binary files were stored as whole files, so transmissions from client to server or vice versa always required the full file. The Subversion server does not concern itself with whether a file is a binary or a text/ASCII file, but rather uses the same binary delta differencing algorithm for both storage and transmissions from client to server or server to client. It leaves any concern with whether the file is binary to the client for purposes of ignoring line ending conversions.

This comparison is not intended to cover all the different nuances of using Subversion commands, but rather to give you a high-level overview of the similarities CVS commands have with their Subversion counterparts. It is not a substitute for Subversion training and command usage should be determined using the online help, as well as a good online or printed reference book.

Profile photo of admin

by admin

SVN Import Defined

March 5, 2009 in Articles, subversion

Product:
Subversion
Component:
Summary:

The import command is the easiest way to migrate snapshots into a Subversion repository. This document clarifies the particular syntax and usage of this command.

Whether the goal is to bring in particular snapshots from other version control tools or to add a local directory structure to your existing Subversion repository, the ‘svn import’ command is a useful and easy to execute tool.

Command Name

svn import – Add and commit an unversioned file or directory structure to an existing Subversion repository.

General Syntax

svn import [PATH] URL

Command Description

Recursively add and commit a copy of PATH to URL. If PATH is not present, then the current directory is assumed. Parent directories are created in the repository as necessary.

Scope of Changes

Directly modifies the repository itself, but not a working copy. Working copies must use the ‘svn update’ command to reflect the changes made to the repository by this command in the local working copy(ies).

Online or Offline Operation

This command requires online access to the repository server and cannot be executed in offline mode.

Applicable Switches

–message (-m) TEXT Indicates the commit log message you want associated with this command’s execution
–file (-F) FILE The file listed should be used for the arguments in the execution of this command
–quiet (-q) Only essential information should be output during the execution of this command
–non-recursive (-N) Do not recurse beyond the directory that is the target of this command (default is recursive)
–username USER User’s login account
–password PASS User’s login password
–no-auth-cache Tells Subversion not to cache the username and password in its administrative directories
–non-interactive Do not prompt for username and/or password
–force-log Tells Subversion to accept a suspicious parameter passed to the message or file switches as valid rather than to produce an error
–editor-cmd EDITOR Tells Subversion to use an external editor to enter the commit log message
–encoding ENC Tells Subversion that your commit log message is encoded in the characterset indicated
–config-dir DIR Informs Subversion to read configuration from the specified directory rather than the default location
–auto-props Enables auto-props (overriding what is set in config file)
–no-auto-props Disables auto-props (overriding what is set in config file)
–ignore-externals Tells Subversion to ignore any defined externals when executing this command

Examples

Import the local myproj directory into the root of the repository:
$ svn import -m “New import” myproj http://svn.open.collab.net/repos/projects/trunk
Adding myproj/sample.txt

Transmitting file data …………
Committed revision 16.
Import the local myproj directory into the trunk/misc directory (which does not need to exist in the repository prior to executing this command as the command will create the directory).
$ svn import -m “New import” myproj http://svn.open.collab.net/repos/projects/trunk/misc
Adding myproj/sample.txt

Transmitting file data …………
Committed revision 19.

Keep in mind that while these examples added directories and files to the repository, they did not create or update a working copy to reflect those changes. You need to either update an existing working copy or do a checkout to create a new one.

Profile photo of admin

by admin

Categorizing Migrations

March 5, 2009 in Articles, subversion

PDF document with information on Subversion migrations.

Profile photo of admin

by admin

Subversion Migration Planning

March 5, 2009 in Articles, subversion

Product:
Subversion
Component:
Summary:

CollabNet, through its seven years of interacting with customers, has defined a best practices approach to migrating projects from their current version control (VC) and source code management (SCM) systems. This document outlines CollabNet’s generic approach to migration. As each project contemplates using Subversion moving forward, it is important that the right steps be taken to determine what and how data may be migrated.

Overall Migration Approach

Needs Analysis

The first step in a project’s migration process is to evaluate the costs and benefits of migrating existing project data to Subversion. It is important that project stakeholders:

  • Acknowledge the monetary and productivity costs associated with migrations
  • Define the need and business / technical value of having the data migrated

CollabNet has found that the majority of projects do not experience value that is significant enough to justify the associated costs:

  • Leaving legacy project data available in a slimmed down legacy SCM system is sufficient to satisfy the need for the occasional review of history.
  • In other cases, taking a small set of revisions provides the right balance of value.
  • In a limited set of cases, a full migration is justified.

During the needs analysis, it is also important to identify the scope of the data (revisions and associated metadata) to be migrated. It is not realistic to assume that a project needs full migration of all revisions and metadata. A change in version-control systems provides a perfect opportunity to review the contents of the project repository, and this clean up should be a significant aspect of identifying the scope of the data to be migrated.

Environment Assessment

In conjunction with the Needs Analysis, the potential technical limitations of a migration between two SCM systems need to be investigated. For pure version-control tools, the migration may not have any significant technical limitations, but for more advanced SCM tools, significant technical issues and limitations will arise and must be considered.

For example, several tools have implemented approaches to merge tracking, while the current version of Subversion does not support this feature. Should this data be ignored and result in lost “history” or should it be stored somewhere within Subversion, possibly with limited accessibility?

The difference in implementations of functionality may have important consequences on the scope and method of data migration. For example, many tools and project teams implement tagging or labeling liberally. Subversion implements tags as directories, so while tag creation is a “cheap” operation, it could result in a huge conglomeration of branches that add more confusion than value.

Configuration Management Planning

Every SCM tool has a unique set of benefits and features that a good configuration plan needs to utilize appropriately. Migration plans that fail to address changes to the project’s ongoing configuration management needs will provide limited value. Significant value and project satisfaction can be achieved with Subversion by modifying the configuration management plan to take advantage of Subversion’s unique benefits. For example, many configuration management plans limit tagging operations because the legacy SCM tool had performance limitations as the repository grew. Since Subversion’s tagging operation is cheap and consistent, there will likely be further uses for tagging that can provide value to the project. Alternatively, Subversion’s use of a single revision number across the repository to mark a snapshot in time could be used instead of tagging.

CollabNet can provide invaluable insight during the configuration management process, due to our unique and extensive expertise gained from developing, using, and supporting Subversion. CollabNet provides various consulting services to help teams recalibrate their development and configuration management processes before adopting Subversion (rather than transplanting current concrete approaches that worked well on the legacy system into Subversion).

Migration Planning

Once the project lead makes the initial project migration strategy assessment, CollabNet has found that an investment in proper upfront planning will result in a faster and easier migration. Strategies such as whether development might continue along in conjunction with some of the migration process versus a full shutdown of development while migration is executed may vary based on numerous factors.

It is also important that several practice migrations are executed, which use a snapshot (or a representative portion) of the project repository in an effort to anticipate any data migration issues prior to the execution of the real migration. Practice migrations significantly reduce potential final migration issues.

Concise information regarding the migration should be communicated to the entire project team. Team members should:

  • Receive notification that the migration effort is in the planning stage so that they can offer relevant input.
  • Understand the approach to the scope and migration method, so they can clearly understand how to access historical project data after the migration is complete.
  • Familiarize themselves with Subversion usage, as well as changes to the current configuration management plan to take advantage of Subversion functionality.
  • Know the actual migration timeframes to prepare and minimize its productivity impact.

Finally, project members should be notified when the migration is complete so they can begin to interact with Subversion on a full-time basis.

Subversion Training

Success with Subversion is not based solely on migrating project artifacts, but more importantly, on enabling users to be successful using Subversion. To that end, it is important that administrators and users receive the appropriate training and coaching that help them become productive in a short period of time. CollabNet offers deep Subversion training for both developers and administrators, which can be customized to meet the needs of specific projects and teams. We can also “train the trainer” when that approach is deemed more appropriate.

Migration Execution

The migration team must understand the impact of not having access to the project’s repository, and work with the project lead to limit that downtime, as well as to clearly communicate when the migration starts and finishes. CollabNet has executed numerous strategies around migration and has strong experience in a wide range of migration execution options.

Profile photo of admin

by admin

Enhancing a Subversion Server

March 5, 2009 in Articles, subversion

Product:
Subversion 1.4.x
Component:
Summary:

When managing a Subversion server for a number of related projects, in an Open Source community or an enterprise, one needs to strike a useful balance between standardizing the development environment to the extent needed for effective collaboration while leaving enough flexibility to individual teams to work in a variety of ways. Individuals and projects will request particular features or customizations’ with some regularity. This article discusses when to customize, how to customize, and suggests a recommended approach to such requests.

The first and most important question is when to customize. Many software engineers and projects have a natural tendency to adapt, or customize, their tools and environment to their likes, dislikes, and needs. They often feel strongly about particular customizations and push for them, not necessarily properly considering or quantifying the costs for themselves, their project, or the organization or community at large. As long as a customization only impacts an individual, you could arguably assert that it is only one person’s problem and responsibility. However, when it impacts multiple people, a project, or the organization at large, you must balance the benefits against the costs.

The Cost of Customization

The obvious cost is the amount of effort needed to create and maintain the customization. It has to be written, tested, and maintained as the environment evolves. In addition, a customization increases the threshold to accessing data: customizations change the behavior of tools and practices and use of the tool, effectively making it more difficult for an outsider to understand, participate in, and contribute to a project. Also, each customization needs to be supported if users have issues with it. In other words, every customization makes it more expensive to operate and maintain the server, increases the threshold for users to collaborate, and adds to the variety of functionality that needs to be supported.

Therefore, there must be either a tangible benefit to the vast majority of projects or a significant benefit to a few projects. Note that the former is arguably not so much a customization but more a request for (and early prototyping of) a feature users want. The latter is arguably a customization that is too specific for the majority of users.

How to Customize

There is a variety of ways to extend the functionality that Subversion already provides. The options can be classified into two flavors: Client-side or server-side customizations.

Client side – Wrappers

Client-side customizations are solutions in which the server remains unchanged, and the customization is done on the client side by wrapping either the command-line client or client-side API calls. Client-side customizations keep the burden of customization on the individual or project requesting it, and forces the requestors to do a proper, honest, cost-benefit analysis as to whether the customization is really wanted or needed. Also, wrappers scale very well in terms of the number of customizations that can be handled: the number of people maintaining the wrappers scales with the customizations.

An example of a client-side customization is svnmerge.py, a Python script on top of the standard Subversion command-line client that allows users to easily merge changes from and to a branch, automatically recording which change sets have already been merged. It can display an always updated list of changes yet to be merged and prevents merge mistakes, such as merging the same change twice. The svnmerge.py script is essentially an early prototype of the merge tracking functionality that is currently being discussed, designed, and implemented for a future release of Subversion.

Server Side – Hook Scripts

Server-side customizations are solutions where the server configuration is changed. Server-side customizations scale in terms of rolling out a customization to all projects on the server. They touch the day-to-day operation of the site and increase the effort and cost of operating the service. Also, they potentially impact the security, availability, and performance of the service.

The primary example of server-side customizations are hook scripts. A hook, or hook script, is a program triggered by some repository event, such as creating a new revision or the modifying of an unversioned property. Each hook is handed enough information to tell what that event is, what target(s) it operates on, and the username of the person who triggered the event. Depending on the hook’s output or return status, the hook program may continue the action, stop it, or suspend it in some way. The Version Control with Subversion book describes this in more detail.

Subversion currently defines nine hooks:

  • The start-commit hook is invoked before a transaction is created in the process of doing a commit.
  • The pre-commit hook is invoked after a transaction has been created but before it is committed.
  • The post-commit hook is invoked after a transaction is committed.
  • The pre-revprop-change and post-revprop-change hooks are invoked before respectively after a revision property is added, modified, or deleted.
  • The pre-lock and post-lock hooks are invoked before respectively after an exclusive lock on a path is created.
  • The pre-unlock and post-unlock hooks are invoked before respectively after an exclusive lock is destroyed.

Hooks are typically used for three kinds of functionality:

  • First, to log or notify interested parties about an event. For example, sending an e-mail message per commit, summarizing key information about that commit such as the author, date, commit message, and the change set.
  • Second, to check a particular condition. For example, verify whether the code complies within coding guidelines or whether the user has the appropriate access rights to the parts of the repository that he wants to commit to.
  • Third, to block certain behavior. For example, allow a user to change a log message but not the author and date of a revision (to maintain traceability), prevent locks from being stolen, or allow locking if and only if the path has the svn:needs-lock property set.

Note that, at present, Subversion does not support a hook performing pre- or post-processing functionality, such as automatically ensuring the code complies with coding guidelines, because the server does not have a means to communicate such changes back to the client. In other words, whatever a hook does, it does not modify the transaction itself. Instead, it can check a condition and accept or reject the action.

Hooks are essentially a way of running arbitrary code on the server in response to actions by the version-control client. Moreover, a hook runs with the same permissions as the web server in general and, with that, has the ability to affect other repositories on the same server. This mechanism is very powerful but has potential implications on the security, availability, and performance of the server. A hook can easily slow down or bring down your server or, even worse, corrupt the data in the repository.

Findings and Recommendations

When managing a Subversion server for a number of projects, you need to strike a useful balance between standardizing the environment to enable effective collaboration and efficient operation, while leaving enough flexibility to projects to work in a variety of ways. Standardization can bring many benefits, such as a reduced time to learn the environment when switching projects and enabling more effective collaboration between teams. However, a one-size-fits-all is neither feasible nor desirable with today’s heterogeneity (in local culture, departmental culture, processes, and so on) in individuals and project teams.

From a technical perspective, client- and server-side customizations differ in what they can and can not do:

  • Client-side customizations are suitable when it affects only a single user or for automatic pre-processing (such as code formatting).
  • Server-side customizations are suitable when the change should be standardized across the repository and is a notification, a check, or blocking certain behavior.

From a cost-benefit perspective, try to keep customizations that are specific to only a limited set of users on the client side. This puts the burden of customization on the project, stimulating them to make a proper and honest cost-benefit analysis, as well as preventing it from impacting others. Generic customizations that are relevant to and requested by a large percentage of the projects fit better on the server side. Server-side customizations typically have a substantially higher cost, mainly because they potentially impact performance, availability, and security of the entire server. They need rigorous testing, both upon creation and with each upgrade of the server.

Especially when deploying hooks, we strongly recommend using only very commonly used hooks, both to mitigate the risks (the more a hook is used, the more it is tested) and to strike a reasonable balance between standardization and customization: hooks are popular if and only if they contribute value to many people and, with that, are worth the effort. Requests for esoteric customizations are likely not worth the effort of creating, testing, and maintaining.

Profile photo of admin

by admin

Configuring svnserve to Run as a Windows Service

March 5, 2009 in Articles, subversion

Product:
Subversion 1.4.x
Component:
Summary:

This article details how to setup svnserve to run as a Windows service. Running the server as a service makes it easy to monitor and manage the server from a remote Windows client and also allows the server to start automatically when the system boots.

This article details how to setup svnserve to run as a Windows service. Running the server as a service makes it easy to monitor and manage the server from a remote Windows client and also allows the server to start automatically when the system boots.

Requirements to Run svnserve as a Windows Service

To run svnserve as a Windows service you need

  • Windows 2000/2003/XP or Vista
  • Subversion 1.4.0 or higher

For this article I used the version of svnserve that you can obtain by downloading the CollabNet Subversion server installer from openCollabNet, but these instructions should also work with versions of the executable that you build yourself or have obtained from other sources.

Versions of Subversion prior to 1.4.0 required that you use a third party program that acts as a service-aware wrapper to the svnserve executable. As of 1.4.0, however, the Subversion svnserve executable now has a new –service option that allows the server itself to respond to all of the required events in the Windows service lifecycle. Going forward, this is the preferred way to run svnserve as a service, and is the technique that I describe in this article.

Finally, before continuing, I would like to credit the source of information I used to write this article. As always, the book about Subversion that is maintained by the Subversion developers, Version Control with Subversion, has been invaluable in its role as the canonical reference for all information pertaining to Subversion. If you view the book online, be sure to reference the version that is labeled as containing content for 1.4 of Subversion so that you are viewing the text with the latest information.

Preparing the Windows Server

The first step is to visit the download section on openCollabNet and grab the installer for the Windows server. The client installer does not contain the svnserve executable, so be sure to get the server installer. Run the installer, which installs the executables, as well as adds the installation folder to your PATH.

The next step is to open a Windows command prompt and create one or more repositories. Technically you can do this step anytime in the process, but the article makes more sense if you do it now, as shown in the following screen shot.

CollabNet Subversion

So what did I just do?

  • The first step, which is not shown, is that I used the CD \ command to get to the root of my C:\ drive.
  • I then created a folder named repositories, which will hold all of my repositories.
  • I then ran the svnadmin command three times to create three repositories which I named dev, mktg and docs respectively.

I created multiple repositories only to demonstrate that you can have multiple repositories that are all served from the single svnserve server instance: you to decide how many repositories you need. Obviously, you need at least one, and consider that the name you give the repository is exposed in the URL that your users use to access it. Generally, it is a very good idea to avoid using spaces in the repository name. Subversion works with spaces, but they need to be escaped as %20 when used in a URL.

Installing svnserve as Service

Now that we have some repositories created, the next step is to install svnserve into Windows as a service. Windows includes a program named sc.exe that allows us to do this. On my Windows XP system, this program was already installed and in my PATH. To my knowledge it should be installed automatically on all Windows systems, as it is just a component of Windows. To install svnserve as a service, open a command prompt and run the command as shown:

CollabNet Subversion

If you want to just cut and paste that command, here it is:

sc create svnserve binpath= "\"C:\Program Files\CollabNet Subversion Server\svnserve.exe\" --service -r C:\repositories" displayname= "Subversion Server" depend= Tcpip start= auto

Let me break down the elements in the command and explain them in more detail.

sc create svnserve

The first part of the command says that you are creating a service and that you want to give it an identifier of svnserve. You could use a different identifier if you prefer. The only time this name matters is if you want to manage the service from the command line. For example the command “net start svnserve” can be used to start the service. In that example, the service identified by the name “svnserve” would be started.

binpath= "\"C:\Program Files\CollabNet Subversion Server\svnserve.exe\" --service -r C:\repositories"

This is the most important, and complicated, part of the command. It tells Windows the name of the program to run when it starts the service, as well as the arguments. Notice that there is some funky usage of double quotes and escaping. Because the parameter value includes spaces, the whole thing needs to be enclosed in double quotes. In addition, because the path to the svnserve executable contains spaces, it also needs to be enclosed with double quotes. However, since those double quotes are inside of another set, they need to be escaped. So be sure to enter the command exactly as shown.

You must include the –service option as an argument, as that tells svnserve to respond to the Windows service life cycle events. You must not include any conflicting options such as –daemon (-d), –tunnel, or –inetd (-i). You can include additional options such as the -r option shown above. If you want to change the default port, you include the –listen-port NNNN option as an argument.

displayname= "Subversion Server"

This value allows you to specify the name you will see for the service when working with the Services GUI in Windows. If you have embedded spaces in the name, be sure to enclose it all in double quotes as shown.

depend= Tcpip

This value configures the service so that Windows does not attempt to start it until the TCP/IP service has started.

start= auto

This value configures the service to start automatically when Windows starts.

Starting svnserve

Now that you added the service, you could start it by running the command:

net start svnserve

Instead, let’s view the service in the Windows GUI and start it from there. The location of the Services application has moved around in various versions of Windows. It is generally available in the Administration Tools folder with the name Services.

CollabNet Subversion

This shows our service and that it is currently not started. First, double-click the service so that we can see its details:

CollabNet Subversion

One of the aspects of the service you might want to change is on the Log On tab, which I am not going to show but will explain. By default, services run as a restricted system account. This is fine for svnserve and works well. The exception might be if you want to use Subversion hook scripts in your repository. The hook scripts run as the same user as the service, and the default user is fairly limited. For example, it has no authority to access other systems on your network. If your scripts need those capabilities, you can use the Log On tab to change the service so that it runs as a user with the privileges you need to access your domain resources.

Click the Start button to start the service. If it fails to start, the most likely problem is that you got the quotes escaping wrong on the command line. Use the sc.exe program to remove and read the service or update the command line.

If you have a firewall running on your server, you might also need to open the port for svnserve so that it can listen on the port. The default port is tcp/3690.

Testing the Server

Open the Windows command prompt and run the svn info command to verify that you can connect to the server.

CollabNet Subversion

In this example, I used the svn info command and gave it the URL to the repository I created and named dev. Note that the repository name is exposed in the URL, but not the on disk location (C:\repositories). I can access different repositories just by using a slightly different URL. The three repositories I created at the beginning would be accessed via these URLs:

svn://localhost/dev

svn://localhost/mktg

svn://localhost/docs

If you ran these commands from a different machine, you would need to specify the IP address or name of the server in place of localhost. If you used a different port than the default, by using the –listen-port option on the svnserve command, then you would include the port number in the URL:

svn://localhost:3691/dev

Finally, the svnserve server is fairly dynamic. You should be able to create additional repositories, as well as make changes to the configurations of existing repositories, without needing to restart the service. It should recognize the changes immediately. In addition, now that the server is running as a service, besides ensuring that the server starts automatically when the system starts, it is also easy to script the stop and start of the server (using the net stop and net start commands) to accommodate your repository backup routines.

Next Steps

An area I did not touch on in this article is repository configuration. There are still things you need to do, such as configure your users and passwords, as well as the access rules. A good source of information about this is the Version Control with Subversion book mentioned at the beginning of the article. The readme file included in the download of CollabNet Subversion also contains a quick walk though of the steps needed to configure your repository.

Finally, if you run into any problems or need help in getting your repository configured correctly, visit the discussion forums that are available on openCollabNet. There are plenty of users willing to provide you with assistance. Be sure to search the forums for answers as well. It is likely that someone else will have asked similar questions in the past, and it is good etiquette to look for those questions and answers before posting a new one. Of course there are also a number of services that are offered by CollabNet to help get you started and train you and your users on the best usage of Subversion.

Profile photo of admin

by admin

Subversion 1.5 Mergeinfo – Understanding the Internals

March 5, 2009 in Articles, subversion

Product:
Subversion 1.5
Author:
Paul Burba
Summary:

Understanding the internals of Subversion’s merge tracking feature.

This article assumes basic familiarity with the topics covered in chapter 4 of the Subversion book and/or some experience with Subversion 1.5; either the early adopter 1.5 binaries available on open.collab.net or one of the 1.5.0 release candidates.  The examples in this article were done with release candidate 4.  As of May 6, 2008 the most current RC is Subversion 1.5.0 Release Candidate 5.

Mergeinfo

Mergeinfo, or more specifically the versioned property svn:mergeinfo, is the history of merges made into a given file or directory.  This article gives a detailed look at mergeinfo; what it means and how it works.

The svn:mergeinfo property value is simply a newline separated list of merge source paths (relative to the repository root), each source path is followed by a colon, and after the colon a list of revisions previously merged from that source.  If you are curious about the exact grammar, it is specified right in the Subversion code, see the comment ‘Overview of the @c SVN_PROP_MERGEINFO property’ in svn_mergeinfo.h.

Note: This is a long article and covers a lot of atypical use cases.  So if you follow the ‘synch and reintegrate’ paradigm described here in the Subversion Book, then much of this article is probably overkill.  In that case you might want to read only the first few sections through “Explicit Mergeinfo Inheritance” and then skip ahead to the “Parting Thoughts” section.

Explicit Mergeinfo

When a path has the svn:mergeinfo property set on it, that path is said to have explicit mergeinfo.  Explicit mergeinfo is typically created (or modified if it already exists) on the working copy target of a merge (we’ll refer to this simply as the merge target from here on).  I say “usually” because there cases where a merge does not set or modify mergeinfo, see the section “Where’s My Mergeinfo?”.

Explicit mergeinfo may also be created or modified on subtrees of the merge target.  Usually these subtrees have explicit mergeinfo prior to the merge because they were once merge targets themselves.  Though in some cases mergeinfo is created on subtrees where prior to the merge there was no explicit mergeinfo, see the section “Mergeinfo Inheritance and Non-Inheritable Ranges”.

Let’s look at some explicit mergeinfo in the wild, in this case a working copy for Subversion’s own 1.5.x branch.  This branch is a copy of trunk that we split off in preparation of releasing 1.5.  Almost immediately after its creation we start selectively merging changes (i.e. cherry picking) from trunk to the branch:

>svn info \svn\src-1.5.x
Path: \SVN\src-1.5.x
URL:
http://svn.collab.net/repos/svn/branches/1.5.x
Repository Root: http://svn.collab.net/repos/svn
Repository UUID: 612f8ebc-c883-4be0-9ee0-a4e9ef946e3a
Revision: 30056
Node Kind: directory
Schedule: normal
Last Changed Author: hwright
Last Changed Rev: 30056
Last Changed Date: 2008-03-26 00:32:28 -0400 (Wed, 26 Mar 2008)

>svn pg svn:mergeinfo \svn\src-1.5.x –recursive
\SVN\src-1.5.x – /trunk:29085-29089,29091,29094-29107,29111,29114,29117,29126-29127, 29129-29133,29135-29150,29153-29164,29166,29174,29176-29186,29188-29189, 29193-29194,29198-29200,29202-29206,29208-29251,29254-29256,29261, 29267-29273,29277,29280-29281,29284,29287-29303,29305-29307,29309-29325, 29327-29343,29345-29348,29358-29379,29381-29392,29397,29399,29401,29409, 29412,29414-29415,29417-29423,29425-29426,29429,29433-29434,29436-29447, 29449-29466,29468-29478,29482,29484,29486-29487,29489,29491,29493,29496, 29498,29508,29527-29528,29531,29533,29539-29540,29542,29544,29546,29551, 29553,29556,29559,29565,29567-29569,29571-29578,29581,29583,29591,29594, 29600,29603,29607,29611,29613-29614,29619,29623,29625-29626,29630-29631, 29634,29642,29648,29650,29656,29659-29660,29663-29664,29671-29672, 29677-29680,29692,29738-29739,29742-29744,29746,29751,29763,29769-29770, 29784,29787,29797,29801,29821,29824,29828,29835,29855,29868-29869,29878, 29883-29884,29895,29898,29900,29914,29920,29925,29930,29940,29950,29958, 29962,29968,29980,29994-29997,30004,30020

Note: You’ll notice a plague of paths with backslashes in these examples.  Don’t try to adjust your monitor or make an appointment with your optometrist…Yes, I do my Subversion development on Windows.

In the previous example there is only one merge source path, /trunk, followed by the list of revisions previously merged from it to the 1.5.x branch.  Note that unlike the familiar -rX:Y notation used by many Subversion subcommands, the version ranges in svn:mergeinfo use an A-B format in which the revision A is inclusive.  In other words, if you merge -r3:7 expect to see mergeinfo with range 4-7.

Inoperative Vs. Operative Revisions In Mergeinfo

It is important to note that not all revisions listed in the svn:mergeinfo property for a given source need be operative on that source.  Looking at the previous example the mergeinfo for the 1.5.x branch shows that r29093:29107 was merged in from trunk.  But if we look at the log for r29095 we see that it is not a change that affects trunk:

>svn log http://svn.collab.net/repos/svn/trunk -v -r29095
------------------------------------------------------------------------

>

So why is r29095 included in the mergeinfo for 1.5.x?  Because at some point someone merged a range of revisions -rX:Y from trunk to 1.5.x in which X < 29095 <= Y. The development community decided to record mergeinfo for inoperative revisions in cases like this so that the mergeinfo property is less fragmented and therefore easier on human eyes.  Looking at the mergeinfo on 1.5.x you may wonder “What’s the point?  It is already quite fragmented!”.  And you’re right, as 1.5.x is a release branch, but for other use cases, say a feature branch that is eventually reintegrated to trunk, this behavior can keep the mergeinfo nice and tidy, often just a single range.

Where’s My Mergeinfo?

There are a few cases where a merge won’t create or modify mergeinfo:

  • When using the –ignore-ancestry option.  Further, merges using this option do not take mergeinfo into consideration when deciding what to merge.
  • If the merge source is from a different repository from the merge target, a.k.a. foreign repository merges.
  • If reverse merging a change from a path’s own history.  This is the common use case where merge is used to revert a committed change. This works, but mergeinfo is not updated because the current mergeinfo design has no way to explicitly record reverse merges.  A bit confusing?  Here is a simple example, say we checkout a simple working copy like this:

>svn co %url% wc
A    wc\A
A    wc\A\B
A    wc\A\B\lambda
A    wc\A\B\E
A    wc\A\B\E\alpha
A    wc\A\B\E\beta
A    wc\A\B\F
A    wc\A\mu
A    wc\A\C
A    wc\A\D
A    wc\A\D\gamma
A    wc\A\D\G
A    wc\A\D\G\pi
A    wc\A\D\G\rho
A    wc\A\D\G\tau
A    wc\A\D\H
A    wc\A\D\H\chi
A    wc\A\D\H\omega
A    wc\A\D\H\psi
A    wc\iota
Checked out revision 1.

Note that this working copy has no mergeinfo:

>svn pg svn:mergeinfo wc -R

>

Then we make a simple text change and commit it as r2:

>echo 'text change to a file' > wc\A\mu< /FONT >

>svn ci -m "" wc
Sending        wc\A\mu
Transmitting file data .
Committed revision 2.

Oops, we realize we don’t want that change and reverse merge r2 from itself. Subversion always allows you to do this type of merge, regardless of what mergeinfo is set:

>svn merge %url%/A/mu wc/A/mu -c-2
--- Reverse-merging r2 into 'wc\A\mu':
U    wc\A\mu

Since the current mergeinfo implementation has no way of representing reverse merges, the previous merge leaves no mergeinfo evidence:

>svn pg svn:mergeinfo wc -R

>

And there we have a merge that doesn’t set or modify any mergeinfo.

Explicit Mergeinfo Inheritance

Look again at the earlier example examining the mergeinfo on the subversion 1.5.x branch working copy.  Notice the --recursive option?  This means we are getting a list of all the svn:mergeinfo properties set on the entire working copy tree rooted at \SVN\src-1.5.x.  But the only explicit mergeinfo we see is set on \SVN\src-1.5.x.  Surely the numerous merges from trunk to \SVN\src-1.5.x must have affected subtrees of \SVN\src-1.5.x?  Picking a recently merged revision from trunk we can see that this is true:

>svn log -v -q -c30020 http://svn.collab.net/repos/svn/trunk
------------------------------------------------------------------------
r30020 | pburba | 2008-03-24 11:15:48 -0400 (Mon, 24 Mar 2008)
Changed paths:
M /trunk/subversion/libsvn_client/copy.c
------------------------------------------------------------------------

So how does \SVN\src-1.5.x\subversion\libsvn_client\copy.c “know” that r30020 from trunk was merged into it?  It knows via mergeinfo inheritance.  If a path doesn’t have explicit svn:mergeinfo it can still have inherited mergeinfo if it has a parent (or grandparent, or great-grandparent, etc.) with explicit mergeinfo.

The concept of “nearest parent” is not limited to the working copy. When determining the inherited mergeinfo on a path with no explicit mergeinfo, Subversion will first crawl as far up the working copy as it can looking for a parent with explicit mergeinfo.  If it reaches the top of the working copy and can’t find such a parent, it will then ask the repository about any other parent paths, going as far as the root of the repository if necessary.  Only if no inheritable mergeinfo is found in the repository can we finally say the path has no mergeinfo whatsoever (and sometimes we can’t even say it then as we’ll see in the next section).

For a real example of inheritance let’s look again at \SVN\src-1.5.x\subversion\libsvn_client\copy.c.  We know from the mergeinfo on \SVN\src-1.5.x that the merge of r30020 from trunk should have changed this file, but we also know that copy.c doesn’t have any explicit mergeinfo itself.  But since we also know that copy.c’s nearest parent with explicit mergeinfo is \SVN\src-1.5.x, then because of mergeinfo inheritance, copy.c’s inherited mergeinfo is equivalent to that on \SVN\src-1.5.x.

Unfortunately there is no easy way to see this directly from the command line.  1.5 does provide the new svn mergeinfo subcommand, but for 1.5 it provides only a list of revisions previously merged (or are eligible for merging) from a given source:

>svn mergeinfo --show-revs merged http://svn.collab.net/repos/svn/trunk/subversion/libsvn_client/copy.c\svn\src-1.5.x\subversion\libsvn_client\copy.c
r229
r282
r316
r326
r380
.
.
<snipping a *long* list of revisions>
.
.
r28472
r28512
r28825
r29374
r30020

As you can see, the output is more intended for input into a script as each revision is listed individually.  Also note that unlike svn:mergeinfo , only operative revisions are listed; each revision listed in the preceding output actually made a change to http://svn.collab.net/repos/svn/trunk/subversion/libsvn_client/copy.c.

It is easier to see mergeinfo inheritance implicitly.  Let’s try to merge two revisions directly to copy.c in the 1.5.x working copy.  One of those revisions is r30020 which should be ignored since copy.c’s inherited mergeinfo shows that is was already merged.  To find another operative revision eligible for merging we use svn mergeinfo again, but this time with the –show-revs eligible option:

>svn mergeinfo --show-revs eligible http://svn.collab.net/repos/svn/trunk/subversion/libsvn_client/copy.c\svn\src-1.5.x\subversion\libsvn_client\copy.c
r29167
r29961

Ok, let’s merge r29167 and r30020 into copy.c (notice that in 1.5 we can now specify multiple revisions with the merge’s -c option):

>svn merge -c29167,30020 http://svn.collab.net/repos/svn/trunk/subversion/libsvn_client/copy.c\svn\src-1.5.x\subversion\libsvn_client\copy.c
--- Merging r29167 into '\SVN\src-1.5.x\subversion\libsvn_client\copy.c':
U    \SVN\src-1.5.x\subversion\libsvn_client\copy.c

Recall from the Subversion book that one of the key features of 1.5 merge tracking is the avoidance of repeat merges.  From the preceding output we see that Subversion only tried to merge r29167 since it realized that revision r30020, based on copy.c’s inherited mergeinfo, was already merged!

That covers most of the basics of explicit mergeinfo and mergeinfo inheritance.  Before moving on to more advanced topics there are two final points to always keep in mind about mergeinfo and inheritance:

First, if a path has explicit mergeinfo then that mergeinfo fully describes the merges done to that path, the path doesn’t inherit anything else from anywhere.  In other words, inheritance can only come into play if a path has no explicit mergeinfo.

Second, and this is probably terribly obvious but it is worth emphasizing: When a path inherits mergeinfo, it does so only from its nearest parent with explicit mergeinfo.  So if we have a working copy with the following mergeinfo:

>svn pg svn:mergeinfo -R src-branch
src-branch\subversion - /trunk/subversion:30045-30191,30210
src-branch - /trunk:30045-30197
src-branch\www - /trunk/www:30045-30212

What mergeinfo would the file src-branch\subversion\libsvn_repos\reporter.c inherit?  There is mergeinfo on src-branch\www but that is not a parent path of reporter.c.  There is mergeinfo on src-branch and that is a parent of reporter.c, but there is a nearer parent, src-branch\subversion.  So that is where reporter.c gets mergeinfo.

Natural History and Implicit Mergeinfo

Regardless of a path’s explicit or inherited mergeinfo, every path has a natural history and Subversion considers this as implicit mergeinfo.  Honestly, this isn’t something a typical user needs to understand, so if you’re just looking for an overview you can safely skip this topic.

Still here?  Then let’s checkout a working copy for a feature branch that one of the Subversion developers is working on and see implicit mergeinfo in action:

>svn co http://svn.collab.net/repos/svn/branches/in-memory-cache mem-cache-wc -q

By running a couple of log subcommands we can see that this branch was recently made from a copy of Subversion’s trunk in r29755:

>svn log --stop-on-copy -q mem-cache-wc
------------------------------------------------------------------------
r30314 | glasser | 2008-04-04 19:08:12 -0400 (Fri, 04 Apr 2008)
------------------------------------------------------------------------
r30313 | glasser | 2008-04-04 19:07:41 -0400 (Fri, 04 Apr 2008)
------------------------------------------------------------------------
r30312 | glasser | 2008-04-04 19:06:14 -0400 (Fri, 04 Apr 2008)
------------------------------------------------------------------------
.
.
<snipping some of the output for brevity>
.
.
------------------------------------------------------------------------
r29775 | glasser | 2008-03-07 13:57:45 -0500 (Fri, 07 Mar 2008)
------------------------------------------------------------------------
r29773 | glasser | 2008-03-07 13:23:54 -0500 (Fri, 07 Mar 2008)
------------------------------------------------------------------------
r29755 | glasser | 2008-03-06 19:46:43 -0500 (Thu, 06 Mar 2008)
------------------------------------------------------------------------

>svn log -v -r29755 mem-cache-wc
————————————————————————
r29755 | glasser | 2008-03-06 19:46:43 -0500 (Thu, 06 Mar 2008) | 14 lines
Changed paths:
A /branches/in-memory-cache (from /trunk:29754)
<snipping the actual log message>
————————————————————————

Now look at the branch’s mergeinfo:

>svn pg svn:mergeinfo -R mem-cache-wc
mem-cache-wc - /branches/svn-mergeinfo-enhancements:30045-30214
/trunk:29755-30312

Notice how the mergeinfo from /trunk starts at r29755?  What if we tried to merge some revisions prior to r29754?  Will Subversion attempt to merge in those changes from the in-memory-cache’s own history?  Let’s try, instead of doing a normal feature branch synchronization with trunk, in which we don’t specify a revision range, let’s explicitly set a range.

>svn merge http://svn.collab.net/repos/svn/trunk mem-cache-wc -r32000:HEAD
--- Merging r30313 through r30422 into 'mem-cache-wc':
U    mem-cache-wc\COMMITTERS
U    mem-cache-wc\subversion\libsvn_fs_base\tree.c
U    mem-cache-wc\subversion\libsvn_fs_base\bdb\node-origins-table.c
.
.
<snip>
.
.
U    mem-cache-wc\contrib\client-side\svnmucc\svnmucc-test.py
U    mem-cache-wc\configure.ac
G    mem-cache-wc

Look at the notification, Subversion only tried to merge r30313 onwards. From what we’ve seen already we don’t expect Subversion to merge r29754:30312 as that range was already in the explicit mergeinfo.  But what about -r32000:29754?  Why did Subversion not attempt to merge those revisions?  The answer lies in the implicit mergeinfo of /branches/in-memory-cache.  Because /branches/in-memory-cache was copied from /trunk in r29755, the former, as a copy, shares it’s history with /trunk from r29754 and earlier.  Subversion considers this “natural” history as implicit mergeinfo when deciding what to merge from trunk.  Effectively this means /branches/in-memory-cache has the implicit mergeinfo “/trunk:1-29754″ in addition to it’s explicit mergeinfo.  Notice also that the mergeinfo on the in-memory-cache branch only has “/trunk:30313-30422″ added to it after the merge (because mergeinfo describing a path’s own history is redundant):

>svn pg svn:mergeinfo -R mem-cache-wc
mem-cache-wc - /branches/1.5.x-r30215:30238
/trunk:29755-30422

Mergeinfo Inheritance and Non-Inheritable Ranges

Subversion allows a working copies which are incomplete representations of the repository.  This is possible with with shallow checkouts, switched subtrees, or because of authorization restrictions that prevent parts of a tree from being checked out.  You can merge into an incomplete tree if you wish and Subversion endeavors to keep your mergeinfo accurate.  It does this primarily with non-inheritable mergeinfo ranges.  The easiest way to show how these work is with an example:

First we create an incomplete working copy by doing a shallow checkout of the Subversion 1.5.x branch:

Note: The –depth option is also new in 1.5 and restricts operation of Subversion subcommands to a certain depth within the target working copy or URL.  In this case we use the ‘immediates’ value which gives us only the root of the 1.5.x branch plus each immediate file or directory child of the root.

>svn co http://svn.collab.net/repos/svn/branches/1.5.x@30435 1.5.x --depth immediates
A    1.5.x\Makefile.in
A    1.5.x\STATUS
A    1.5.x\build.conf
A    1.5.x\www
A    1.5.x\win-tests.py
A    1.5.x\COMMITTERS
A    1.5.x\TRANSLATING
A    1.5.x\notes
A    1.5.x\README
A    1.5.x\subversion
A    1.5.x\build
A    1.5.x\tools
A    1.5.x\BUGS
A    1.5.x\contrib
A    1.5.x\configure.ac
A    1.5.x\HACKING
A    1.5.x\doc
A    1.5.x\INSTALL
A    1.5.x\COPYING
A    1.5.x\CHANGES
A    1.5.x\autogen.sh
A    1.5.x\gen-make.py
A    1.5.x\aclocal.m4
A    1.5.x\packages
U   1.5.x
Checked out revision 30435.

Now change the depth of the ‘subversion’ directory so the whole subtree is present:

Note: Here we use the –set-depth option, also new in 1.5.  Where –depth is essentially a limit on subcommands in general, –set-depth is an active operation for update.  Here we use the ‘infinity’ value which expands the 1.5.x\subversion subtree to its full extent.

>svn up 1.5.x\subversion --set-depth infinity --quiet

Take a moment to look at the mergeinfo we have on the 1.5.x branch:

>svn pg svn:mergeinfo -R 1.5.x
1.5.x\CHANGES - /branches/1.5.x-r30215/CHANGES:30236,30238,30245,30288
/branches/svn-mergeinfo-enhancements/CHANGES:30122
/trunk/CHANGES:29085-29089,29091,29094-29107,29111,29114,29117,29126-29127,29129-29133,29135-29150,29153-29164,29166-29170,29174,29176-29186,29188-29189,29193-29194,29198-29206,29208-29251,29254-29256,29261,29267-29273,29277,29280-29281,29284,29287-29303,29305-29307,29309-29343,29345-29348,29358-29379,29381-29392,29397,29399,29401,29409,29412,29414-29415,29417-29423,29425-29426,29429,29433-29434,29436-29447,29449-29466,29468-29478,29482,29484,29486-29487,29489,29491,29493,29496,29498,29508,29527-29528,29531,29533,29539-29540,29542,29544,29546,29551,29553,29556,29559,29565,29567-29569,29571-29578,29581,29583,29591,29594,29600,29603,29607,29611,29613-29614,29619,29623,29625-29626,29630-29631,29633-29634,29642,29645,29648,29650,29656,29659-29660,29663-29666,29671-29672,29677-29680,29692,29738-29739,29741-29744,29746,29751,29763,29767,29769-29770,29784,29786-29787,29797,29801,29815,29821,29824,29828,29835,29852,29854-29855,29857-29859,29868-29869,29876,29878,29883-29884,29895,29898,29900,29914,29920,29922,29925,29930,29939-29940,29942,29950,29958,29962,29965,29967-29968,29980,29986,29994-29997,30004,30009,30020,30030,30050,30053-30054,30059,30061-30062,30067,30070,30074,30086,30098,30101,30112,30117,30124,30129-30130,30137,30145,30151,30159,30161-30162,30180-30181,30185,30210,30233,30237,30239,30246,30249,30256,30278-30279,30281,30285,30297,30299,30304,30319-30321,30328,30335-30336,30340,30342,30347,30362,30368,30373,30375,30378,30380,30392,30402,30407-30409,30412
1.5.x - /trunk:29085-29089,29091,29094-29107,29111,29114,29117,29126-29127,29129-29133,29135-29150,29153-29164,29166-29170,29174,29176-29186,29188-29189,29193-29194,29198-29206,29208-29251,29254-29256,29261,29267-29273,29277,29280-29281,29284,29287-29303,29305-29307,29309-29343,29345-29348,29358-29379,29381-29392,29397,29399,29401,29409,29412,29414-29415,29417-29423,29425-29426,29429,29433-29434,29436-29447,29449-29466,29468-29478,29482,29484,29486-29487,29489,29491,29493,29496,29498,29508,29527-29528,29531,29533,29539-29540,29542,29544,29546,29551,29553,29556,29559,29565,29567-29569,29571-29578,29581,29583,29591,29594,29600,29603,29607,29611,29613-29614,29619,29623,29625-29626,29630-29631,29633-29634,29642,29645,29648,29650,29656,29659-29660,29663-29666,29671-29672,29677-29680,29692,29738-29739,29741-29744,29746,29751,29763,29767,29769-29770,29784,29786-29787,29797,29801,29815,29821,29824,29828,29835,29852,29854-29855,29857-29859,29868-29869,29876,29878,29883-29884,29895,29898,29900,29914,29920,29922,29925,29930,29939-29940,29942,29950,29958,29962,29965,29967-29968,29980,29986,29994-29997,30004,30009,30020,30030,30050,30053-30054,30059,30061-30062,30067,30070,30074,30086,30098,30101,30117,30124,30129-30130,30137,30145,30151,30159,30161-30162,30180-30181,30185,30210,30233,30237,30239,30246,30249,30256,30278-30279,30281,30285,30297,30299,30304,30319-30321,30328,30335-30336,30340,30342,30347,30362,30368,30373,30375,30378,30380,30392,30402,30407-30409,30412

1.5.x\CHANGES has had merges done directly to it (i.e. it was once a merge target), so notice that it has its own explicit mergeinfo.  When merging to 1.5.x we refer to 1.5.x\CHANGES as a subtree with differing mergeinfo. We’ll get back to these subtrees later on.

Now let’s merge some changes from trunk into this working copy.  We’ll merge two revisions from trunk in one go.  One revision will make changes to parts of the working copy that are present and one will try to make some changes to paths not present due to the sparse checkout.

The specific revisions we’ll use are r30431 and r30435:

>svn log --verbose --quiet -r30430:30435 http://svn.collab.net/repos/svn/trunk
------------------------------------------------------------------------
r30431 | epg | 2008-04-07 19:40:11 -0400 (Mon, 07 Apr 2008)
Changed paths:
M /trunk/subversion/tests/cmdline/changelist_tests.py
------------------------------------------------------------------------
r30435 | julianfoad | 2008-04-08 09:56:02 -0400 (Tue, 08 Apr 2008)
Changed paths:
A /trunk/notes/tree-conflicts/policy.txt
------------------------------------------------------------------------

Ok, let’s do the merge:

Note: We use –depth infinity here to force the merge into any children of 1.5.x which are present.  Due to the original shallow checkout 1.5.x’s depth is ‘immediates’ and the depth of the merge operation defaults to the depth of the merge target.  If we didn’t specify infinite depth, Subversion wouldn’t even attempt to merge r30435.

>svn merge --depth infinity http://svn.collab.net/repos/svn/trunk 1.5.x -r30430:30435
Skipped missing target: '1.5.x\notes\tree-conflicts\policy.txt'
Skipped missing target: '1.5.x\notes\tree-conflicts'
--- Merging r30431 through r30435 into '1.5.x':
U    1.5.x\subversion\tests\cmdline\changelist_tests.py

Since 1.5.x\notes\tree-conflicts and 1.5.x\notes\tree-conflicts\policy.txt are not present due to the sparse working copy, they are skipped when we attempt to merge r30435.  The file 1.5.x\subversion\tests\cmdline\changelist_tests.py is present though, so the changes from r30431 get merged into that.

Let’s look at what mergeinfo exists on 1.5.x now:

>svn pg svn:mergeinfo 1.5.x
/trunk:29085-29089,29091,29094-29107,29111,29114,29117,29126-29127,29129-29133,29135-29150,29153-29164,29166-29170,29174,29176-29186,29188-29189,29193-29194,29198-29206,29208-29251,29254-29256,29261,29267-29273,29277,29280-29281,29284,29287-29303,29305-29307,29309-29343,29345-29348,29358-29379,29381-29392,29397,29399,29401,29409,29412,29414-29415,29417-29423,29425-29426,29429,29433-29434,29436-29447,29449-29466,29468-29478,29482,29484,29486-29487,29489,29491,29493,29496,29498,29508,29527-29528,29531,29533,29539-29540,29542,29544,29546,29551,29553,29556,29559,29565,29567-29569,29571-29578,29581,29583,29591,29594,29600,29603,29607,29611,29613-29614,29619,29623,29625-29626,29630-29631,29633-29634,29642,29645,29648,29650,29656,29659-29660,29663-29666,29671-29672,29677-29680,29692,29738-29739,29741-29744,29746,29751,29763,29767,29769-29770,29784,29786-29787,29797,29801,29815,29821,29824,29828,29835,29852,29854-29855,29857-29859,29868-29869,29876,29878,29883-29884,29895,29898,29900,29914,29920,29922,29925,29930,29939-29940,29942,29950,29958,29962,29965,29967-29968,29980,29986,29994-29997,30004,30009,30020,30030,30050,30053-30054,30059,30061-30062,30067,30070,30074,30086,30098,30101,30117,30124,30129-30130,30137,30145,30151,30159,30161-30162,30180-30181,30185,30210,30233,30237,30239,30246,30249,30256,30278-30279,30281,30285,30297,30299,30304,30319-30321,30328,30335-30336,30340,30342,30347,30362,30368,30373,30375,30378,30380,30392,30402,30407-30409,30412,30431-30435

We can see that -r30430:30435 from /trunk was added to 1.5.x’s mergeinfo. That makes sense, it is the range we requested in the merge!  Note also that r30432, r30433, r30434, while all inoperative revisions on trunk are still recorded in the mergeinfo as discussed in the “Inoperative Vs. Operative Revisions In Mergeinfo” section.

Now you might be thinking, “Fine, but what happens if we commit this change and then someone else checks out a full 1.5.x working copy?  Won’t their copy of notes\tree-conflicts\policy.txt incorrectly inherit r30430:30435 from the root?  That file was never actually changed in the merge!”  That’s absolutely correct, but I was a bit misleading only looking at the mergeinfo on the root of the working copy earlier.  Let’s look a bit further and check the status of the working copy:

>svn st 1.5.x
M     1.5.x
M     1.5.x\www
M     1.5.x\notes
M     1.5.x\build
M     1.5.x\subversion\tests\cmdline\changelist_tests.py
M     1.5.x\contrib
M     1.5.x\tools
M     1.5.x\doc
M     1.5.x\CHANGES
M     1.5.x\packages

Clearly Subversion did more than it notified us about during the merge, as there are property modifications on each child of 1.5.x that is checked out at depth empty.  So what exactly are all those property modifications?  Let’s look at one using svn diff:

>svn diff 1.5.x\www
Property changes on: 1.5.x\www
___________________________________________________________________
Added: svn:mergeinfo
Merged /trunk/www:r29085-29089,29091,29094-29107,29111,29114,29117,29126-29127,29129-29133,29135-29150,29153-29164,29166-29170,29174,29176-29186,29188-29189,29193-29194,29198-29206,29208-29251,29254-29256,29261,29267-29273,29277,29280-29281,29284,29287-29303,29305-29307,29309-29343,29345-29348,29358-29379,29381-29392,29397,29399,29401,29409,29412,29414-29415,29417-29423,29425-29426,29429,29433-29434,29436-29447,29449-29466,29468-29478,29482,29484,29486-29487,29489,29491,29493,29496,29498,29508,29527-29528,29531,29533,29539-29540,29542,29544,29546,29551,29553,29556,29559,29565,29567-29569,29571-29578,29581,29583,29591,29594,29600,29603,29607,29611,29613-29614,29619,29623,29625-29626,29630-29631,29633-29634,29642,29645,29648,29650,29656,29659-29660,29663-29666,29671-29672,29677-29680,29692,29738-29739,29741-29744,29746,29751,29763,29767,29769-29770,29784,29786-29787,29797,29801,29815,29821,29824,29828,29835,29852,29854-29855,29857-29859,29868-29869,29876,29878,29883-29884,29895,29898,29900,29914,29920,29922,29925,29930,29939-29940,29942,29950,29958,29962,29965,29967-29968,29980,29986,29994-29997,30004,30009,30020,30030,30050,30053-30054,30059,30061-30062,30067,30070,30074,30086,30098,30101,30117,30124,30129-30130,30137,30145,30151,30159,30161-30162,30180-30181,30185,30210,30233,30237,30239,30246,30249,30256,30278-30279,30281,30285,30297,30299,30304,30319-30321,30328,30335-30336,30340,30342,30347,30362,30368,30373,30375,30378,30380,30392,30402,30407-30409,30412,30431-30435*

1.5.x\www now has explicit mergeinfo describing merges from /trunk/www.  Look carefully and you’ll see that the revisions themselves are identical to those on 1.5.x, with one important exception: The range r30430:30435 that was added to 1.5.x in this merge has a ‘*’ suffix for 1.5.x\www. This ‘*’ is the marker for a non-inheritable mergeinfo range.  The ‘*’ means that only the path on which the mergeinfo is explicitly set has had this range merged into it.

I won’t show this in detail, but now each empty child of 1.5.x has its own explicit mergeinfo now.  This mergeinfo is a combination of what the child inherited from 1.5.x prior to the merge, plus the addition of 30431-30435* signifying that those revisions where merged only as far as the empty child.

If we were to commit this change and another user was to do a full --depth infinity checkout of the 1.5.x branch, then their copy of notes\tree-conflicts\policy.txt would inherit all the mergeinfo from notes except 30431-30435, which is a good thing since those revisions were never actually merged into policy.txt.

A similar process happens any time Subversion can’t access a subtree of a merge target.  Subtrees are inaccessible if they are switched to another URL or simply aren’t present on disk.  The latter can be caused by a shallow working copy as above, or may be due to authorization restrictions.  Regardless of the reason a subtree is missing, Subversion always tries to do what we just saw above:

  • Do as much of the merge as is possible.
  • If the parents of any missing paths don’t already have explicit mergeinfo then set some equivalent to the parent’s inherited mergeinfo.
  • Add a non-inheritable range describing the merge to each parent of any missing paths.
  • Set or update the mergeinfo on the merge target describing the merge.

Note: Subtrees missing because they are switched are even a bit more complicated than what is described here, since the switched paths themselves also get explicit mergeinfo, but I’ll leave this for a future article.

Now typically most users won’t merge to a target with missing subtrees, but if you need to, rest assured that Subversion attempts to keep the explicit/inherited mergeinfo on each path an accurate reflection of only what was merged to that path.

Empty Mergeinfo

Empty mergeinfo is a svn:mergeinfo property which has the empty string as a value.  It simply means “the state of this path is as if nothing was ever merged into it”.  Empty mergeinfo typically occurs when doing a working copy to working copy move or copy (the subject of a forthcoming Submerged post) or when reverse merging all prior merges out of a subtree.  Let’s look at an example of the latter:

Our current working directory is a simple branch:

>svn ls -R .
B/
B/E/
B/E/alpha
B/E/beta
B/F/
B/lambda
C/
D/
D/G/
D/G/pi
D/G/rho
D/G/tau
D/H/
D/H/chi
D/H/omega
D/H/psi
D/gamma
mu

Looking at the mergeinfo we see that several revisions were merged in to this branch already:

>svn pl -vR .
Properties on '.':
svn:mergeinfo : /A:2-6

What happens if we reverse merge all of those revisions out of subtree of the branch?

>svn merge %url92%/A/D/H .\D\H -r6:1
--- Reverse-merging r6 through r2 into 'D\H':
U    D\H\omega
U    D\H\psi
>svn pl -vR .
Properties on '.':
svn:mergeinfo : /A:2-6
Properties on 'D\H':
svn:mergeinfo :

Empty mergeinfo is what happens.  If .\D\H didn’t have explicit empty mergeinfo, it would incorrectly inherit the mergeinfo /A/D/H:2-6 from ‘.’.

Mergeinfo Elision

At the end of every merge Subversion tries to “consolidate” any redundant subtree mergeinfo.  This consolidation process is called elision.  Once a merge is completed, Subversion walks the working copy tree rooted at the merge target.  If it finds a path with explicit mergeinfo which has a subtree with equivalent explicit mergeinfo, then the subtree’s mergeinfo is elided (removed).  “Equivalency” here is defined this way:

  • Path PARENT has explicit mergeinfo PXM.
  • PARENT  has a subtree CHILD with explicit mergeinfo CXM.
  • Imagine for a moment that CHILD has no explicit mergeinfo, then CIM is the mergeinfo CHILD inherits from PARENT.
  • If CIM is exactly the same as CXM, then CXM is equivalent to PXM and CXM can elide.

In other words, removing the subtree’s mergeinfo is safe to do since, if the subtree’s mergeinfo is equivalent to its nearest parent with explicit mergeinfo, then the mergeinfo the subtree inherits from that parent is already sufficient to describe the merges to the subtree.  It’s probably becoming clear now that mergeinfo inheritance and elision are just two ways of looking at the same thing:  Inhertance is mergeinfo sliding “down” the tree from parent to child, elision is mergeinfo sliding “up” the tree from child to parent.

Enough theory, let’s take a look at elision in action.  In this example we’ll checkout a working copy for a current Subversion feature branch:

>svn co http://svn.collab.net/repos/svn/branches/dont-save-plaintext-passwords-by-default no_pass_wc -q

>svn pg svn:mergeinfo -R no_pass_wc
no_pass_wc – /branches/1.5.x-r30215:30238
/branches/diff-callbacks3:29985-30687
/branches/svn-mergeinfo-enhancements:30045-30214
/trunk:30654-30731

The developers of this branch have periodically synched it up with Subversion’s trunk.  But development on trunk happens almost constantly, and using the svn mergeinfo subcommand we see that there are some revisions on trunk available for merging:

>svn mergeinfo --show-revs eligible http://svn.collab.net/repos/svn/trunk no_pass_wc
r30735
r30736
r30738
r30741
r30743
r30745
r30746
r30747
r30748
r30749
r30750
r30751
r30753
r30754
r30756

Let’s say we are working on this branch and we are a Windows developer.  We know that a change on trunk (r30754) is needed to build the branch on Windows, but we don’t want to merge all available changes from trunk just now (perhaps we know that there will be many conflicts to resolve and we are not ready to do that).  We decide to merge just the small fix we need directly to the offending file build.conf on our branch:

>svn merge http://svn.collab.net/repos/svn/trunk/build.confno_pass_wc\build.conf -c30754
--- Merging r30754 into 'no_pass_wc\build.conf':
U    no_pass_wc\build.conf

As expected the diff of the branch shows that explicit mergeinfo was added to build.conf and it looks equivalent to mergeinfo on the root of the working copy with the exception of r30754.

>svn diff no_pass_wc
Index: no_pass_wc/build.conf
===================================================================
--- no_pass_wc/build.conf       (revision 30756)
+++ no_pass_wc/build.conf       (working copy)
<snip text diff>
Property changes on: no_pass_wc\build.conf
___________________________________________________________________
Added: svn:mergeinfo
Merged /branches/diff-callbacks3/build.conf:r29985-30687
Merged /trunk/build.conf:r30654-30731,30754
Merged /branches/1.5.x-r30215/build.conf:r30238
Merged /branches/svn-mergeinfo-enhancements/build.conf:r30045-30214

Later we decide that we are ready to synch up with trunk, and do that, momentarily forgetting about the merge we’ve already done to build.conf:

>svn merge http://svn.collab.net/repos/svn/trunk no_pass_wc
--- Merging r30732 through r30753 into 'no_pass_wc':
U    no_pass_wc\subversion\include\svn_client.h
A    no_pass_wc\subversion\include\private\svn_opt_private.h
U    no_pass_wc\subversion\include\svn_opt.h
U    no_pass_wc\subversion\libsvn_wc\status.c
U    no_pass_wc\subversion\libsvn_subr\opt.c
U    no_pass_wc\subversion\libsvn_subr\mergeinfo.c
A    no_pass_wc\subversion\libsvn_client\cmdline.c
U    no_pass_wc\subversion\libsvn_client\merge.c
U    no_pass_wc\subversion\libsvn_client\prop_commands.c
U    no_pass_wc\subversion\libsvn_client\mergeinfo.h
U    no_pass_wc\subversion\tests\libsvn_client\client-test.c
U    no_pass_wc\subversion\tests\cmdline\special_tests.py
U    no_pass_wc\subversion\tests\cmdline\basic_tests.py
U    no_pass_wc\subversion\tests\cmdline\merge_tests.py
U    no_pass_wc\subversion\tests\cmdline\depth_tests.py
U    no_pass_wc\subversion\svn\merge-cmd.c
U    no_pass_wc\subversion\svn\cl.h
U    no_pass_wc\subversion\svn\propdel-cmd.c
U    no_pass_wc\subversion\svn\checkout-cmd.c
U    no_pass_wc\subversion\svn\move-cmd.c
U    no_pass_wc\subversion\svn\mkdir-cmd.c
U    no_pass_wc\subversion\svn\cat-cmd.c
U    no_pass_wc\subversion\svn\revert-cmd.c
U    no_pass_wc\subversion\svn\diff-cmd.c
U    no_pass_wc\subversion\svn\copy-cmd.c
U    no_pass_wc\subversion\svn\mergeinfo-cmd.c
U    no_pass_wc\subversion\svn\list-cmd.c
U    no_pass_wc\subversion\svn\util.c
U    no_pass_wc\subversion\svn\blame-cmd.c
U    no_pass_wc\subversion\svn\propget-cmd.c
U    no_pass_wc\subversion\svn\changelist-cmd.c
U    no_pass_wc\subversion\svn\log-cmd.c
U    no_pass_wc\subversion\svn\update-cmd.c
U    no_pass_wc\subversion\svn\resolved-cmd.c
U    no_pass_wc\subversion\svn\cleanup-cmd.c
U    no_pass_wc\subversion\svn\commit-cmd.c
U    no_pass_wc\subversion\svn\add-cmd.c
U    no_pass_wc\subversion\svn\propset-cmd.c
U    no_pass_wc\subversion\svn\switch-cmd.c
U    no_pass_wc\subversion\svn\delete-cmd.c
U    no_pass_wc\subversion\svn\import-cmd.c
U    no_pass_wc\subversion\svn\proplist-cmd.c
U    no_pass_wc\subversion\svn\resolve-cmd.c
U    no_pass_wc\subversion\svn\export-cmd.c
U    no_pass_wc\subversion\svn\status-cmd.c
U    no_pass_wc\subversion\svn\propedit-cmd.c
U    no_pass_wc\subversion\svn\lock-cmd.c
U    no_pass_wc\subversion\svn\info-cmd.c
U    no_pass_wc\subversion\svn\unlock-cmd.c
U    no_pass_wc\subversion\libsvn_fs_fs\structure
--- Merging r30754 through r30757 into 'no_pass_wc':
U    no_pass_wc\subversion\include\svn_config.h
U    no_pass_wc\subversion\libsvn_wc\merge.c

Notice that Subversion realizes r30754 was already merged to build.conf and doesn’t attempt to repeat that portion of the merge.  Not only that, but once the merge is done, Subversion notices that the mergeinfo on the root of the working copy and the mergeinfo on build.conf is equivalent and elides the latter away.  We can see that this is true in the status of the working copy or by looking at the svn:mergeinfo property directly:

>svn st no_pass_wc
M     no_pass_wc
M     no_pass_wc\build.conf
<snip>
M     no_pass_wc\subversion\svn\unlock-cmd.c
M     no_pass_wc\subversion\libsvn_fs_fs\structure

>svn pg svn:mergeinfo -R no_pass_wc
no_pass_wc – /branches/1.5.x-r30215:30238
/branches/diff-callbacks3:29985-30687
/branches/svn-mergeinfo-enhancements:30045-30214
/trunk:30654-30757

If we could have stopped subversion after the merge was completed but before elision occurred we would have seen this mergeinfo:

no_pass_wc - /branches/1.5.x-r30215:30238
/branches/diff-callbacks3:29985-30687
/branches/svn-mergeinfo-enhancements:30045-30214
/trunk:30654-30757
no_pass_wc\build.conf - /branches/1.5.x-r30215/build.conf:30238
/branches/diff-callbacks3/build.conf:29985-30687
/branches/svn-mergeinfo-enhancements/build.conf:30045-30214
/trunk/build.conf:30654-30757

Because the mergeinfo on no_pass_wc/build.conf is equivalent to that on no_pass_wc, the former is elided away.

Record Only Merges

A new merge subcommand option in 1.5 is –record-only.  Merges done with the --record-only option don’t attempt to merge anything, but they do record and elide mergeinfo as if a real merge took place.  This is useful for making it appear that a change was merged without actually merging it (blocking) and also for cleaning up subtree mergeinfo.

Blocking in 1.5 is quite simple, just use –record-only to merge a revision and Subversion makes the mergeinfo look like a merge has taken place.  From then on, Subversion sees the mergeinfo and acts as if the revision really was merged, blocking future attempts to merge it.  For example, looking again at the clean checkout from our previous example, we first merge an eligible revision without --record-only:

>svn merge http://svn.collab.net/repos/svn/trunk no_pass_wc -c30757
--- Merging r30757 into 'no_pass_wc':
U    no_pass_wc\subversion\include\svn_config.h

As expected this merge updates the mergeinfo on the merge target:

>svn st no_pass_wc
M   no_pass_wc
M   no_pass_wc\subversion\include\svn_config.h

>svn diff no_pass_wc –depth empty
Property changes on: no_pass_wc
___________________________________________________________________
Modified: svn:mergeinfo
Merged /trunk:r30757

Now we revert that merge and repeat it, but this time using --record-only:

>svn revert -R no_pass_wc
Reverted 'no_pass_wc'
Reverted 'no_pass_wc\subversion\include\svn_config.h'

>svn merge http://svn.collab.net/repos/svn/trunk no_pass_wc -c30757 –record-only

>

There is no output from the merge, which makes sense as nothing was merged.  The mergeinfo has changed though, as a diff shows:

>svn diff no_pass_wc
Property changes on: no_pass_wc
___________________________________________________________________
Modified: svn:mergeinfo
Merged /trunk:r30757

If we try the merge yet again without --record-only we see that it is blocked:

>svn merge http://svn.collab.net/repos/svn/trunk no_pass_wc -c30757

>

Note: 1.5 blocking isn’t what some refer to as “true blocking”, where it is readily obvious what was actually merged vs. what was only blocked.  It is possible in 1.5 to determine if all the sources and ranges represented by some path’s given mergeinfo were actually merged or only blocked, but this is not always trivial.  The development community is considering some type of true blocking for a future release.

As to using --record-only to clean up subtree mergeinfo, let’s look at a simple example.  Checkout a very simple repository:

>svn co %SIMPLEURL% simple_wc
A    simple_wc\A
A    simple_wc\A\B
A    simple_wc\A\B\lambda
A    simple_wc\A\B\E
A    simple_wc\A\B\E\alpha
A    simple_wc\A\B\E\beta
A    simple_wc\A\B\F
A    simple_wc\A\mu
A    simple_wc\A\C
A    simple_wc\A\D
A    simple_wc\A\D\gamma
A    simple_wc\A\D\G
A    simple_wc\A\D\G\pi
A    simple_wc\A\D\G\rho
A    simple_wc\A\D\G\tau
A    simple_wc\A\D\H
A    simple_wc\A\D\H\chi
A    simple_wc\A\D\H\omega
A    simple_wc\A\D\H\psi
A    simple_wc\iota
A    simple_wc\A_branch
A    simple_wc\A_branch\B
A    simple_wc\A_branch\B\lambda
A    simple_wc\A_branch\B\E
A    simple_wc\A_branch\B\E\alpha
A    simple_wc\A_branch\B\E\beta
A    simple_wc\A_branch\B\F
A    simple_wc\A_branch\mu
A    simple_wc\A_branch\C
A    simple_wc\A_branch\D
A    simple_wc\A_branch\D\gamma
A    simple_wc\A_branch\D\G
A    simple_wc\A_branch\D\G\pi
A    simple_wc\A_branch\D\G\rho
A    simple_wc\A_branch\D\G\tau
A    simple_wc\A_branch\D\H
A    simple_wc\A_branch\D\H\chi
A    simple_wc\A_branch\D\H\omega
A    simple_wc\A_branch\D\H\psi
Checked out revision 4.

From the log we see that this repos consists of a tree rooted at A which was added in r1, copied to a branch in r2, and then modified in r3 and r4:

>svn log --verbose -r1:HEAD simple_wc
------------------------------------------------------------------------
r1 | jrandom | 2008-04-23 13:00:56 -0400 (Wed, 23 Apr 2008) | 1 line
Changed paths:
A /A
A /A/B
A /A/B/E
A /A/B/E/alpha
A /A/B/E/beta
A /A/B/F
A /A/B/lambda
A /A/C
A /A/D
A /A/D/G
A /A/D/G/pi
A /A/D/G/rho
A /A/D/G/tau
A /A/D/H
A /A/D/H/chi
A /A/D/H/omega
A /A/D/H/psi
A /A/D/gamma
A /A/mu
A /iota
Log message for revision 1.
------------------------------------------------------------------------
r2 | pburba | 2008-04-23 13:02:04 -0400 (Wed, 23 Apr 2008) | 1 line
Changed paths:

A /A_branch (from /A:1)
Make a branch from A
------------------------------------------------------------------------
r3 | pburba | 2008-04-23 13:02:44 -0400 (Wed, 23 Apr 2008) | 1 line
Changed paths:
M /A/D/H/psi
------------------------------------------------------------------------
r4 | pburba | 2008-04-23 13:03:09 -0400 (Wed, 23 Apr 2008) | 1 line
Changed paths:
M /A/B/E/beta
------------------------------------------------------------------------

First we merge r3 directly to A_branch\D\H\psi:

>svn merge %SIMPLEURL%/A/D/H/psi simple_wc\A_branch\D\H\psi -c3
--- Merging r3 into 'simple_wc\A_branch\D\H\psi':
U    simple_wc\A_branch\D\H\psi

>svn ci -m “merged r3 to A_branch/D/H/psi” simple_wc
Sending        simple_wc\A_branch\D\H\psi
Transmitting file data .
Committed revision 5.

>svn pl -vR merge_tests-92

>svn pl -vR simple_wc
Properties on ‘simple_wc\A_branch\D\H\psi':
svn:mergeinfo : /A/D/H/psi:3

Then we merge r4 to the root of the branch:

>svn merge %SIMPLEURL%/A simple_wc\A_branch -c4
--- Merging r4 into 'simple_wc\A_branch':
U    simple_wc\A_branch\B\E\beta

>svn pl -vR simple_wc
Properties on 'simple_wc\A_branch':
svn:mergeinfo : /A:4
Properties on 'simple_wc\A_branch\D\H\psi':
svn:mergeinfo : /A/D/H/psi:3-4

Since simple_wc is at revision 5 in the repository we must update the working copy to avoid an out-of-date error whencommitting:

>svn up simple_wc
At revision 5.
>svn ci -m "" simple_wc
Sending        simple_wc\A_branch
Sending        simple_wc\A_branch\B\E\beta
Sending        simple_wc\A_branch\D\H\psi
Transmitting file data .
Committed revision 6.

Notice that the subtree of the merge target, A_branch\D\H\psi got its mergeinfo updated too:

>svn pl --verbose -R simple_wc
Properties on 'simple_wc\A_branch':
svn:mergeinfo : /A:4
Properties on 'simple_wc\A_branch\D\H\psi':
svn:mergeinfo : /A/D/H/psi:3-4

Oops, we just remembered that our company policy is to merge only to the root of branches so as to keep the explicit mergeinfo consolidated there. We forgot about that when we did the first merge and now our pointy haired boss is giving us heat.  How can we fix this?  Well, we know that r3 affects only A/D/H/psi, so if we added “/A:r3″ to the mergeinfo for A_branch then our mergeinfo is still semantically equivalent to what we have now.  Of course if we did that, the mergeinfo on A_branch\D\H\psi is then equivalent to that on A_branch and could elide.  We can do this in one quick step with a –record-only merge. First though we need to update the working copy (see “Mixed Revision Working Copies” for why this is necessary):

>svn up simple_wc
At revision 6.

>svn merge %SIMPLEURL%/A simple_wc\A_branch -c3 –record-only

>svn st simple_wc
M     simple_wc\A_branch
M     simple_wc\A_branch\D\H\psi

>svn pl -vR simple_wc
Properties on ‘simple_wc\A_branch':
svn:mergeinfo : /A:3-4

See that the --record-only merge added r3 to the mergeinfo on A_branch and then elided the equivalent mergeinfo on A_branch\D\H\psi.  Time to ask the PHB for a raise!

Mixed Revision Working Copies and Mergeinfo

One of Subversion’s basic design principles is to be as flexible as possible.  Often flexibility also means greater complexity and with greater complexity there is always the possibility for confusion.  Such is the case with mixed revision working copies and mergeinfo:

The flexibility is: Subversion allows you to merge into mixed-revision working copies.

The complexity is: Mergeinfo inheritance and elision are dependent on uniform working revisions across the working copy.

The confusion is: Mergeinfo inheritance and elision may not work like you expect when dealing with a mixed revision working copy.

The problems are most likely to arise when doing “subtree merges” (i.e. merges not to the root of a branch, but to a subtree of the root).  For the following example we use the sample repository from openCOLLABNET’s merge-tracking project .

First we checkout a new working copy of the sample repository:

>svn co %URL% wc --quiet

The repos is structured in fairly common way, with trunk, branches, and tags folders off the root…

>svn ls wc
branches/
tags/
trunk/

…and several copies of trunk under branches:

>svn ls wc\branches
a/
b/
c/

We are going to work with the c branch, so let’s look at the mergeinfo there:

>svn pg svn:mergeinfo -R wc\branches\c
wc\branches\c - /branches/a:3-11
/branches/b:10-13
/trunk:5-14

Now we make some changes to trunk:

Doing some work...
>svn ci -m "some changes under trunk" wc
Sending        wc\trunk\jobs\index.html
Transmitting file data .
Committed revision 18.

Doing some work...
>svn ci -m "some changes under trunk" wc
Sending        wc\trunk\jobs\index.html
Transmitting file data .
Committed revision 19.

Doing some work...
>svn ci -m "some changes under trunk" wc
Sending        wc\trunk\about\index.html
Sending        wc\trunk\jobs\index.html
Transmitting file data ..
Committed revision 20.

Now to start merging.  Assume we need to merge the changes from r18, r19, and r20 to the c branch, but we only want the changes that affect the jobs subtree, not the changes to the about subtree, so we merge the changes directly to branches\c\jobs:

>svn merge %URL%/trunk/jobs wc\branches\c\jobs -c18,19,20
--- Merging r18 into 'wc\branches\c\jobs':
U    wc\branches\c\jobs\index.html
--- Merging r19 into 'wc\branches\c\jobs':
G    wc\branches\c\jobs\index.html
--- Merging r20 into 'wc\branches\c\jobs':
G    wc\branches\c\jobs\index.html

This of course means that branches\c\jobs now has it’s own explicit mergeinfo that differs from branches\c:

>svn pg svn:mergeinfo -R wc\branches\c
wc\branches\c - /branches/a:3-11
/branches/b:10-13
/trunk:5-14
wc\branches\c\jobs - /branches/a/jobs:3-11
/branches/b/jobs:10-13
/trunk/jobs:5-14,18-20

Once we commit the merge notice that we now have a mixed revision working copy:

>svn ci -m "Merged some changes from trunk to a subtree of c branch" wc
Sending        wc\branches\c\jobs
Sending        wc\branches\c\jobs\index.html
Transmitting file data .
Committed revision 25.

>svnversion wc\branches\c
17:25

Revision 25?  Looks like some other developer made changes to the repository resulting in new revisions r21-r24 in the time between our initial checkout and this commit.

Later we need to merge the other changes from r20 to branches\c that touch the about subtree.  Recalling our boss’ rant about merging to the root of branches, we decide to merge r20 to the root of the branch.  We also elect to re-merge r18 and r19, knowing that the merge tracking logic should prevent a repeated merge and also elide away the mergeinfo on branches\c\jobs:

>svn merge %URL%/trunk wc\branches\c -c18,19,20
--- Merging r20 into 'wc\branches\c':
U    wc\branches\c\about\index.html

Before we commit we check the mergeinfo on the branches\c.  What’s this? branches\c\jobs\index.html still has mergeinfo, and it looks perfectly equivalent to that on branches\c?  Why didn’t the mergeinfo on branches\c\jobs elide to branches\c?

>svn pg svn:mergeinfo -R wc\branches\c
wc\branches\c - /branches/a:3-11
/branches/b:10-13
/trunk:5-14,18-20
wc\branches\c\jobs - /branches/a/jobs:3-11
/branches/b/jobs:10-13
/trunk/jobs:5-14,18-20

The problem lies with our mixed revision working copy:

>svn st -v wc\branches\c
M              17       16 cuser        wc\branches\c
17       15 merger       wc\branches\c\products
17        2 user         wc\branches\c\products\little.html
17       15 merger       wc\branches\c\products\medium.html
17        2 user         wc\branches\c\products\big.html
17       15 merger       wc\branches\c\products\roadmap.html
17       15 merger       wc\branches\c\products\index.html
17       15 merger       wc\branches\c\about
M              17       15 merger       wc\branches\c\about\index.html
17       15 merger       wc\branches\c\index.html
17       15 merger       wc\branches\c\news
17       15 merger       wc\branches\c\news\index.html
17        2 user         wc\branches\c\support
17        2 user         wc\branches\c\support\index.html
25       25 pburba       wc\branches\c\jobs
25       25 pburba       wc\branches\c\jobs\index.html

Notice that branches\c\jobs is at working revision 25, while the rest of branches\c is at r17.  Subversions elision logic won’t try to elide the mergeinfo on branches\c\jobs up the working copy to branches\c because the two are at different working revisions.  Why not?  Because there is no way of knowing if the mergeinfo found on branches\c@17 is the same as branches\c@25.  It’s possible the changes our colleague made in r21-r24 changed to mergeinfo of branches\c.  If this were the case and we try to commit this merge we’d get an out of date error and have to update the working copy before committing.

We can check if r21-r24 affect the branches\c branch with a –show-updates status:

>svn st --show-updates --verbose wc\branches\c
17        2 user         wc\branches\c\products\little.html
17       15 merger       wc\branches\c\products\medium.html
17        2 user         wc\branches\c\products\big.html
17       15 merger       wc\branches\c\products\roadmap.html
17       15 merger       wc\branches\c\products\index.html
17       15 merger       wc\branches\c\products
M              17       15 merger       wc\branches\c\about\index.html
17       15 merger       wc\branches\c\about
17       15 merger       wc\branches\c\index.html
17       15 merger       wc\branches\c\news\index.html
17       15 merger       wc\branches\c\news
17        2 user         wc\branches\c\support\index.html
17        2 user         wc\branches\c\support
25       25 pburba       wc\branches\c\jobs\index.html
25       25 pburba       wc\branches\c\jobs
M              17       16 cuser        wc\branches\c
Status against revision:     25

Nope, no one has made changes to branches\c, if they had we’d see the ‘*’ out of date marker in the 8th column.  So we decide to update branches\c:

>svn up wc\branches\c
At revision 25.

Now the branch is at a uniform working revision (and still has the local modifications from the uncommitted merge):

>svnversion wc\branches\c
25M

We still want to get rid of that redundant explicit mergeinfo on branches/c/jobs so we repeat the merge, relying on the merge tracking logic to avoid any repeated merge (we could also use a --record-only merge here, in this case there is no effective difference between the two):

>svn merge %URL%/trunk wc\branches\c -c18,19,20

>

Good, no merging occurred, but sure enough, mergeinfo elision, now that the working copy is at a uniform revision, has taken place:

>svn st wc\branches\c -v
M              25       25 pburba       wc\branches\c
25       15 merger       wc\branches\c\products
25        2 user         wc\branches\c\products\little.html
25       15 merger       wc\branches\c\products\medium.html
25        2 user         wc\branches\c\products\big.html
25       15 merger       wc\branches\c\products\roadmap.html
25       15 merger       wc\branches\c\products\index.html
25       15 merger       wc\branches\c\about
M              25       15 merger       wc\branches\c\about\index.html
25       15 merger       wc\branches\c\index.html
25       15 merger       wc\branches\c\news
25       15 merger       wc\branches\c\news\index.html
25        2 user         wc\branches\c\support
25        2 user         wc\branches\c\support\index.html
M              25       25 pburba       wc\branches\c\jobs
25       25 pburba       wc\branches\c\jobs\index.html

>svn pg svn:mergeinfo -R wc\branches\c
wc\branches\c - /branches/a:3-11
/branches/b:10-13
/trunk:5-14,18-20

This example demonstrated problems with mergeinfo elision.  As mentioned previously elision and inheritance are essentially the same thing so you can see problems with mixed-revision working copies involving inheritance too.  Let’s say we have the following mixed-revision working copy sometime after we’ve committed the previous merge as r26:

>svn st -v wc\branches\c
26       26 pburba       wc\branches\c
26       15 merger       wc\branches\c\products
26        2 user         wc\branches\c\products\little.html
26       15 merger       wc\branches\c\products\medium.html
26        2 user         wc\branches\c\products\big.html
26       15 merger       wc\branches\c\products\roadmap.html
26       15 merger       wc\branches\c\products\index.html
25       15 merger       wc\branches\c\about
26       26 pburba       wc\branches\c\about\index.html
26       15 merger       wc\branches\c\index.html
26       15 merger       wc\branches\c\news
26       15 merger       wc\branches\c\news\index.html
26        2 user         wc\branches\c\support
26        2 user         wc\branches\c\support\index.html
26       26 pburba       wc\branches\c\jobs
26       25 pburba       wc\branches\c\jobs\index.html

>svn pg svn:mergeinfo -R wc\branches\c
wc\branches\c - /branches/a:3-11
/branches/b:10-13
/trunk:2,5-14,18-20

If we try to merge revisions 17:20 directly into branches\c\about, Subversion tries to figure out what the mergeinfo for branches\c\about is so it can avoid any repeat merges.  Unfortunately it can’t inherit the explicit mergeinfo from branches\c because that is at a different working revision.  So instead it Subversion asks the server what the explicit or inherited mergeinfo for branches\c\about@25 is.  That path has no explicit mergeinfo at that revision but does inherit mergeinfo from branches\c@25, but this doesn’t include r18-20 since that was not committed until r26!  So Subversion thinks r18-20wasn’t applied to the tree rooted at branches/c/about and repeats the merge:

>svn merge %URL%/trunk/about wc\branches\c\about -c18,19,20
--- Merging r20 into 'wc\branches\c\about':
U    wc\branches\c\about\index.html

In this case all that happens is that branches\c\about gets explicit mergeinfo, though a conflict from the repeated merge could easily occur:

>svn st wc\branches\c
M     wc\branches\c\about

>svn diff wc\branches\c
Property changes on: wc\branches\c\about
___________________________________________________________________
Added: svn:mergeinfo
Merged /branches/b/about:r10-13
Merged /trunk/about:r2,5-14,18-20
Merged /branches/a/about:r3-11

If we revert that merge and update the merge target to a uniform working copy revision and then repeat the merge we see that nothing happens:

>svn revert -R wc
Reverted 'wc\branches\c\about'

>svn up wc
At revision 26.

>svn merge %URL%/trunk/about wc\branches\c\about -c18,19,20

>svn st wc

>

This is because branches\c\about could now inherit the explicit mergeinfo from branches\c, which included r18-20.  Seeing that these revisions were already merged to branches\c\about Subversion doesn’t attempt any merge at all.

Admittedly both of these example are a bit contrived, and you may never run into anything like them.  But if you see mysterious behavior regarding mergeinfo always check if a mixed-revision working copy is the culprit.

The author sheepishly admits to being confounded a few times by the intersection of mixed-rev working copies and mergeinfo inheritance/elision.  Once even going as far as starting to write up a new issue in Subversions issue tracker before realizing what was going on…

Parting Thoughts

Whew, that covered a lot of ground!  Hopefully you have a better understanding of how mergeinfo works, particularly in some atypical use cases.  But what if you’d simply like to avoid this whole lot of complexities?  Depending on your needs it isn’t so hard to do.  If they are compatible with your development processes, these following rules will keep your mergeinfo (and your merges) as straightforward as possible:

  • For feature branches (those that are copied from a “trunk” to a “branch” and eventually merged back to “trunk”) use the process described in the Subversion book: Keep your branch synchronized with trunk and use --reintegrate when merging the branch back to trunk.
  • For release branches merge only to the root of the branch, forgoing subtree merges.  This helps keep your explicit mergeinfo consolidated.
  • Don’t merge into mixed-revision working copies if at all possible.  A quick update to the merge target before merging can avoid a lot of confusion.
  • Don’t “hand edit” svn:mergeinfo with svn propset or svn propedit.  The svn command line client does syntax checking of any mergeinfo you try to change with these subcommands, but it doesn’t check for semantic correctness.  It is awfully easy to mess-up your mergeinfo and end up with, oh, mergeinfo that refers to non-existent paths in your repository.  Instead use svn merge with the --record-only option.  That is adequate to fix most problems and is a lot less likely to leave you with any headaches.
  • Don’t merge to targets that have switched subtrees.
  • Don’t merge from sources which you don’t have complete authorization to and/or to targets you don’t have complete authorization to.
  • Don’t merge to non-infinite --depths (‘empty‘, ‘files‘, or ‘immediates‘).

Yes, that’s a lot of don’ts, and here’s one more: Don’t hesitate to do all of these things if you need to. As I said at the start, Subversion always attempts to “Do the Right Thing” for you regarding mergeinfo, and hopefully this post will help you to do the same.

Profile photo of admin

by admin

CollabNet and Subversion

March 5, 2009 in Articles, subversion

This document outlines Subversion from beginning to today.