tonyduckles / svn2svn Goto Github PK
View Code? Open in Web Editor NEWReplicate/replay Subversion revision history
License: GNU General Public License v3.0
Replicate/replay Subversion revision history
License: GNU General Public License v3.0
When running under cmd.exe (CPython 2.7.2 on Windows XP 64), the color escape codes aren't interpreted and make debug output rather messy. I ended up just commenting out:
#if color in _colors:
# msg = '%s%s%s' % ("\x1b["+_colors[color]+(";1" if bold else "")+"m", msg, "\x1b[0m")
in ui.py to get going. I'm not aware of the coding necessary for cmd.exe, so perhaps it's easiest to just check the SHELL environment variable or os.name and disable it.
I don't think the current tests/make-ref-repo.sh script covers all the code-paths in svn2svn/run/svn2svn.py. The automated tests should have full code-coverage, to make sure we're testing all the nooks and crannies.
Add support for running an optional external shell command before each commit, for doing any pre-commit filtering.
There are some "TODO" questions in do_svn_add_dir(). Need to follow-up on those.
Do better KeyboardInterrupt try/except handling in real_main(). It's fine for a process_svn_log_entry
call to be interrupted, but other calls here should be considered atomic, e.g. commit_from_svn_log_entry
.
Traceback (most recent call last):
File "C:\stephen\working\svnreplay.py", line 5, in
sys.exit(main() or 0)
File "C:\stephen\working\svn2svn\run\svnreplay.py", line 1106, in main
return real_main(args)
File "C:\stephen\working\svn2svn\run\svnreplay.py", line 910, in real_main
target_rev = commit_from_svn_log_entry(source_start_log, target_revprops=target_revprops)
File "C:\stephen\working\svn2svn\run\svnreplay.py", line 103, in commit_from_svn_log_entry
rev_num = parse_svn_commit_rev(output) if output else None
File "C:\stephen\working\svn2svn\run\svnreplay.py", line 46, in parse_svn_commit_rev
return int(rev_num)
ValueError: invalid literal for int() with base 10: '2.\r'
If options.keep_date is True, should we try doing a "svn propset
" on an existing revision as a sanity check, so we check if the pre-revprop-change hook script is correctly setup before doing first replay-commit?
Come across this issue when replaying SVN repo with symbolic links.
Code in in_svn
function uses svn status
to check is file is under Subversion control, but svn status
, svn info
and possibly other commands seem to follow symbolic links when called directly on them.
Traceback (most recent call last):
File ".../svn2svn/run/svnreplay.py", line 957, in real_main
process_svn_log_entry(log_entry, source_ancestors, commit_paths)
File ".../svn2svn/run/svnreplay.py", line 718, in process_svn_log_entry
sync_svn_props(source_url, source_rev, path_offset)
File ".../svn2svn/run/svnreplay.py", line 391, in sync_svn_props
run_svn(["propset", prop, source_props[prop], svnclient.safe_path(path_offset)])
File ".../svn2svn/shell.py", line 208, in run_svn
args=args, bulk_args=bulk_args, fail_if_stderr=fail_if_stderr, no_fail=no_fail)
File ".../svn2svn/shell.py", line 141, in run_command
return _run_raw_command(cmd, map(_transform_arg, args), fail_if_stderr, no_fail)
File ".../svn2svn/shell.py", line 114, in _run_raw_command
% (pipe.returncode, cmd_string, err, out))
ExternalCommandFailed: External program failed (return code 1): svn propset 'svn:special' '*' '.../symlink'
svn: E200009: Cannot set 'svn:special' on a directory ('.../_wc_target/.../symlink')
Furthermore, cleanup code is not designed to work with symbolic links to directories.
Traceback (most recent call last):
File ".../svn2svn/svnreplay.py", line 5, in <module>
sys.exit(main() or 0)
File ".../svn2svn/run/svnreplay.py", line 1110, in main
return real_main(args)
File ".../svn2svn/run/svnreplay.py", line 994, in real_main
full_svn_revert()
File ".../svn2svn/run/svnreplay.py", line 361, in full_svn_revert
shell.rmtree(path)
File ".../svn2svn/shell.py", line 72, in rmtree
return shutil.rmtree(path, False, _rmtree_error_handler)
File "/usr/lib64/python2.6/shutil.py", line 197, in rmtree
onerror(os.path.islink, path, sys.exc_info())
File "/usr/lib64/python2.6/shutil.py", line 195, in rmtree
raise OSError("Cannot call rmtree on a symbolic link")
OSError: Cannot call rmtree on a symbolic link
Add optional command-line arg(s) for verifying the replayed target history matches the (logical) source history.
If the user-supplied source_url
or target_url
URLs aren't properly URL-encoded, some of the comparisons in real_main() will fail because the repos_url value obtained from "svn info
" will be URL-encoded.
Should be able to use urllib.quote
for this.
When replacing dir, working copy must be updated before committing. Svn2svn does this correctly for dir -> dir replaces, but not for cases when dir is replaced by file:
Traceback (most recent call last):
File ".../svn2svn/run/svnreplay.py", line 960, in real_main
target_rev = commit_from_svn_log_entry(log_entry, commit_paths, target_revprops=target_revprops)
File ".../svn2svn/run/svnreplay.py", line 102, in commit_from_svn_log_entry
output = run_svn(args)
File ".../svn2svn/shell.py", line 208, in run_svn
args=args, bulk_args=bulk_args, fail_if_stderr=fail_if_stderr, no_fail=no_fail)
File ".../svn2svn/shell.py", line 141, in run_command
return _run_raw_command(cmd, map(_transform_arg, args), fail_if_stderr, no_fail)
File ".../svn2svn/shell.py", line 114, in _run_raw_command
% (pipe.returncode, cmd_string, err, out))
ExternalCommandFailed: External program failed (return code 1): svn commit --force-log -m 'Replace dir with file' --with-revprop 'svn2svn:source_uuid=46727182-e339-4f52-8b88-d18f8a30a573' --with-revprop 'svn2svn:source_rev=3' --with-revprop 'svn2svn:source_url=file://...' dir
svn: E155011: Commit failed (details follow):
svn: E155011: File '.../_wc_target/dir' is out of date
svn: E160028: Directory '/dir' is out of date
I'm getting a python error. my-project/trunk
does exist, but my-project/trunk/my-lib
does not.
$ ./svnreplay.py --verbose --keep-author --keep-date --keep-prop file:///var/svn/my-lib/trunk file:///var/svn/my-project/trunk/my-lib
Traceback (most recent call last):
File "./svnreplay.py", line 3, in <module>
from svn2svn.run.svnreplay import main
File "/var/svn/svn2svn/svn2svn/run/svnreplay.py", line 251
is_diff = True if sum1 <> sum2 else False
^
SyntaxError: invalid syntax
$ python --version
Python 3.4.6
An excellent piece of work and is just what I needed for dis-tangling some projects out of a massive repo!
It all seems to work well under Cywin on my Windows machine.
One small error I think I've found is that the when preserving revision numbers, the first lot of padding if skipped, as you've used source_rev instead of source_start_rev and this meant the first commit from the source always went in as rev #1 in the destination and then there was a bunch of padding to get it back in step for the next commit. Diff file for the fix is:
--- C:/Temp/SVN Migration/svn2svn-master/svn2svn-master/svn2svn/run/svnreplay.py Sat Sep 19 23:18:50 2015
+++ C:/Temp/SVN Migration/svn2svn-master/svn2svn-master/svn2svn/run/svnreplay - Copy.py Fri Jan 22 10:33:35 2016
@@ -876,8 +876,8 @@ def real_main(args):
source_start_rev = int(source_start_log['revision'])
ui.status("Starting at source revision %s.", source_start_rev, level=ui.VERBOSE)
ui.status("", level=ui.VERBOSE)
- if options.keep_revnum and source_rev > target_rev_last:
- target_rev_last = keep_revnum(source_rev, target_rev_last, wc_target_tmp)
+ if options.keep_revnum and source_start_rev > target_rev_last:
+ target_rev_last = keep_revnum(source_start_rev, target_rev_last, wc_target_tmp)
# For the initial commit to the target URL, export all the contents from
# the source URL at the start-revision
When working with Subversion repo with non-ASCII characters in paths one gets KeyError in urllib.quote:
Traceback (most recent call last):
File ".../svnreplay.py", line 5, in <module>
sys.exit(main() or 0)
File ".../svn2svn/run/svnreplay.py", line 1108, in main
return real_main(args)
File ".../svn2svn/run/svnreplay.py", line 896, in real_main
svnclient.export(join_path(source_start_url, path_offset), source_start_rev, path_offset, force=True)
File ".../svn2svn/svnclient.py", line 506, in export
args += [safe_path(svn_url, rev_number), safe_path(path)]
File ".../svn2svn/svnclient.py", line 43, in safe_path
path = urllib.quote(path, ":/+")
File ".../python2.7/urllib.py", line 1303, in quote
return ''.join(map(quoter, s))
KeyError: u'\u0430'
This can be fixed with explicit encoding unicode paths with user-provided encoding when forming safe path.
I would like to extract multiple directories, e.g. trunk/foo and trunk/bar, keeping revision numbers, into a new repo. Is this possible with svn2svn?
In svn2svn.py, rmtree should be imported from ..shell and all instances of shutil.rmtree changed to the rmtree override defined in shell.py.
In svn2svn.py, I had to change any calls to
top_paths.strip("\n").split("\n")
to
top_paths.strip("\r\n").split("\r\n").
Otherwise, the carriage returns stay in and mess up subsequent svn commands. I'm not certain about the .split() portion, but it worked for me.
In my SVN history I have a symbolic link that points to itself. Chashes when replaying SVN history.
While script is performing svn export
, this error message occurs:
svn: E000040: Can't check path '...': Too many levels of symbolic links
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.