rcls / crap Goto Github PK

I'm trying to migrate a cvs repository to git. The repository's root is on a pserver... :-(

Now, the server uses passwords, and I have a password - in ~/.cvspass . The file has a single line, which reads as follows:

/1 :pserver:[email protected]:2401/cvs abcdefghi

(but that's not the real password obviously.)

When I use crap-clone, like so:

./crap/crap-clone  :pserver:[email protected]:/cvs removedupes

The password crap-clone provides is just A - which is the fallback string.

Hi,

I was just realizing that there is no license file in the repository, and no declaration of license intent in the README file. This makes it hard for people to collaborate with you or to package it for use. :(

I assume that you intend for others to provide patches and would like it easily used by those who need it, anyhow.

Is there a way to build the crap-clone statically linked? Then I can build it once on my ubuntu-machine, and just distribute it. Or have a version without the libpipeline? Or a docker-image to run it in?

I get the following error

...
2002-08-02 04:33:18 RDT COMMITMissed first time round: plugins/saint/makefile.win32-msvc 1.1
cvs checkout - failed to get plugins/saint/makefile.win32-msvc 1.1
fatal: stream ends early
...

After checking out the repo with CVS, the following works

cvs up -r1.1 -p Makefile.win32-msvc

Unfortunately I cannot share the repository with you as this is company code! crap has been working for years on that repository, flawlessly so. I am at a loss at what could have happened suddenly.

This is with CVS 1.11.23 on Cygwin with crap at commit d985bbb (the merge of the pull request of mine adding keyword support).

After an incremental import, the working directory was dirty with a bunch of uncommitted changes. However, importing seemed to be correct, and I could simply issue a "git reset HEAD --hard".

On the other hand, if the current git branch was not master to start with, the working tree is clean after the import.

I apologise to highjack the issues but I haven't found another way to get in touch. I have been searching for years a tool to incrementally import a CVS repo into git that can cope with real life mess (I read the caveat in your readme about "real time"). I had given up when I found crap-clone. Works perfectly for me, on a repo with 15 years of commit history and nearly 30,000 ones on the trunk, so immense thank you!

$ make
gcc -O2 -Wall -Werror -std=gnu99 -D_GNU_SOURCE -g3 -MMD -MP -MF.deps/branch.o.d -c -o branch.o -c branch.c
In file included from branch.c:15:0:
bitset.h: In function 'bitset_init':
bitset.h:17:16: error: 'LONG_BIT' undeclared (first use in this function)
bitset.h:17:16: note: each undeclared identifier is reported only once for each function it appears in
bitset.h: In function 'bitset_set':
bitset.h:31:45: error: 'LONG_BIT' undeclared (first use in this function)
bitset.h: In function 'bitset_reset':
bitset.h:41:45: error: 'LONG_BIT' undeclared (first use in this function)
make: *** [branch.o] Error 1

LONG_BIT is not a standard define in limits.h.

I am trying to convert the repository of privoxy. crap-clone chokes on a file it regards as malformed:

$ crap-clone  :pserver:[email protected]:/cvsroot/ijbswa  current
Pserver 'anonymous'@'ijbswa.cvs.sourceforge.net':'2401' '/cvsroot/ijbswa'
Password 'A'
Logged in successfully
Valid-requests Root Valid-responses valid-requests Repository Directory Max-dotdot Static-directory Sticky Entry Kopt Checkin-time Modified Is-modified UseUnchanged Unchanged Notify Questionable Argument Argumentx Global_option Gzip-stream wrapper-sendme-rcsOptions Set Gssapi-authenticate expand-modules ci co update diff log rlog add remove update-patches gzip-file-contents status rdiff tag rtag import admin export history release watch-on watch-off watch-add watch-remove watchers editors init annotate rannotate noop version
Log (/cvsroot/ijbswa/current/doc/source/Attic/announce.sgml,v) has malformed version 2.0

It would be nice if such files could at least be ignored (automatically or by an option or a blacklist) so the import could proceed.

After an incremental import, I always get this warning message:

warning: Not updating refs/tags/TAG_NAME (new tip d2b66baaf49921267e9fcd25bff3f13dd4f6c27d does not contain 0af95a69da66bc184f096c90af2d7a845a3e3ee4)

What does this mean?

Hi,

is it possible to get some release tarball of your software?

It should be possible to specify a password without requiring ~./cvspass file. Working on that in my fork.

Hi Ralph Loader,

First of all, thank you so much for this amazing tool, it's really promising, except one thing which is stopping me from being able to proceed.

I'm trying to import a CVS module at work to git repositories. Let's say the module's name is AB+ which contains about 20 sub directories inside a directory called "project", say, project/1-1, project/1-2, ... project/1-20. I was able to import each directory 1-1 to 1-20 individually into 20 git repositories by using:

$ crap-clone :pserver:[email protected]:2401/cvsroot/software project/1-N

where N is an integer in the range [1, 20].

However, I was unable to import the module AB+ into a single git repository by:

$ crap-clone :pserver:[email protected]:2401/cvsroot/software AB+

and I always get the error:

RCS file name '/cvsroot/software/project/1-1/.donotprune,v' does not start
with prefix '/cvsroot/software/AB+/

I've tried to workaround by using 20 git repositories. But unfortunately that doesn't seem to work well because people always commit changes in different directories (1-1 to 1-20) in one CVS commit. The 20 sub directories can not be treated like 20 independent repositories. But instead, they are closely related and are really one single project.

I would really appreciate it if you would help me with this.

Thanks in advance!

York

Currently, one can't just take a CVS command and intuit the syntax for crap. It would be nice for users if this was possible.

To use a fairly good (well, good for illustration...) example, MirBSD's jupp documents their command for a CVS checkout as follows (on https://www.mirbsd.org/jupp.htm#docs ):

$ env CVS_RSH=ssh cvs -d [email protected]:/cvs co -PA jupp

It would be sensical to lots of people for this to work:

$ crap-clone :ext:[email protected]:/cvs jupp

However, this reports:

ssh: Could not resolve hostname anoncvs.mirbsd.org:: Name or service not known
Reading from CVS server failed: Connection reset by peer

It's not exactly documented well that one ought to replace the : with a / ending up with :ext:[email protected]//cvs.

However, this particular repository is a bit "special" in that even that specification won't work:

Valid-requests Root Valid-responses valid-requests Command-prep Referrer Repository Directory Relative-directory Max-dotdot Static-directory Sticky Entry Kopt Checkin-time Modified Is-modified UseUnchanged Unchanged Notify Hostname LocalDir Questionable Argument Argumentx Global_option Gzip-stream wrapper-sendme-rcsOptions Set expand-modules ci co update diff log rlog list rlist global-list-quiet ls add remove update-patches gzip-file-contents status rdiff tag rtag import admin export history release watch-on watch-off watch-add watch-remove watchers editors edit init annotate rannotate noop version suck
RCS file name '/cvs/contrib/code/jupp/Attic/.jmacsrc,v' does not start with prefix '/cvs/jupp/'

Instead, it requires:

$ crap-clone :ext:[email protected]//cvs contrib/code/jupp

This only vaguely resembles the original command, though it seems reasonably sensical once you've thought about what's going on. But, it seems that it should be possible to make it work as someone new to the tool might expect.

I have attached a zipped repo, repo.zip, created on Linux. It consists of the renaming of a file aaa.txt into AAA.txt, plus a change of the content in the process. This was done as follow

mv aaa.txt AAA.txt
cvs add AAA.txt
cvs rm aaa.txt
cvs commit -m 'aaa.txt --> AAA.txt' aaa.txt AAA.txt

As a result, the removal and the addition have exactly the same timestamp. Then I tested crap-clone on both Linux and Cygwin, with the following results:

• crap-cloning it on Linux works fine and result in

% git log --stat                                                  
commit 61c501b227c8ab45c0ef9eebb986176f0fd96fa9
Author: luc <luc>
Date:   Mon Feb 19 14:52:59 2018 +0000

    aaa.txt --> AAA.txt

 AAA.txt | 2 ++
 aaa.txt | 2 --
 2 files changed, 2 insertions(+), 2 deletions(-)

commit 4fbece18919f1c0d2bd103698746b502896c7a59
Author: luc <luc>
Date:   Mon Feb 19 14:42:33 2018 +0000

    Added lowercase filename

 aaa.txt | 2 ++
 1 file changed, 2 insertions(+)

• crap-cloning it on Cygwin results in the broken history

% git log --stat               
commit 15b4e1fb95e8ebfc16f4dc3db032feb414b7dcef (HEAD -> master)
Author: luc <luc>
Date:   Mon Feb 19 14:52:59 2018 +0000
 
    aaa.txt --> AAA.txt
 
aaa.txt | 2 --
1 file changed, 2 deletions(-)
 
commit 4fbece18919f1c0d2bd103698746b502896c7a59
Author: luc <luc>
Date:   Mon Feb 19 14:42:33 2018 +0000
 
    Added lowercase filename
 
aaa.txt | 2 ++
1 file changed, 2 insertions(+)

This is obviously caused by the fact that the filesystem is case-insensitive on Cygwin but compounded by the fact that the addition and the removal have the same timestamp.

The patch to allow relative destination paths (commit 09547d1) was (probably unintentionally) reverted in commit 5f2ee87. This smells like manual version mangement of crap-clone.c ;-). You can most easily re-apply the patch with "git cherry-pick 09547d1" on the master branch.

When running crap-clone, after the run has started, I'm getting the following error message;

opening .git/crap/version-cache.txt failed: No such file or directory

The .git subdirectory does exist; the rest is up to you, not me, to create... why aren't you creating it? :-(

Some tags which are acceptable in CVS, are unacceptable in git. See this StackOverflow answer for a list of disallowed character sequences in tag names.

crap doesn't check tag names against these restrictions, and just goes ahead and accepts them into the tag database in read_file_versions(). Now, when it later encounters such a tag in going over the commit / tagging history, something fails/crashes (perhaps cvs-fast-export? git-fast-import?) and the process is terminated.

I'm not exactly sure what the solution should be, since I'm having trouble following to code to realize exactly where the tag name is used later on.

A (poor) workaround which I've somehow managed to hack together is simply not inserting problematic tag into the tag DB within read_file_versions(); but it's not the right thing to do. What is the right thing to do?

Option 1:

• Keep the tag in the tag DB, but mark it as git-unacceptable
• Whenever we see the tag used/applied, emit a warning message, and skip its use.

Option 2:

• Keep the tag in the tag DB, but "escape" it somehow, e.g. badtag/ becomes crap_escaped_badtag_slash. (Perhaps it's a good idea to ensure that no such tag already exists in the repository)
• Whenever we see the tag used/applied, apply instead the escaped tag.

Option 3:

Support both options 1 and 2, and let a command-line switch choose between them. Default to escaping rather than deleting, I would say.

Apparently our CVS repository is corrupted. It would be nice to be able to ignore broken versions somehow.

I do not much care if some old versions are corrupted in git, the old files are just of historical interest. The important thing is to have a valid file representing the most recent version.

It will not be possible for us to recover (from backup) or attempt to repair broken RCS files.

It is possible that, in our case, the broken files are all in Attic, so a way to ignore Attic files may help.

~/crap/crap-clone --force ~/cvs_src ...
...
1998-01-09 12:04:34 PST COMMITcvs: cvs [update aborted]: invalid change text in .../Win32/Attic/sp3w32rt.wse,v
Did not get Update line: 'error  '
spdev@ubuntu:~/git_src3$ fatal: stream ends early
fast-import: dumping crash report to .git/fast_import_crash_83447

Hi,

the last time I used crap-clone, I thought it would be nice to use tab-completion for the path to a local CVS repository, and to use a relative path. This (like so many things) is made impossible by a CVS misfeature that requires local repository paths to be absolute and not end in a slash. Now I have finally found the time to write a patch. A git bundle is here:

http://volkerschatz.com/tmp/crap-clone-relpath.bundle

It contains just one commit; the patch simply removes trailing slashes and prepends the current working directory when needed. I hope you consider it useful.

$ make
gcc -O2 -Wall -Werror -std=gnu99 -D_GNU_SOURCE -g3 -MMD -MP -MF.deps/cvs_connection.o.d -c -o cvs_connection.o -c cvs_connection.c
In file included from /opt/local/lib/gcc48/gcc/x86_64-apple-darwin13/4.8.3/include/stdint.h:9:0,
from /usr/include/netdb.h:87,
from cvs_connection.c:9:
/opt/local/lib/gcc48/gcc/x86_64-apple-darwin13/4.8.3/include-fixed/stdint.h:27:32: fatal error: sys/_types/_int8_t.h: No such file or directory
#include <sys/_types/_int8_t.h>
^
compilation terminated.
make: *** [cvs_connection.o] Error 1

The code does

        size_t len = strrchr (i->version, '.') - i->version;

which is a bad idea if i->version is non-empty but does not contain a '.'.

In my case the following gdb session shows how memcpy is called with insane values because i->version contains the string "1".

#0  fill_in_versions_and_parents (file=0x6b4200, attic=false, file_tags=0x6ba520, file_tags_end=0x6ba7b0, tags=0x7fffffffb0f0) at log_parse.c:460
460         memcpy (vers, i->version, len);
(gdb) print *i
$8 = {tag = 0x6328a0, version = 0x6655b0 "1"}
(gdb) print *i->tag
$9 = {tag = 0x632880 "MT", tag_files = 0x6b8980, tag_files_end = 0x6b8a80, branch_versions = 0x6163e8 <dummy_pointer>, parents = 0x0, parents_end = 0x0, tags = 0x0, tags_end = 0x0, is_released = false, fixup = false, dummy = false, deleted = false, merge_source = false, rank = 0, parent = 0x0, changeset = {time = 862914216, type = ct_tag, unready_count = 0, ready_index = 18446744073709551615, mark = 0, versions = 0x0, versions_end = 0x0, children = 0x0, children_end = 0x0, merge = 0x0, merge_end = 0x0}, last = 0x0, fixups = 0x0, fixups_end = 0x0, fixups_curr = 0x0}
(gdb) print len
$10 = 18446744073702845008

(Unfortunately I have no idea how crap or CVS works, so I do not know what a suitable fix would be here.)

I tried to compile the program:

$ make
gcc -O2 -std=c99 -Wall -Wextra -Werror -D_GNU_SOURCE -g3 -MMD -MP -MF.deps/branch.o.d   -c -o branch.o -c branch.c
gcc -O2 -std=c99 -Wall -Wextra -Werror -D_GNU_SOURCE -g3 -MMD -MP -MF.deps/changeset.o.d   -c -o changeset.o -c changeset.c
gcc -O2 -std=c99 -Wall -Wextra -Werror -D_GNU_SOURCE -g3 -MMD -MP -MF.deps/cvs_connection.o.d   -c -o cvs_connection.o -c cvs_connection.c
cvs_connection.c:10:10: fatal error: pipeline.h: No such file or directory
 #include <pipeline.h>
          ^~~~~~~~~~~~
compilation terminated.
Makefile:25: recipe for target 'cvs_connection.o' failed
make: *** [cvs_connection.o] Error 1

Can you tell me which library I have to install?
Thanks!

I am importing from a local directory. After import I do not see any files. How do I proceed?

Here is what I did:

$ crap-clone /mnt/d/cvs/cvs 2203601
..
...
2011-09-05 15:38:44 CEST COMMIT
2011-09-05 15:44:37 CEST COMMIT
2011-09-05 15:44:37 CEST TAG V702-01-003
Emitted 159 commits (= total 159).
Exact     2 +     2 =     4 branches + tags.
Fixup     0 +     3 =     3 branches + tags.
Download 418 cvs versions in 147 transactions.
String cache: 593 items, 448/1024 buckets used, mean search 1.32378
fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects:       5000
Total objects:          803 (        74 duplicates                  )
      blobs  :          345 (        73 duplicates        202 deltas of        343 attempts)
      trees  :          307 (         1 duplicates        220 deltas of        299 attempts)
      commits:          151 (         0 duplicates          0 deltas of          0 attempts)
      tags   :            0 (         0 duplicates          0 deltas of          0 attempts)
Total branches:           7 (         5 loads     )
      marks:           1024 (       569 unique    )
      atoms:             79
Memory total:          2493 KiB
       pools:          2141 KiB
     objects:           351 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 35184372088832
pack_report: pack_used_ctr            =         30
pack_report: pack_mmap_calls          =          9
pack_report: pack_open_windows        =          1 /          1
pack_report: pack_mapped              =     612777 /     612777
---------------------------------------------------------------------

$ git gc --aggressive
Enumerating objects: 803, done.
Counting objects: 100% (803/803), done.
Delta compression using up to 12 threads
Compressing objects: 100% (792/792), done.
Writing objects: 100% (803/803), done.
Total 803 (delta 496), reused 259 (delta 0), pack-reused 0

$ ls | wc
      0       0       0

$ ls -la .git
total 60
drwxr-xr-x 9 me me 4096 Aug 25 15:06 .
drwxr-xr-x 3 me me 4096 Aug 25 15:04 ..
-rw-r--r-- 1 me me  176 Aug 25 15:06 COMMIT_EDITMSG
-rw-r--r-- 1 me me   21 Aug 25 15:04 HEAD
drwxr-xr-x 2 me me 4096 Aug 25 15:04 branches
-rw-r--r-- 1 me me   92 Aug 25 15:04 config
drwxr-xr-x 2 me me 4096 Aug 25 15:05 crap
-rw-r--r-- 1 me me   73 Aug 25 15:04 description
drwxr-xr-x 2 me me 4096 Aug 25 15:04 hooks
-rw-r--r-- 1 me me   65 Aug 25 15:05 index
drwxr-xr-x 2 me me 4096 Aug 25 15:05 info
drwxr-xr-x 3 me me 4096 Aug 25 15:05 logs
drwxr-xr-x 5 me me 4096 Aug 25 15:05 objects
-rw-r--r-- 1 me me  484 Aug 25 15:05 packed-refs
drwxr-xr-x 4 me me 4096 Aug 25 15:04 refs

README file claims that any C99 system should work, but I actually have an older server I want to compile this on that only has C99 and C99 does not work. Even the Makefile specifies C11.