when a disk is NOT on MusicBrainz about arver HOT 14 OPEN

Hello and welcome! I'm surprised someone other than me is using this software. How did you find it? 🙂

Regarding the issue, I considered this idea earlier in the development cycle, but in the end I decided not to implement it. The main reason is that it would be too hard to get it right. It could only work reliably with files ripped from red book CDs with zero pregap - which is probably your case.

The crucial piece of information is the pregap length, and if it's greater than zero the only way to find it would be by trial and error. That would be a complex task.

Do you have an idea of how this could work in practice? Can you describe how you imagine it?

from arver.

Hey there,

I'm new to using AccurateRip, and only learned about it a month ago. I asked at the AR forum if there was a way to query the database directly, since I use linux and don't have any of those fancy Windows programs. They said such tools probably already exist, so I looked around and found ARver. It's really just what I needed (thank you), because in addition to unripped CDs I had a bunch of rips sitting on my drive that I had not done anything with yet, and besides, I like command-line programs. It seemed as though I only had to write a thing to get the MB disk ID for me, and that was not too hard, if I know the first offset. To deal with the drive offset, I shifted the right number of samples between .wav files with a C program. So far I have verified 36 albums (more than half from old rips without the physical CD), found 9 that failed (scratches, probably), a handful that are not in MB (included in the 36), and one that is in neither MB nor AR (I think). For that last one, I am waiting to get ahold of the physical disk again so that I can see what the deal is.

For the others that are not in MB, I wrote a little script to print the data that ARver gets from MB, viz. the location of the lead-out and the offsets of all the tracks, by reading the lengths of the .wav files. I assume that the offset of the first track is 150, but can input a different number if I need to. So far, 150 seems most common, 182 is next most common, and anything else is even less common. That one disk I am still puzzled by doesn't have 150 or 182, if I wrote down the right 8-digit FreeDB ID (which I use to check). I modified disc/info.py so that if the ID passed to it has a "+" in it, it parses it as a list of offsets and skips the call to MB.

My vision for how this might work in ARver is not pretty. First, use libsndfile to get the number of samples in the .wav files, then do the division by 588. For the first offset, try 150 sectors first, 182 next. If those fail to return a result from AR, only then go to looping from 0 until it is found, with maybe a cut-off at 20,000 (enough for a hidden song). I don't know how the AR people will feel about that, though. It might be reasonable to only try 150 and 182, then give up, don't you think? That would cover at least 90% of such cases, which is already a nice improvement on coverage. And if we find another common number, we can add it later.

Another related feature could be to add an option that takes a full FreeDB ID, like
63081e08+8+150+17077+30617+68480+86187+91987+124067+145200+2080,
finds the length of the last .wav file via libsndfile, and then constructs the AR ID. This would be handy for users who kept track of the FreeDB information, for example from the "cd-discid" program or from some old ripping software.

from arver.

For that last one, I am waiting to get ahold of the physical disk again so that I can see what the deal is.

It might be an enhanced CD (two sessions with data track in the end). You can't correctly calculate AccurateRip IDs of these CDs without knowing about the existence of the data track, and knowing its length.

So far, 150 seems most common, 182 is next most common, and anything else is even less common.

Here's a table of values from my CDs:

Add 150 to "pregap" values to make them directly comparable with yours. We're in agreement here: 0 (150) is by far the most common, 32 (182) is next. Another one is 33 (183), and the remaining ones are all over the place. The big one in the end is a hidden audio track ("HTOA").

I remember seeing a chart of common pregap lengths somewhere, but I can't find it now. I think there were 4 or 5 common non-zero values.

I don't know how the AR people will feel about that, though.

I'm sure I know, without having to ask them. 😉

from arver.

I recently ran across a box set in which they all have 183 = 33, so we are in even more agreement there.

I found another purpose for ARver: verifying someone else's rip. For those cases, the pregap is usually in the log file or cue file, if one exists.

from arver.

After some consideration I realized that I want to support this feature in ARver.

I pushed guess-id branch, where I added arver-guess-id program. It calculates the disc ID based on a set of files and attempts to download checksums from AccurateRip database. It requires the user to specify the correct track 1 pregap length if it's non-zero. I don't want to automate guessing - that would require spamming the database with multiple requests.

This is just proof of concept code, and it essentially only demonstrates that ARver already has most of the building blocks necessary to implement the feature. You may even use arver-guess-id together with arver-ripinfo to verify the checksums manually. But this is definitely not the long-term solution, and I don't think a separate program is needed.

I'll think about how to integrate this with existing code. When I have the details figured out I can ask you for help, if you're still interested in helping out.

from arver.

I will try the new thing.

I'll think about how to integrate this with existing code. When I have the details figured out I can ask you for help, if you're still interested in helping out.

Yes, of course.

from arver.

I just pushed a preliminary implementation to guess-id branch. Could you help me test it?

arver will still try to read a disc by default, but now if you give it -t option it will derive the TOC from track lengths. There are also two new options for providing the lengths of pregap track, and of the data track, so it's possible to verify tracks ripped from enhanced CDs.

Example with tracks ripped from a disc with non-zero track one pregap:

$ arver ~/dance_hall_at_louse_point/*.wav
Could not read disc. Is there a CD in the drive?
Failed to get disc info, exiting.

$ arver ~/dance_hall_at_louse_point/*.wav -t
AccurateRip disc ID: 012-0011d2d4-00a95078-9b095b0c
MusicBrainz disc ID: K.v3WUQFdarZ5g0tR8s6USAoSRM-
Disc type: None (disc IDs guessed from track lengths)
...
Got 1 AccurateRip response (max confidence: 1)
...

$ arver ~/dance_hall_at_louse_point/*.wav -t --pregap-length 32
AccurateRip disc ID: 012-0011d474-00a95bd7-99095c0c
MusicBrainz disc ID: TCYiHbtahi2SbOHwpyk54_6EJr8-
Disc type: None (disc IDs guessed from track lengths)
...
Got 5 AccurateRip responses (max confidence: 104)
...

Another example with tracks ripped from an enhanced CD:

$ arver ~/liquid/*.wav -t
AccurateRip disc ID: 010-00154c2f-00af4fd4-890e120a
MusicBrainz disc ID: 9AQWqQ0eCCwktwPPrIUIYkUw2Uo-
Disc type: None (disc IDs guessed from track lengths)
...
Got 3 AccurateRip responses (max confidence: 3)
...

$ arver ~/liquid/*.wav -t --data-length 52066
AccurateRip disc ID: 010-00164419-00b9f6e2-9e11600b
MusicBrainz disc ID: 9AQWqQ0eCCwktwPPrIUIYkUw2Uo-
Disc type: None (disc IDs guessed from track lengths)
...
Got 5 AccurateRip responses (max confidence: 17)
...

from arver.

Yes, I'll give it a try. Stay tuned.

from arver.

OK, I have it installed and working. For now, just two notes:

1. In the help dialog, it might be nice to mention that the pregap is different by 150 frames from what I think it is, you know, for idiots like me.
2. If the internet crashes while arver is running, urllib3 throws an exception. This was true of 1.3.0 as well, but I didn't mention it.

Will keep testing and also try the -d feature very soon.

from arver.

(2) continued: Sometimes it gives the rip info instead of crashing.

from arver.

(1) continues: Might also say "for use with -t"

from arver.

Thank you for testing. 👍

In the help dialog, it might be nice to mention that the pregap is different by 150 frames from what I think it is

Ah, the LBA vs. LSN addressing difference. You're right, it would be useful to clarify.

If the internet crashes while arver is running, urllib3 throws an exception (...) Sometimes it gives the rip info instead of crashing.

Can you tell me what is the first exception that's raised when it happens? I thought I fixed it in 449b451, but I'm only handling requests.HTTPError and there's probably all sorts of exceptions that can be raised on network errors.

Will keep testing and also try the -d feature very soon.

It's work in progress, so -d doesn't do anything useful yet. 🙂

from arver.

Correct, -d does nothing.

Here is output from one trial:

Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/urllib3/connection.py", line 171, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/lib/python3.7/site-packages/urllib3/util/connection.py", line 56, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.7/http/client.py", line 1229, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1275, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1224, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1016, in _send_output
self.send(msg)
File "/usr/lib/python3.7/http/client.py", line 956, in send
self.connect()
File "/usr/lib/python3.7/site-packages/urllib3/connection.py", line 196, in connect
conn = self._new_conn()
File "/usr/lib/python3.7/site-packages/urllib3/connection.py", line 180, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f3c6a740cf8>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/requests/adapters.py", line 497, in send
chunked=chunked,
File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3.7/site-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='www.accuraterip.com', port=80): Max retries exceeded with url: /accuraterip/1/6/2/dBAR-013-00149261-00d2d7a0-b40b0e0d.bin (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3c6a740cf8>: Failed to establish a new connection: [Errno -2] Name or service not known'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/bin/arver", line 11, in
load_entry_point('ARver==1.3.0', 'console_scripts', 'arver')()
File "/usr/lib/python3.7/site-packages/arver/arver_main.py", line 64, in main
disc.fetch_accuraterip_data()
File "/usr/lib/python3.7/site-packages/arver/disc/info.py", line 342, in fetch_accuraterip_data
self.accuraterip_data = fetcher.fetch()
File "/usr/lib/python3.7/site-packages/arver/disc/database.py", line 289, in fetch
timeout=FETCH_TIMEOUT_SECONDS)
File "/usr/lib/python3.7/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/usr/lib/python3.7/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.7/site-packages/requests/adapters.py", line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='www.accuraterip.com', port=80): Max retries exceeded with url: /accuraterip/1/6/2/dBAR-013-00149261-00d2d7a0-b40b0e0d.bin (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3c6a740cf8>: Failed to establish a new connection: [Errno -2] Name or service not known'))

from arver.

socket.gaierror: [Errno -2] Name or service not known

Is that a DNS resolution problem? I'll have to think about more generic network exception handling...

In any case, this is not related to the -t feature, so let's put this aside for now.

from arver.