Using WARCs with standard PyWb about brozzler HOT 3 CLOSED

@thomaspreeceBBC

Just to add, if you index your WARC with cdx-indexer -p, pywb 2.0 will index HEAD requests differently, so they should not conflict with regular get requests, eg. the key should be com,example)/?__pywb_method=head

This should also be happening automatically if you're using wb-manager to add WARCs also.

from brozzler.

You're probably looking at a HEAD request. (You can verify this by inspecting the request record which follows the response in the warc.) Youtube-dl makes HEAD requests and unfortunately pywb and other wayback implementations don't have special handling for those. So what you should probably do is run warcprox with --method-filter=GET --method-filter=POST. This will prevent it from writing HEAD requests to the warc.

from brozzler.

I'm gonna close this issue. Feel free to reopen if this doesn't solve the problem.

from brozzler.