Redirect from bare domain to www subdomain throws Self Redirect error. about pywb HOT 4 CLOSED

The same kind of thing happens if you archive http://news.ycombinator.com/ , which 301 redirects to https://news.ycombinator.com/ .

from pywb.

Yeah, the self-redirect error is a common occurence, given the current canonicalization scheme, it may make sense to adjust the canonicalization scheme in surt to address this, although self-redirects are still possible for other reasons (eg. cookies).

The way this works is that pywb catches the CaptureException, and tries the next capture, as the two captures should be close together in the cdx. For example, it is common to have something like this in the cdx:

com,metafilter)/ 20140126200927 http://www.metafilter.com/ text/html 302 ...
com,metafilter)/ 20140126200928 http://www.metafilter.com/ text/html 200 ...

If a request comes in for 20140126200927, it is skipped (by throwing the CaptureException) and the next one is tried instead.

Hopefully your cdx has two entries for com,metafilter)/ and will work in this way too?

(You can check out this latest branch for an example of how this is working).

from pywb.

Oh, that makes sense. Yeah, I'll have to update my super-naive cdx server to return all the matches. Thanks!

from pywb.

The current version now has a local cdx server that reads from source .cdx files, if that's of any help.
The idea is that you can configure the cdx source dir, and warc source dir in yaml, and have everything be automated.
Although, things are still in flux, hopefully we'll have a stable 0.1 that you can update to soon..

The cdx server also sorts results by distance from requested date.. more documentation coming too soon!

from pywb.