...because who says I can't write yet another link checker?
This thing was created from Hello Rust Episode
10. It's a link checker that treats Github links
specially by using a GITHUB_TOKEN
to avoid getting blocked by the rate
limiter.
TODO: Add screenshots here
The existing link checkers were not flexible enough for my use-case. lychee runs all requests fully asynchronously and has a low memory/CPU footprint.
lychee can...
- handle links inside Markdown, HTML, and other documents
- handle chunked encodings
- handle gzip compression
- fake user agents (required for some firewalls)
- skip non-links like anchors or relative URLs
- exclude some websites with regular expressions
- handle a configurable number of redirects
- disguise as a different user agent (like curl)
- optionally ignore SSL certificate errors (
--insecure
) - check multiple files at once (supports globbing)
- support checking links from any website URL
- limit scheme (e.g. only check HTTPS links with "https")
- accept custom headers (e.g. for cases like rust-lang/crates.io#788)
- show final summary/statistics
- optionally use
HEAD
requests instead ofGET
- show colored output
- filter based on status codes (tcort/markdown-link-check#94)
(e.g.
--accept 200,204
) - accept a request timeout (
--timeout
) in seconds. Default is 20s. Set to 0 for no timeout. - check e-mail links using check-if-mail-exists
SOON:
- automatically retry and backoff
- check relative (
base-url
to set project root) - show the progress interactively with progress bar and in-flight requests (
--progress
) - usable as a library (raviqqe/liche#13)
- exclude private domains (https://github.com/appscodelabs/liche/blob/a5102b0bf90203b467a4f3b4597d22cd83d94f99/url_checker.go)
- recursion
- extended statistics: request latency
- use colored output (https://crates.io/crates/colored)
cargo install lychee
Set an environment variable with your token like so GITHUB_TOKEN=xxxx
.
Run it inside a repository with a README.md
or specify a different Markdown
file with
lychee <yourfile>
Collecting other link checkers here to crush them in comparison. :P
- https://github.com/dkhamsing/awesome_bot
- https://github.com/tcort/markdown-link-check
- https://github.com/raviqqe/liche
- https://github.com/raviqqe/muffet
...to my Github sponsors and Patreon sponsors for supporting these projects. If you want to help out as well, go here.