Crawl all pages for a given domain and output a simple site map.
mvn package
mvn test
java -jar ./target/web-crawler-1.0.jar --starting.url=<domain url>
- Fails when the domain is slow to respond and retries are exhausted.
- There are Thread.sleep in the codebase whose durations need be made configurable.
- Test coverage could be improved
- Can introduce BDD style code coverage, may be using cucumber