A program to crawl the links of a given site URL and produce a sitemap. It only follows links which have the same domain name as the original site URL.
p-karanthaker / site-crawler Goto Github PK
View Code? Open in Web Editor NEWA single site web crawler.