When calling crawler.Crawl, getting NullReferenceException at RenderJavascript about abotx HOT 5 CLOSED

Not enough information to keep this bug open. Need full example and configuration.

from abotx.

The problem is when you pass to crawler the Impls

       var crawler = new CrawlerX(crawlConfig, new ImplementationContainer() {
                PageRequester = new PageRequester(crawlConfig, proxy)
            });

Changing this to:

            var crawler = new CrawlerX(crawlConfig);
            crawler.Impls.PageRequester = new PageRequester(crawlConfig, proxy);

Solves the problem. But this issue is still valid. Maybe ImplementationContainer should be abstract and CrawlerX could throw a meaningful exception.

from abotx.

If you pass in an ImplementaionContainer it is expecting that all properties must set. If you use the ImplementationOverride class, then you can just set the impls that you care about and the defaults will be used for all other impls. See an example here.

I understand this is confusing and will consider making the ImplementationOverride an abstract class or by just requiring the ImplementationOverride explicitly in the constructor.

from abotx.

Todo:
1: Either make ImplementationContainer an abstract class or require ImplementationOverride in the constructor.
2: Do the same for parallelCrawler
3: Update parallelCrawler docs page

from abotx.

Now Both CrawlerX and ParallelCrawlerEngine REQUIRE an ImplementationOverride instead of ImplementationContainer. This has been fixed with nuget version 1.2.126

from abotx.