Giter Club home page Giter Club logo

Comments (6)

andrewbaxter avatar andrewbaxter commented on May 28, 2024 1

FWIW I'm getting an exception since it seems like jsonschema gets imported when using the extension:

Traceback (most recent call last):
  File "/myproject/env/bin/scrapy", line 11, in <module>
    sys.exit(execute())
  File "/myproject/env/lib/python3.6/site-packages/scrapy/cmdline.py", line 150, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "/myproject/env/lib/python3.6/site-packages/scrapy/cmdline.py", line 90, in _run_print_help
    func(*a, **kw)
  File "/myproject/env/lib/python3.6/site-packages/scrapy/cmdline.py", line 157, in _run_command
    cmd.run(args, opts)
  File "/myproject/env/lib/python3.6/site-packages/scrapy/commands/crawl.py", line 57, in run
    self.crawler_process.crawl(spname, **opts.spargs)
  File "/myproject/env/lib/python3.6/site-packages/scrapy/crawler.py", line 170, in crawl
    crawler = self.create_crawler(crawler_or_spidercls)
  File "/myproject/env/lib/python3.6/site-packages/scrapy/crawler.py", line 198, in create_crawler
    return self._create_crawler(crawler_or_spidercls)
  File "/myproject/env/lib/python3.6/site-packages/scrapy/crawler.py", line 203, in _create_crawler
    return Crawler(spidercls, self.settings)
  File "/myproject/env/lib/python3.6/site-packages/scrapy/crawler.py", line 55, in __init__
    self.extensions = ExtensionManager.from_crawler(self)
  File "/myproject/env/lib/python3.6/site-packages/scrapy/middleware.py", line 58, in from_crawler
    return cls.from_settings(crawler.settings, crawler)
  File "/myproject/env/lib/python3.6/site-packages/scrapy/middleware.py", line 34, in from_settings
    mwcls = load_object(clspath)
  File "/myproject/env/lib/python3.6/site-packages/scrapy/utils/misc.py", line 44, in load_object
    mod = import_module(module)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/myproject/env/lib/python3.6/site-packages/spidermon/contrib/scrapy/extensions.py", line 11, in <module>
    from spidermon.python import factory
  File "/myproject/env/lib/python3.6/site-packages/spidermon/python/__init__.py", line 3, in <module>
    from . import factory
  File "/myproject/env/lib/python3.6/site-packages/spidermon/python/factory.py", line 3, in <module>
    from jsonschema import validate
ModuleNotFoundError: No module named 'jsonschema'

I'm not using the item validation tools in this project, just item count/error count monitors.

from spidermon.

raphapassini avatar raphapassini commented on May 28, 2024 1

Or we can fix the problem by optionally import the jsonschema.
IMHO I can see Spidermon being used without the validation, but doesn't make sense if an error raises if you don't install it

from spidermon.

rennerocha avatar rennerocha commented on May 28, 2024

@wRAR please see my comment in #88 . IMHO, optional requirements are optional, as the core of spidermon functionality still work without them .

from spidermon.

wRAR avatar wRAR commented on May 28, 2024

@andrewbaxter exactly, if you want to use the extension you definitely need validation and probably monitoring.

from spidermon.

muzaffaryousaf avatar muzaffaryousaf commented on May 28, 2024

@renne I think we should add this requirement as part of spidermon itself since it could be widely used in many projects.

from spidermon.

rennerocha avatar rennerocha commented on May 28, 2024

Required dependencies was updated in #100.

from spidermon.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.