papercast-dev / papercast Goto Github PK

Issue: Pylance in VSCode is unable to recognize the dynamically imported plugins in the papercast package, leading to issues like "Go to definition" not working as expected for the installed plugins.

Example:

from papercast.processors import ArxivProcessor

In the above import statement, Pylance does not recognize the ArxivProcessor imported from the papercast.processors package after papercast-arxiv plugin is installed.

Possible solution:

• Change the plugin structure to one folder with four files: subscribers.py, processors.py, publishers.py, types.py #11 . Classes of each type should belong to each file. Maybe not a bad idea anyway.
• At install time for each plugin, generate stubs for each file, and place them in the corresponding submodule folders e.g. papercast/processors/stubs/. Alternatively, figure out a way to generate stubs for only a specific class from within a module using stubgen.
• Modify the __init__.pyi file to be able to import from papercast.processors import ArxivProcessor instead of from papercast.processors.arxiv_processor import ArxivProcessor.
  • Note: if we're ok with the messier import suggested in #10 we might be able to skip this step?
• Make an init.pyi file with contents from .stubs import *
• Add a line to the papercast/processors/__init__.py:
  from papercast.processors import *

Notes:

• stubs/init.pyi is shared across plugins, maybe not ideal

Enfore a file structure for Python packages from each plugin that looks like:

plugin_dir
├── processors.py
├── publishers.py
├── subscribers.py
└── types.py

Benefits:

• Easier stub generation for #9
• Consistent init.py across plugins
• Consistency/structure might be a good thing for plugin contributors

Cons:

• Slightly harder to open files for developers working with several plugins at once

We currently import stuff from plugins with

from papercast.processors import ArxivProcessor, SemanticScholarProcessor

This would change to

from papercast.processors.arxiv import ArxivProcessor
from papercast.processors.semanticscholar import SemanticScholarProcessor

Pros:

• Possibility to simplify static code analysis?
  • Take a look at processors/init.py and see whether this would actually simplify anything. See #9
• Maybe make it clearer where each object is coming from

Cons:

• Import statement is not as clean, importing types from several locations take up multiple lines