Split LangCheck into language-specific packages such as `langcheck[ja]`, `langcheck[de]`, etc about langcheck HOT 4 OPEN

hello, kenny, I may have time add a small update for this

but I have few questions

1. what should default install command pip install langcheck do, would it only install support for default language, like [en] e.g. or let it install supoprt for all language? install former way seem easier to realized.
2. if split like this

dev = [
    "yapf==0.40.1",
    "flake8",
    "isort",
    "pyright",
    "pandas-stubs",
    "pytest",
    "sphinx",
    "sphinx-book-theme",
    "myst-parser",
    "ipykernel"
]
zh = [
    'hanlp >= 2.1.0b33',
]
ja = [
    'fugashi',  # For tokenizer of metrics.ja.toxicity()
    'janome >= 0.3.1',
]

the github action file may also need a small update like, pip install -e .[dev,zh,ja], e.g.

what's your opinion ?

from langcheck.

Hi @Vela-zz, that would be fantastic!

• what should default install command pip install langcheck do, would it only install support for default language, like [en] e.g. or let it install supoprt for all language? install former way seem easier to realized.

I think we can make pip install langcheck install a default language (English) to improve the default installation process.

the github action file may also need a small update like, pip install -e .[dev,zh,ja], e.g.

Agreed, we'll need to update all GH Actions, including the new Pip Install Matrix (we might want to add a third matrix dimension for "language").

dev = [
    "yapf==0.40.1",
    "flake8",
    "isort",
    "pyright",
    "pandas-stubs",
    "pytest",
    "sphinx",
    "sphinx-book-theme",
    "myst-parser",
    "ipykernel"
]
zh = [
    'hanlp >= 2.1.0b33',
]
ja = [
    'fugashi',  # For tokenizer of metrics.ja.toxicity()
    'janome >= 0.3.1',
]

@liwii do you know which dependencies in pyproject.toml are Japanese-only?

@ischender do you know which dependencies in pyproject.toml are German-only? (if any?)

I'm happy to help you with parts of this PR, e.g. I can update GitHub Actions, if you want.

from langcheck.

Also, not sure if this is possible, but it might be nice to have pip install langcheck[all] which installs all languages without dev dependencies.

from langcheck.

That would be neat!
@kennysong , I am pretty sure the only German only part is that we download the translation model from Huggingface, but that happens when it's run the first time.

from langcheck.