Starting a branch to do some final work on notebooks ahead of v0.1.0 release

• Integrating in the CEC2 commandline usage notebook
• Adding a README notebook index with readymade colab links

Is your feature request related to a problem? Please describe.

Before #42 can be implemented we need to run pylint against the existing code base and resolve all issues.

Describe the solution you'd like

Changes should be made on a separate branch and commited with --ignore-revs to avoid mis-attributing the "blame" (see example in Introducing Black to your project - Black 22.6.0 documentation).

Further a custom .pylintrc file will need introducing that configures pylint and the rules that are to be included/excluded (e.g. we may wish to ignore may invalid-name errors).

The EarModel in evaluator/haspi/eb computes the center frequency using shift = 0.02

However, the function CenterFreq in evaluator/haspi/eb forces the parameter to be None to keep consistency with Matlab

What is the point of setting shift in center freq if it is never used?
Setting the value in the Ear Model gives the impression that a shift is applied, won't that confuse people while reading the code?

What would be the best way to clean this part of the code?
1- Changing shift to None in EarModel?
2- Removing shift parameter from CenterFreq?
3- Removing the shift=None statement in CenterFreq?

Is your feature request related to a problem? Please describe.

Making pyclarity available on PyPI would make installation easier, particularly as development progresses.

Describe the solution you'd like

Automated publishing of commits tagged with versions to PyPI should occur. This is possible with pypi-publish GitHub Action.

Describe alternatives you've considered

Manual releases are also possible, but should not be needed if the process can be automated.

Additional context

N/A.

When I train the e009_sheffield model, both train loss and val loss (SNRLoss) will converge on 0.0043, which means all outputs for any inputs will be 0. When checking the train dataset, I find that the model using '_target_anechoic.wav' as ground truth, but the '_target_anechoic.wav' has about 20ms latency compared with both '_mix_CH1.wav' and '_target_CH1.wav'. In this way, the model is expected to predict the future information, which is impossible. If I change the ground truth from '_target_anechoic.wav' to '_target_CH1.wav', the problem is solved and the model can converge normally. Could you give me some advice on why the appoximate 20ms latency is exising between 'target_CH1.wav' and '_target_anechoic.wav' and how can I solve it? Thanks!

Migrate HASQI index from Matlab

Hi,

Thanks for your fantastic works!

I'm trying to use the head rotation signal, but I find that the length of the audio files (e.g. mixed_CH1.wav, target_CH1.wav) and head rotation file (hr.wav) is different. The head rotation signal is 255 points shorter than audio files. For example, S06001 in dev dataset, the length of S06001_hr.wav is 287735 and the length of S06001_mix_CH0.wav is 287990.

My question is how should I deal with the misalignment, should I neglect 255 points in the beginning or the end of the audio files？

Best regards,

Chengwei Ouyang

PR test failing due to py is no longer being pulled in and installed as a required dependency of pytest-regtest. (related to this issue in pytest-regtest https://gitlab.com/uweschmitt/pytest-regtest/-/issues/19)

PRs #50 / #57 /#60 / #61 / #62 addressed the need for regression tests (see #48).

Now that these are in place it is possible to start introducing finer grained unit-tests across the codebase. This will provide immediate benefits...

• Ability to identify errors with higher resolution.
• Improve the range and breadth of conditions under which tests are made.
• Facilitate refactoring of code going forward.

This issue serves as an "Epic" for recording progress of tests written across modules/sub-modules Individual issue and pull-requests can be added against each module/sub-module for reference.

• #81
  • data/demo_data.py
  • data/HOA_tools_cec2.py
  • data/scene_builder_cec1.py
  • data/scene_render_cec1.py
  • data/scene_render_cec2.py
  • data/utils.py #88
• #82
  • dataset/cec1_dataset.py
• #83
  • engine/losses.py
  • engone/system.py
• #270
  • recipes/cad1/task1 (partially complete)
  • recipes/cad1/task2 (partially complete)
  • recipes/cec1/baseline
  • recipes/cec1/data_preparation/prepare_cec1_data.py
  • recipes/cec1/e009_sheffield
  • recipes/cec2/baseline
  • recipes/cec2/data_preparation
  • recipes/cpc1/baseline
  • recipes/cpc1/e029_sheffield
  • recipes/cpc1/e032_sheffield
  • recipes/cpc1/test_listener_responses
  • recipes/cpc2 (partially complete)
  • recipes/icassp_2023
• #84
  • enhancer/compressor.py
  • enhancer/nalr.py
  • enhancer/dnn
    • enhancer/dnn/mc_conv_tasnet.py
  • enhancer/dsp
    • enhancer/dsp/filter.py
  • enhancer/gha
    • enhancer/gha/audiogram.py
    • enhancer/gha/gainrule_camfit.py
    • enhancer/gha/gha_interface.py
    • enhancer/gha/gha_utils.py
• #85
  • evaluator/haspi
    • evaluator/haspi/eb.py
    • evaluator/haspi/ebm.py
    • evaluator/haspi/haspi.py
    • evaluator/haspi/ip.py
  • evaluator/hasqi
    • evaluator/hasqi/hasqi.py
  • evaluator/haaqi
    • evaluator/haaqi/haaqi.py
  • evaluator/mbstoi
    • evaluator/mbstoi/mbstoi.py
    • evaluator/mbstoi/mbstoi_utils.py
  • evaluator/msbg
    • evaluator/msbg/audiogram.py
    • evaluator/msbg/cochlea.py
    • evaluator/msbg/masbg.py
    • evaluator/msbg/masbg_utils.py
    • evaluator/msbg/smearing.py
• #86
  • predictor/torch_msbg.py
  • predictor/torch_stoi.py

Currently the nomenclature of class, function and variables is a bit of a mixed bag.

It would improve legibility if they were all PEP8 compliant and had meaningful names (although there are some cases where exceptions may be considered reasonable such as implementations of algorithms).

To which end this issue should address changing as many class, function and variable names as possible to be PEP8 compliant and disable checks for those that are considered not worthy of bringing into line.

Currently the existing .pylintrc excludes these checks from being made because C0103: invalid-name is disabled. Excluding this temporarily and running pylint against the clarity directory shows the invalid-name errors detailed in this gist (including it in the body of this issue exceeded the character limit!).

EDIT : Many changes can be made simply and consistently across the code base by virtue of using an IDE that supports Language Server Protocol which supports the "renane" directive which will Rename: project-wide rename of a symbol..

Personally I use Emacs and lsp-mode and so can M-x lsp-rename when the cursor is on a variable/function/class and rename variables.

Hi, thanks so much for making this, I'd like to know how to run inference on a single wav file using this but I'm running into trouble with DictConfig on Colab taking inspiration from the infer.py file. This is the function I wrote:

import copy
import json
import logging
import os

import hydra
import speechbrain as sb
import torch
from hyperpyyaml import load_hyperpyyaml
from omegaconf import DictConfig
from speechbrain.utils.distributed import run_on_main
from tqdm import tqdm
from transformer_cpc1_ensemble_decoder import S2STransformerBeamSearch

logger = logging.getLogger(__name__)
def init_asr(asr_config):
    hparams_file, run_opts, overrides = sb.parse_arguments([asr_config])
    with open(hparams_file) as fin:
        hparams = load_hyperpyyaml(fin, overrides)

    tokenizer = hparams["tokenizer"]
    bos_index = hparams["bos_index"]

    # We download the pretrained LM from HuggingFace (or elsewhere depending on
    # the path given in the YAML file). The tokenizer is loaded at the same time.
    run_on_main(hparams["pretrainer"].collect_files)
    hparams["pretrainer"].load_collected(device=run_opts["device"])

    asr_brain = ASR(
        modules=hparams["modules"],
        opt_class=hparams["Adam"],
        hparams=hparams,
        run_opts=run_opts,
        checkpointer=hparams["checkpointer"],
    )
    asr_brain.init_evaluation()

    return asr_brain, tokenizer, bos_index

def compute_uncertainty(left_proc_path, asr_model, bos_index, tokenizer):
    wav_len = torch.tensor([1], dtype=torch.float32)
    tokens_bos = torch.LongTensor([bos_index]).view(1, -1)

    right_proc_path = left_proc_path.replace("left", "right")
    left_proc_wav = sb.dataio.dataio.read_audio(left_proc_path).view(1, -1)
    right_proc_wav = sb.dataio.dataio.read_audio(right_proc_path).view(1, -1)

    left_uncertainty = asr_model.compute_uncertainty(left_proc_wav, wav_len, tokens_bos)
    right_uncertainty = asr_model.compute_uncertainty(
        right_proc_wav, wav_len, tokens_bos
    )
    conf = max(
        left_uncertainty[0]["confidence"].detach().cpu().numpy(),
        right_uncertainty[0]["confidence"].detach().cpu().numpy(),
    )
    neg_ent = -min(
        left_uncertainty[0]["entropy"].detach().cpu().numpy(),
        right_uncertainty[0]["entropy"].detach().cpu().numpy(),
    )
    return conf, neg_ent






# --- AND NOW MY NOT-WORKING CODE ---- 


@hydra.main(config_path=".", config_name="config")
def infer_single(wav_path):

    cfg = DictConfig
    if cfg.cpc1_track == "open":
        track = "_indep"
    elif cfg.cpc1_track == "closed":
        track = ""
    else:
        logger.error("cpc1_track has to be closed or open")
    print(dir(cfg))
    asr_model, tokenizer, bos_index = init_asr(cfg.asr_config)
    wav_len = torch.tensor([1], dtype=torch.float32)
    tokens_bos = torch.LongTensor([bos_index]).view(1, -1)
    wav = sb.dataio.dataio.read_audio(wav_path).view(1, -1)
    uncertainty = asr_model.compute_uncertainty(wav, wav_len, tokens_bos)
    return uncertainty

But running in colab:

wav_path = '/content/clarity/recipes/cpc1/e029_sheffield/test.wav'
uncertainty = infer_single(wav_path)

It's giving me this error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-29-43c026e4011b>](https://localhost:8080/#) in <module>
      1 wav_path = '/content/clarity/recipes/cpc1/e029_sheffield/test.wav'
----> 2 uncertainty = infer_single(wav_path)

1 frames
[/usr/local/lib/python3.7/dist-packages/hydra/main.py](https://localhost:8080/#) in decorated_main(cfg_passthrough)
     77         def decorated_main(cfg_passthrough: Optional[DictConfig] = None) -> Any:
     78             if cfg_passthrough is not None:
---> 79                 return task_function(cfg_passthrough)
     80             else:
     81                 args_parser = get_args_parser()

[<ipython-input-28-f4867e54a086>](https://localhost:8080/#) in infer_single(wav_path)
    183 
    184     cfg = DictConfig
--> 185     if cfg.cpc1_track == "open":
    186         track = "_indep"
    187     elif cfg.cpc1_track == "closed":

AttributeError: type object 'DictConfig' has no attribute 'cpc1_track'

I've tried researching docs for DictConfig from Omegaconf but I'm very confused what it actually does and am having a really hard time writing this function. Would love any help whatsoever!

Hello, I am a researcher on speech enhancement and I am new to the use of objective intelligibility measures in the area of hearing impairment. I was able to figure out how to use MBSTOI, but HASPI has multiple parameters that I am not sure how to specify.

What are the standard specifications used for HASPI? Specifically clarity.evaluator.haspi.haspi_v2.
https://github.com/claritychallenge/clarity/blob/main/clarity/evaluator/haspi/haspi.py

Here is my attempt making an example:

! pip3 install https://github.com/claritychallenge/clarity/archive/refs/heads/main.zip --quiet
from clarity.evaluator import haspi, mbstoi

import os

root_url  = "https://github.com/microsoft/DNS-Challenge/raw/interspeech2020/master/datasets/test_set/synthetic/no_reverb"
clean_url = root_url + "/clean/clean_fileid_0.wav"
noisy_url = root_url + "/noisy/clnsp126_3Wjw0nadnM4_snr15_tl-22_fileid_0.wav"

if "clean.wav" not in os.listdir() or "noisy.wav" not in os.listdir():
    os.system("wget -O clean.wav " + clean_url)
    os.system("wget -O noisy.wav " + noisy_url)

import librosa

clean = librosa.load("clean.wav", sr=16000)[0]
noisy = librosa.load("noisy.wav", sr=16000)[0]

Y1 = mbstoi.mbstoi(clean, clean, noisy, noisy, fs_signal=16000)
Y2, _ = haspi.haspi_v2(clean, 16000, noisy, 16000, [250, 500, 1000, 2000, 4000, 6000])

print("MBSTOI ", Y1, "\n", "HASPI  ", Y2, sep="")

However, the observed HASPI value is really small. Is this correct usage? What are the standard ways to specify the evaluation?

Matlab version of NALR has 3 parameters, HL, nfir, fs

function [nalr,delay]=eb_NALR(HL,nfir,fsamp)

In Clarity, NALR is implemented as a Class in enhancer.nalr.

class NALR:
    def __init__(self, nfir, fs):
        [...]
    def hl_interp(self, hl, cfs):
        [...]
    def build(self, HL, cfs):
        HL = self.hl_interp(np.array(HL), np.array(cfs))
        [...]

For the cases when cfs is not available and to match MATLAB's use of HAAQI and HASQI, it is needed to make cfs=None and add the if conditional

def build(self, HL, cfs=None):
    if not cfs is None:
         HL = self.hl_interp(np.array(HL), np.array(cfs))
    [...]

Migrate HAAQI from Matlab

Hi everyone,

Let me thank the organizers for this fantastic and interesting challenge!

When I check the scripts for calculating HASPI, I find some randomness in eb and ebm. As a result, the score for the same signals slightly fluctuated in some runs. Although the fluctuation is very small, it will be good to fix the seed.

Probably, I misunderstand something. Please check whether the fluctuation occurs in your environment. Thank you!

Develop unit tests for the data module (see #80 for overview).

Specific modules within predictor are...

• predictor/torch_msbg.py
• predictor/torch_stoi.py

Hi, Thanks so much for organizing the challenge, and providing detailed docuement.
I am very interested on it and have a few questions below:

1). There is 5-ms latency limitation for the enhancement model. Will the frame length be counted in latency? I read some papers in CEC1. One team (paper) used a 32-ms window. I am wondering if this long window size (beyond 5ms) is allowed in CEC2?

2). Regarding on the use of external data. I plan to use external noisy data to train a more robust denoising module. If I do this, seems I have to submit two systems, with using and without using external data. If I plan to submit entries for HASPI and listening tests, respectively, does this mean I need to submit four entries (2 tests X 2 data uses) in total?

3). Just one thing to confirm: Was ch0 (eardrum) signal also included in the release of CEC1 dataset?
I found ch0 was included in CEC1 data (written in document), but seems many submissions only used 6 channels (ch1~3 pairs), and no team used ch0. So just curious on it.

Again, thank you so much for your work and time!

There are a number of deprecation warnings that have been captured by the test code that should be dealt with to avoid possible problems later.

The license applied on GitHub (Creative Commons Attribution Share Alike 4.0 International) is out of sync with that specified in the Python configuration file setup.py which lists MIT.

The later is a more open and permissive license that we will apply to the project.

Update LICENSE to MIT.

The repository requires a code of conduct for contributors etc.

Add a GitHub action that runs the pre-commit linting hooks when running pull requests into main.

As we move to tagged versions and releases to PyPI we need to automate the flow of versioning the package to avoid the work of updating setup.cfg and clarity/__init__.py.

The versioneer package appears to offer a solution to this as it automates aligning the version with GitHub tags/releases.

After the initial 0.1.0 release has been made manually the process should shift to using versioneer to handle versions.

The last Clarity Challenge included links to two Jupyter notebooks hosted on Google Colab (see Tutorials).

Including these in the code base here so that they and other examples/tutorials are version controlled too is desirable.

Develop unit tests for the data module (see #80 for overview).

Specific modules within data are...

• data/demo_data.py
• data/HOA_tools_cec2.py
• data/scene_builder_cec1.py
• data/scene_render_cec1.py
• data/scene_render_cec2.py
• data/utils.py #88

Develop unit tests for the data module (see #80 for overview).

Specific modules within engine are...

• engine/losses.py
• engone/system.py

Hi everyone,

first let me thank you for organizing this great challenge ! I am excited to participate.

I am looking at the data right now and in particular I am checking the delay (in terms of samples) between the left and right channels at each mic. Now it is not an easy task with Audacity to do this and I may be wrong.
But it seems to me that the delay for e.g S06001 in dev set is way off from what it should be from the metadata.
I count around 19-20 samples of delay when looking at S06001_target_anechoic_CH1.wav:

According to the metadata the head rotation should be:

"listener": {
      "rotation": [
        {
          "sample": 90952.7142,
          "angle": -125.8235
        },
        {
          "sample": 100246.7142,
          "angle": -112.0054
        }

And so I would expect the delay to be much less. Probably I am doing something wrong but just in case, maybe it could be useful if you can double check.

Also one thing I expected is that regarding the anechoic response the two channels should be equal except for a small delay and a gain factor (including inverted polarity). But this is not the case. Is it because the scenes are generated via higher order ambisonics RIRs and then the actual array is approximated from such HOA rirs ?
EDIT: I think I've figured out, it should be the head related impulse response.
This makes difficult to check the delay manually on audacity. So it is very possible I am over-estimating it.

When I run the enhance.py and evaluate.py in CEC1 baseline folder, I get different results comparing with the sii.csv in the repo. Do you have any advice on this problem? Thanks for your help!

Numba code is generating a warning about use of a deprecated type

Encountered the use of a type that is scheduled for deprecation: type 'reflected list'

This needs fixing to prevent errors arising when newer versions of numba are released.

Information on how to fix can be found here.

Develop unit tests for the data module (see #80 for overview).

Specific modules within dataset are...

• dataset/cec1_dataset.py

Describe the bug
pytest-cov coverage report only considering source files that are import in existing tests

To Reproduce
run pytest --cov=clarity

Expected behavior
Report to list many source files with 0% coverage given that we have a lot of untested code.

Fixable with simple change to coverage to coverage config in setup.cfg

The regression test fails when running the test tests/regression/test_predictors.py::test_torch_msbg_stoi
having a very small difference that may result from to randomness

=================================== FAILURES ===================================
_____________________________ test_torch_msbg_stoi _____________________________

regression test output differences for tests/regression/test_predictors.py::test_torch_msbg_stoi:

>   --- current
>   +++ tobe
>   @@ -1,2 +1,2 @@
>   -Torch MSBG STOILoss -0.46198, ESTOILoss -0.32999
>   +Torch MSBG STOILoss -0.46198, ESTOILoss -0.33000

This results in PR failing some tests.
The problem can be solved by reruning the test manually one or 2 times.

I notice that the amplification model (AudiometricFIR) doesn't take audiogram as inputs, so it seems that I have to train an individual amplification model for each listener, which is weird for me. Could you give me some suggestions on how to generalize the amplification model or I have to train 100 individual models for 100 listeners in CEC1 challenge? Thank you!

Having Clarity available on PyPI will be useful and ease installation for participants.

This requires the following actions...

• Establishing a generic email address for registering on PyPI with and can be done at the University of Sheffield.
• Registering on PyPI and the the test PyPI
• Building package for release using Twine (see Packaging projects).
• Tagging a release on GitHub and testing on PyPi Test server before releasing to PyPI proper.
• Establishing a GitHub Action for automating release when tags are applied (see example here although its snippets that fit into a larger configuration file).

Current regression test only covers CEC2 baseline, i.e. signal mixing, NALR and haspi. Need a more thorough set of tests covering CEC1, CPC1 and CEC2 code so that later refactoring can be performed securely.

Describe the bug

Pinned version of matplotlib in dependencies messes up the colab environment.

No good reason to be pinning the version.

Thanks for your fantastic works, and I have a question about the dataset. In my understanding, the purpose of this challenge is to get target_anechoic audio by mixed_CH1, mixed_CH2, mixed_CH3, which is the signal received by the front, mid, and rear microphones. My question is that what is the purpose of mixed_CH0 and target_CH0, which is close to eardrum and have a large difference with other channels.

As the Cadenza challenge is up-coming it would be appropriate to include details on the front page as well as the logo...

A first draft might be...

Machine learning challenges for hearing aid processing.

We are organising a series of machine learning challenges to enhance hearing-aid signal processing and to better predict how people perceive speech-in-noise (Clarity) and speech-in-music (Cadenza). For further details of the Clarity Project visit the Clarity project website, and for details of our latest Clarity challenges visit our challenge documentation site. You can contact the Clarity Team by email at [email protected]. For further details of the Cadenza Project visit the Cadenza project website, and to find out about the latest Cadenza challenges join the Cadenza Challenge Group.

In this repository, you will find code to support all Clarity and Cadenza Challenges, including baselines, toolkits, and systems from participants. We encourage you to make your system/model open source and contribute to this repository.

The 2nd Clarity Enhancement Challenge (CEC2) has launched! Take part🔥🔥🔥

Installation

PyPI

Clarity is available on the Python Package Index (PyPI) to install create and/or
activate a virtual environment and then use pip to install.

conda create --name clarity python=3.8
conda activate clarity

pip install pyclarity

GitHub Cloning

# First clone the repo
git clone https://github.com/claritychallenge/clarity.git
cd clarity

# Second create & activate environment with conda, see https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html
conda create --name clarity python=3.8
conda activate clarity

# Last install with pip
pip install -e .

GitHub pip install

Alternatively pip allows you to install packages from GitHub sources directly. The following will install the current
main branch.

pip install -e git+https://github.com/claritychallenge/clarity.git@main

Challenges

Current challenge(s)

• The 2nd Clarity Enhancement Challenge (CEC2)

Upcoming challenges

• The first Cadenza Challenge will launch Spring 2023. Join the Cadenza Challenge Group

Previous challenges

• The 1st Clarity Prediction Challenge (CPC1)
• The 1st Clarity Enhancement Challenge (CEC1)

Available tools

We provide also a number of tools in this repository:

• Hearing loss simulation
  • Cambridge MSBG hearing loss simulator: descriptions can be found in the CEC1 description; an usage example can be found in the CEC1 baseline evaluation script evaluate.py.
• Objective intelligibility measurement
  • Modified binaural STOI (MBSTOI): a python implementation of MBSTOI. It is jointly used with the MSBG hearing loss model in the CEC1 baseline. The official matlab implementation can be found here: http://ah-andersen.net/code/
  • Hearing-aid speech perception index (HASPI): a python implementation of HASPI Version 2, and the better-ear HASPI for binaural speech signals. For official matlab implementation, request here: https://www.colorado.edu/lab/hearlab/resources
• Hearing aid enhancement
  • Cambridge hearing aid fitting (CAMFIT): a python implementation of CAMFIT, translated from the HörTech Open Master Hearing Aid (OpenMHA); the CAMFIT is used together with OpenMHA enhancement as the CEC1 baseline, see enhance.py.
  • NAL-R hearing aid fitting: a python implementation of NAL-R prescription fitting. It is used as the CEC2 baseline, see enhance.py.

In addition, differentiable approximation to some tools are provided:

• Differentiable MSBG hearing loss model. See also the BUT implementation: https://github.com/BUTSpeechFIT/torch_msbg_mbstoi
• Differentiable HASPI (coming)

Open-source systems

• CPC1:
  • Exploiting Hidden Representations from a DNN-based Speech Recogniser for Speech Intelligibility Prediction in Hearing-impaired Listeners
  • Unsupervised Uncertainty Measures of Automatic Speech Recognition for Non-intrusive Speech Intelligibility Prediction
• CEC1:
  • A Two-Stage End-to-End System for Speech-in-Noise Hearing Aid Processing

Currently the Clarity Challenge focuses on the challenge itself.

It is desirable to have the code better documented including the following...

• Installation instructions - including how to install from PyPI (when #13 is completed) and latest GitHub branch.
• API documentation - code is well documented with docstrings and these can be used to auto-generate API documentation using Sphinx.
• Include links to Tutorials - #15 will pull the Jupyter Notebooks in to the repository but they are also hosted/deployed on Google CoLab and a section should be included linking out to these.
• Existing Software Resources - A section detailing software and research tools for hearing aid research.

The documentation should use the GitHub Action - Sphinx Docs to GitHub Pages to build and deploy the documentation automatically.

Please add any other areas of documentation to this issue.

Notebooks have problems in revision control as they contain a lot of non-content data, i.e. outputs and metadata that change each time the notebook is run. This can be fixed using pre-commit hooks.

• Can use nb-ensure-clean https://gitlab.com/iamlikeme/nbhooks to remove output cells plus metadata such as execution counts and times
• Can also add notebook pre-commit hooks for black and flake8 via https://github.com/nbQA-dev/nbQA.

Typehints are useful to use in Python, despite it being a dynamically typed language.

Much of the code has typehints in place for parameters and some for the return values but it is currently incomplete. This should be remedied and the code linted using MyPy which is a static type checker.

Currently on the most complete branch/commit ( #7c0a505 ) there are 48 errors across 22 files (out of 42 source files), although most pertain to the imported modules and so can be handled as described in Running mypy and managing imports...

 ❱ mypy clarity                                                                                                  
clarity/data/demo_data.py:4: error: Skipping analyzing "gdown": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/msbg/smearing.py:156: error: Incompatible types in assignment (expression has type "None", variable has type "ndarray[Any, Any]")
clarity/evaluator/msbg/smearing.py:157: error: Incompatible types in assignment (expression has type "None", variable has type "ndarray[Any, Any]")
clarity/evaluator/msbg/smearing.py:158: error: Incompatible types in assignment (expression has type "None", variable has type "ndarray[Any, Any]")
clarity/evaluator/msbg/smearing.py:159: error: Incompatible types in assignment (expression has type "None", variable has type "ndarray[Any, Any]")
clarity/evaluator/msbg/msbg_utils.py:7: error: Skipping analyzing "scipy": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/msbg/msbg_utils.py:8: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/msbg/msbg_utils.py:9: error: Skipping analyzing "soundfile": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/mbstoi/mbstoi_utils.py:4: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/haspi/ebm.py:4: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/haspi/eb.py:2: error: Skipping analyzing "numba": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/haspi/eb.py:3: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/enhancer/nalr.py:4: error: Skipping analyzing "scipy": module is installed, but missing library stubs or py.typed marker
clarity/enhancer/nalr.py:5: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/enhancer/gha/gainrule_camfit.py:5: error: Skipping analyzing "scipy.interpolate": module is installed, but missing library stubs or py.typed marker
clarity/data/utils.py:4: error: Skipping analyzing "scipy": module is installed, but missing library stubs or py.typed marker
clarity/data/utils.py:5: error: Skipping analyzing "scipy.io": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_builder_cec2.py:11: error: Skipping analyzing "tqdm": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_builder_cec2.py:18: error: Module has no attribute "c_make_encoder"
clarity/data/scene_builder_cec2.py:27: error: Module has no attribute "float"
clarity/data/HOA_tools_cec2.py:5: error: Skipping analyzing "numba": module is installed, but missing library stubs or py.typed marker
clarity/data/HOA_tools_cec2.py:5: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
clarity/data/HOA_tools_cec2.py:6: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/data/HOA_tools_cec2.py:7: error: Skipping analyzing "scipy.spatial.transform": module is installed, but missing library stubs or py.typed marker
clarity/data/HOA_tools_cec2.py:8: error: Skipping analyzing "scipy.special": module is installed, but missing library stubs or py.typed marker
clarity/predictor/torch_stoi.py:9: error: Skipping analyzing "torchaudio": module is installed, but missing library stubs or py.typed marker
clarity/predictor/torch_stoi.py:10: error: Skipping analyzing "pystoi.stoi": module is installed, but missing library stubs or py.typed marker
clarity/predictor/torch_stoi.py:11: error: Skipping analyzing "pystoi.utils": module is installed, but missing library stubs or py.typed marker
clarity/predictor/torch_stoi.py:179: error: Item "None" of "Optional[Tensor]" has no attribute "mean"
clarity/predictor/torch_msbg.py:11: error: Skipping analyzing "torchaudio": module is installed, but missing library stubs or py.typed marker
clarity/predictor/torch_msbg.py:12: error: Skipping analyzing "scipy.fftpack": module is installed, but missing library stubs or py.typed marker
clarity/predictor/torch_msbg.py:13: error: Skipping analyzing "scipy.interpolate": module is installed, but missing library stubs or py.typed marker
clarity/predictor/torch_msbg.py:14: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/msbg/cochlea.py:4: error: Skipping analyzing "scipy": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/mbstoi/mbstoi.py:5: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/enhancer/gha/gha_utils.py:6: error: Skipping analyzing "scipy.interpolate": module is installed, but missing library stubs or py.typed marker
clarity/engine/system.py:5: error: Cannot find implementation or library stub for module named "pytorch_lightning"
clarity/dataset/cec1_dataset.py:4: error: Skipping analyzing "librosa": module is installed, but missing library stubs or py.typed marker
clarity/dataset/cec1_dataset.py:7: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/dataset/cec1_dataset.py:8: error: Skipping analyzing "soundfile": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_renderer_cec2.py:7: error: Skipping analyzing "librosa": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_renderer_cec2.py:12: error: Skipping analyzing "scipy.io": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_renderer_cec2.py:13: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_renderer_cec2.py:14: error: Skipping analyzing "tqdm": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_renderer_cec1.py:7: error: Skipping analyzing "soundfile": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_renderer_cec1.py:8: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/msbg/msbg.py:5: error: Skipping analyzing "scipy": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/msbg/msbg.py:6: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/enhancer/gha/gha_interface.py:8: error: Skipping analyzing "soundfile": module is installed, but missing library stubs or py.typed marker
Found 48 errors in 22 files (checked 42 source files)

We may wish to apply the same standards to tests/*...

 ❱ mypy tests  
clarity/evaluator/haspi/ebm.py:4: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/haspi/eb.py:2: error: Skipping analyzing "numba": module is installed, but missing library stubs or py.typed marker
clarity/evaluator/haspi/eb.py:3: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/data/utils.py:4: error: Skipping analyzing "scipy": module is installed, but missing library stubs or py.typed marker
clarity/data/utils.py:5: error: Skipping analyzing "scipy.io": module is installed, but missing library stubs or py.typed marker
clarity/data/HOA_tools_cec2.py:5: error: Skipping analyzing "numba": module is installed, but missing library stubs or py.typed marker
clarity/data/HOA_tools_cec2.py:6: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/data/HOA_tools_cec2.py:7: error: Skipping analyzing "scipy.spatial.transform": module is installed, but missing library stubs or py.typed marker
clarity/data/HOA_tools_cec2.py:8: error: Skipping analyzing "scipy.special": module is installed, but missing library stubs or py.typed marker
clarity/enhancer/nalr.py:4: error: Skipping analyzing "scipy": module is installed, but missing library stubs or py.typed marker
clarity/enhancer/nalr.py:5: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_renderer_cec2.py:7: error: Skipping analyzing "librosa": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_renderer_cec2.py:12: error: Skipping analyzing "scipy.io": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_renderer_cec2.py:13: error: Skipping analyzing "scipy.signal": module is installed, but missing library stubs or py.typed marker
clarity/data/scene_renderer_cec2.py:14: error: Skipping analyzing "tqdm": module is installed, but missing library stubs or py.typed marker
tests/test_full_pipeline.py:9: error: Skipping analyzing "scipy.io": module is installed, but missing library stubs or py.typed marker
tests/test_full_pipeline.py:9: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 16 errors in 7 files (checked 4 source files)

Once these have been addressed we can enable the mypy automated static type checking/linting available under pre-commit.

Develop unit tests for the data module (see #80 for overview).

Specific modules within evaluator are...

• evaluator/haspi
• evaluator/haspi/eb.py
• evaluator/haspi/ebm.py
• evaluator/haspi/ip.py
• evaluator/haspi/haspi.py
• evaluator/hasqi/hasqi.py
• evaluator/haaqi/haaqi.py
• evaluator/mbstoi
• evaluator/mbstoi/mbstoi.py
• evaluator/mbstoi/mbstoi_utils.py
• evaluator/msbg
• evaluator/msbg/audiogram.py No change required.
• evaluator/msbg/cochlea.py
• evaluator/msbg/msbg.py
• evaluator/msbg/msbg_utils.py
• evaluator/msbg/smearing.py

Docstrings are currently written in Python's ReSructured Text.

An easier to read and more flexible docstring style is the Google Style docstrings style which Sphinx supports additional sections and renders with Sphinx via the napoleon Sphinx extension.

In particular including examples is very useful.

Code was written for hydra 1.1. - Newer hydra 1.2 has introduced some breaking changes in the way that working directories operate. This will require changes to the config code to fix. Suggest for now that we pin the version to 1.1 and upgrade to hydra 1.2 in the next release.

It is useful to have contribution guidelines for projects so that participants know what is expected of them when they consider contributing to the code base.

MSBG code (i.e.clarity/evaluator/msbg) being worked on

• improving pylint score, missing docstrings etc
• adding typehints for more robust type checking
• new regression tests

Is your feature request related to a problem? Please describe.

No problem, but as part of improving the code base we can incorporate pylint into the Continuous Integration/Development (CI/CD) pipeline as part of lint.yml via pre-commit.

Describe the solution you'd like

Code should be linted with pylint prior to commits (using pre-commit).

However, first all code needs linting (see separate issue #).

Develop unit tests for the data module (see #80 for overview).

Specific modules within enhancer are...

• enhancer/compressor.py
• enhancer/nalr.py
• enhancer/dnn
• enhancer/dnn/mc_conv_tasnet.py
• enhancer/dsp
• enhancer/dsp/filter.py
• enhancer/gha
• enhancer/gha/audiogram.py
• enhancer/gha/gainrule_camfit.py
• enhancer/gha/gha_interface.py
• enhancer/gha/gha_utils.py

Having issue templates for participants to use when reporting bugs or feature requests will be useful in helping obtain useful information up front that aids in resolving the problem or implementing the feature.

Possible templates include...

• Bug reporting - when the clarity software breaks we need to understand why and under what circumstances so we can investigate and resolve.
• Feature request - allowing users to request new features will be invaluable to growing the software and increase its functionality.
• Challenge problems - somewhere for people to report problems they are having with the challenge itself.

There may be more, please add them if you think there are other scenarios that may be useful.

A repository of templates for issue reporting can be found at stevemao/github-issue-templates: A collection of GitHub issue and pull request templates.

Pre-commit is using Black 19.10b0 from 2019 (actually a beta version). The latest version is 22.6.0 and a lot of issues have been fixed since 19.10b0 - particularly with handling parentheses and docstrings.

Moving to the latest version will lead to the reformatting of 25 files.

The latest iteration of Python's setuptools deprecates the use of setup.py except in some specific circumstances (see note).

Instead it uses a combination of pyproject.toml and setup.cfg to achieve the configuration of the package, its dependencies and usage.

The settings in setup.py should be migrated to pyproject.toml and setup.cfg.