1. Features
2. Issues
3. Prerequisites
4. Installation
5. Running
6. User guide
7. Plugins HOWTO

• Interactive tools
  • Brush
  • Knife
  • Bucket
• Change the working image size
• Plugin system
  • Region computations - plugin features that compute regions
  • Property computations - plugin features that compute properties of regions
    • deadline - Jan 4 2022
  • Tools - plugin specific tools
    • deadline - TBD

1. The UI is still WIP, so when you click buttons when no folder is opened yet, the App may throw Exceptions
2. The label modifying logic (e.g. Brush painting...) is done in the main Thread, therefore the interactive tools are still a bit unresponsive. I'll be optimizing this and moving critical stuff into separate process soon.

Download and install miniconda from: https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html

NOTE: On Windows, run steps 3 - 7 in the Anaconda Prompt

1. Clone this repository
2. Go to the root directory of the cloned repository
3. Create a new virtual environment by running conda create -n arth python=3.8 in terminal.
4. Activate the virtual environment: conda activate arth
5. Install the requirements: python -m pip install -r requirements.txt
6. Install the app: python -m pip install -e .

1. In the repo's root directory activate the virtual environment (if not active): conda activate arth
2. Run: python arthropod_describer/app.py

1. Import a folder with photos by clicking File -> Open folder.
   • The app will restructure the folder as follows:
   • 4 folders will be created:
     • images: all the images from the opened folder will be moved here.
     • masks: binary images marking bugs.
     • sections: integer label images marking sections of bug bodies
     • reflections: binary images marking reflection spots on bugs
   • NOTE: as the app modifies the folder structure, maybe it's better to work with a copy of the data.
2. On the left a photo can be selected to view and modify its label images.
3. At top of photo view, active label image can be selected: Bug, Segments, Reflections
   • Bug serves as a mask for Segments and Reflections
4. On the right, a colormap can be selected; for a selected colormap the active label can be selected.
   • The selected label will be used by tools, which can be selected below the colormap selection widget.
5. On the Plugin tab you can browse the installed plugins and the Computations that they provide.
6. In photo view:
   • Mouse wheel: zooms in/out
   • Mouse drag + holding Mouse wheel: view panning
   • Left click: active tool action
   • Right click: nothing yet
   • Twice SHIFT press - quick label search

• Currently the label images are saved to disk when the user changes switches to another photo. This should be discussed, as in the old ArthDescriber version, the label images had to be explicitly "Approved" to be saved.

This implementation of Arthropod Describer is essentially just a viewer for photos and their label images with three interactive tools; the main functionality comes from plugins.

Plugins (each in its own separate directory) must be placed into {root}/arthropod_describer/plugins/ .

An example of plugin directory structure:

.
├── __init__.py
├── plugin.py
├── properties
│   ├── __init__.py
│   └── average_intensity.py
├── tools
│   ├── __init__.py
│   └── some_cool_tool.py
├── plugin.py
└── regions
    ├── __init__.py
    ├── body.py
    ├── eraser.py
    └── legs.py

A plugin consists of RegionComputations, PropertyComputations and Tools.

RegionComputations must be placed in regions dir, and similarly for PropertyComputations and Tools

To create a plugin, follow the above example directory structure, placed into the directory plugins/ Plugin is essentially just an aggregation of Computations and Tools, so start with creating them first.

You must provide a name and description for every Computation that your plugin provides. You do that by providing a class docstring in the required format. Example:

class BodyComp(RegionComputation):
  """
  NAME: Primitive bug body finder.
  DESC: Labels the whole body of a bug based on thresholding the blue channel.
  """

For Computations that require user input, you can provide the user the option to tweak parameters. Again, you define the user parameters in the docstring:

class BodyComp(RegionComputation):
  """
  NAME: Primitive bug body finder.
  DESC: Labels the whole body of a bug based on thresholding the blue channel.

  USER_PARAMS:
      NAME: Blue threshold
      KEY: threshold
      PARAM_TYPE: INT
      VALUE: 200
      DEFAULT_VALUE: 200
      MIN_VALUE: 0
      MAX_VALUE: 255

      NAME: Filter size for small components
      KEY: filter_size
      PARAM_TYPE: INT
      VALUE: 25
      DEFAULT_VALUE: 25
      MIN_VALUE: 1
      MAX_VALUE: 55
  """

• NAME - this will be displayed in the App
• KEY - for internal identification, must be unique within the defining Computation
• PARAM_TYPE must be from {INT, STR, BOOL}
• VALUE - current value
• DEFAULT_VALUE - self-explanatory
• MIN_VALUE, MAX_VALUE - used only when PARAM_TYPE: INT, also self-explanatory

Computations that can operate on individual regions, they can be declared as REGION_RESTRICTED, as follows:

class RegionEraser(RegionComputation):
  """
  NAME: Region eraser
  DESC: Erases regions with the selected labels.

  REGION_RESTRICTED
  """

Basically, put REGION_RESTRICTED anywhere in the docstring. When the user will activate a REGION_RESTRICTED Computation, besides the Computation's user parameters, the user will be also provided the option to select individual region labels on which the Computation will operate.

Subclass the class RegionComputation located in the arthropod_describer.common.plugin module and override the methods/properties:

@property
def requires(self) -> Set[LabelType]:
    pass

@property
def computes(self) -> Set[LabelType]:
    pass

def __call__(self, photo: Photo, labels: Optional[Set[int]] = None) -> Set[LabelType]:
    pass

The requires and computes property indicates which label types (LabelType.BUG, LabelType.REGIONS, LabelType.REFLECTION) it needs for computation or produces as a results respectively.

The most important is the __call__ method, where the actual computation takes place. It takes Photo as a parameter, and you have an access to these images in the numpy.ndarray format:

1. image - the input RGB image
2. bug_mask - mask indicating the whole insect (currently the value 1000 indicates insect, 0 everything else)
3. regions_image - label image indicating various parts of the insect (legs, thorax...)
4. reflection_mask - mask indicating spots on the insect that should be excluded from computations of properties. The value 1000 indicates that a particular pixel should be excluded.

The result of the __call__ method is a Python set of LabelType indicating which label images were modified.

Example __call__ implementation:

def __call__(self, photo: Photo, labels: Optional[Set[int]] = None) -> Set[LabelType]:
    cop = photo.regions_image.label_img.copy()
    for lab in labels:
        cop = np.where(cop == lab, 0, cop)
    photo.regions_image = cop
    return {LabelType.REGIONS}

When finished with implementing Computation, assembly them into Plugin.

Subclass the class Plugin from arthropod_describer.common.plugin.

Example:

from arthropod_describer.common.plugin import Plugin, PropertyComputation, RegionComputation
from arthropod_describer.common.common import Info
from arthropod_describer.common.tool import Tool


class TestPlugin(Plugin):
    """
    NAME: Test
    DESCRIPTION: This is a test plugin for the arthropod describer tool
    """

    def __init__(self, info: Optional[Info] = None):
        super().__init__(info)