Augmentation pipeline for rendering synthetic paper printing, faxing, scanning and copy machine processes.

Training neural networks to work with images requires us to augment them in a variety of ways so that they learn to generalize. Networks designed to work with scanned document images must be trained with images that have the type of distortions seen in the wild.

Augraphy is an augmentation library developed to emulate these effects in a pipeline designed to emulate the real world process of printing and scanning a document.

Augraphy's augmentation pipeline starts with augmentations that emulate effects like Ink Bleed, Dusty Ink and Low Ink. Augraphy then virtually prints it to paper that has been generated mathematically or cropped from paper texture images. This image is then augmented further with distortions that can be created by scanners.

The end result is an image that mimics real scanned document images.

Example

Use the package manager pip to install augraphy.

pip install augraphy

from Augraphy import AugraphyPipeline
from Augraphy.Augmentations import *

from Augraphy import default_augraphy_pipeline

pipeline = default_augraphy_pipeline()

from Augraphy import default_augraphy_pipeline

pipeline = default_augraphy_pipeline()
pipeline.ink_phase.augmentations.append(BrightnessAugmentation('ink'))

paper_factory = pipeline.paper_phase.augmentations[0]
paper_factory.p = 1.0

jpeg_aug = pipeline.post_phase.augmentations[-1]
jpeg_aug.quality_range = (10, 20)

ink_phase = AugmentationSequence([
    InkBleedAugmentation(),
    DustyInkAugmentation(),
    LowInkBlobsAugmentation(), 
    OneOf([
        LowInkRandomLinesAugmentation(use_consistent_lines=False),
        LowInkRandomLinesAugmentation(use_consistent_lines=True), 
        LowInkPeriodicLinesAugmentation(use_consistent_lines=False), 
        LowInkPeriodicLinesAugmentation(use_consistent_lines=True), 
    ]),
    GaussianBlurAugmentation('ink', p=1)
])

paper_phase = AugmentationSequence([
    PaperFactory(),
    OneOf([
    AugmentationSequence([
        NoiseTexturizeAugmentation(p=1.0),
        BrightnessTexturizeAugmentation(),
        GaussianBlurAugmentation('paper', [(3,3), (3,5), (5,3), (5,5)]),
        ]),
    AugmentationSequence([
        BrightnessTexturizeAugmentation(p=1.0),
        NoiseTexturizeAugmentation(),
        GaussianBlurAugmentation('paper', [(3,3), (3,5), (5,3), (5,5)]),
    ])]),
    BrightnessAugmentation('paper')
])


post_phase = AugmentationSequence([
    OneOf([
            LightingGradientAugmentation(),
            BrightnessAugmentation('post')
    ]),
    SubtleNoiseAugmentation(),
    JpegAugmentation()
])

pipeline = AugraphyPipeline(ink_phase, paper_phase, post_phase)

img = cv2.imread("image.png")
data = pipeline.augment(img)

The output of the pipeline will be a dictionary containing the image at various stages of processing along with augmentations applied. Additional metadata can be added by augmentations in the pipeline.

Augmentation Sequenece creates a sequence of augmentations that will be executed in order.

Usage:

augmentation = AugmentationSequence(
        augmentations=[
            # Add Augmentations Here
        ]
        p=1.0
    )

Augmentation list that will execute one of the specified augmentations randomly. Probabilities for the specified augmentations will be used as weights for which one will be selected and the probability at the OneOf level will be used to determine if any are selected.

Usage:

augmentation = OneOf(
        augmentations=[
            # Add Augmentations Here
        ]
        p=0.5
    )

The Gaussian Blur augmentations applies a gaussian blur to the whole image.

Usage:

augmentation = GaussianBlurAugmentation(
        kernels=[(3,3)], 
        sigmaX=0,
        p=0.5
    )

The Brightness augmentation adjusts the brightness of the whole image by a chosen multiplier.

Usage:

augmentation = BrightnessAugmentation(
        range=(0.8, 1.4)
        p=0.5
    )

The Ink Bleed augmentation relies on sobel edge detection to create a mask of all edges, then applies random noise to those edges. When followed by a blur this creates a fuzzy edge that emulates an ink bleed effect.

Usage:

augmentation = InkBleedAugmentation(
        intensity_range=(.1, .2), 
        color_range=(0, 224)
        p=0.5
    )

Example:

Before and After Blur

The Dusty Ink augmentation applies random noise to the ink itself, emulating a dusty or inconsistent ink tone when followed by a blur.

Usage:

augmentation = DustyInkAugmentation(
        intensity_range=(.1, .2), 
        color_range=(0, 224)
        p=0.5
    )

Example:

Before and After Blur

The Low Ink Blobs augmentation uses sklearn.datasets.make_blobs to create random blobs of "low ink" that will be applied to the image.

Usage:

augmentation = LowInkBlobsAugmentation(
        count_range=(5, 25), 
        size_range=(10, 20), 
        points_range=(5, 25), 
        std_range=(10, 75), 
        features_range=(15, 25), 
        value_range=(180, 250),
        p=0.5
    )

Example:

LowInkLinesRandomAugmentation and LowInkLinesPeriodicAugmentation inherit from LowInkLineAugmentation. LowInkLinesRandomAugmentation adds low ink lines randomly throughout the image while LowInkLinesPeriodicAugmentation creates a set of lines that repeat in a periodic fashion throughout the image.

Usage:

augmentation = LowInkRandomLinesAugmentation(
        count_range=(5, 10), 
        use_consistent_lines=True,
        p=0.5
    )

augmentation = LowInkPeriodicLinesAugmentation(
        count_range=(5, 10),
        period_range=(10, 30),
        use_consistent_lines=True,
        p=0.5
    )

Example:

The Augmentations specified in the paper phase will be applied to the background paper image returned by the paper factory before the ink is printed to it. A solid grayscale color between 255 and 196 image can also be used as a source and will be texturized by the default augmentations documented here.

Initial Image used for Examples:

(Blank Paper Page)

Randomly replaces the starting paper image with a texture chosen from a directory and resized to fit or cropped and tiled to fit.

Usage:

augmentation = PaperFactory(
        tile_texture_shape=(250,250), 
        texture_path="./paper_textures"
        p=0.5
    )

The Noise Texturize augmentation creates a random noise based texture pattern to emulate paper textures.

Usage:

augmentation = NoiseTexturizeAugmentation(
        sigma_range=(3, 10), 
        turbulence_range=(2, 5)
        p=0.5
    )

Example:

The Noise Texturize augmentation creates a random noise in the brightness channel to emulate paper textures.

Usage:

augmentation = BrightnessTexturizeAugmentation(
        range=(0.9, 0.99), 
        deviation=0.03
        p=0.5
    )

Example:

When generating or augmenting paper textures, the final blur step results in realistic looking paper textures.

Usage:

augmentation = AugmentationSequence([
        NoiseTexturizeAugmentation(),
        BrightnessTexturizeAugmentation()
        GaussianBlurAugmentation([(3,3), (3,5), (5,3), (5,5)])
    ])

Example:

The Dirty Rollers augmentation emulates an effect created by certain document scanners.

Usage:

augmentation = DirtyRollersAugmentation(
        line_width_range=(8, 12)
        p=0.5
    )

Example:

The Lighting Gradient augmentation generates a decayed light mask generated by a light strip given its position and direction and applies it to the image as a lighting or brightness gradient.

Usage:

augmentation = LightingGradientAugmentation(
        light_position=None, 
        direction=None, 
        max_brightness=255, 
        min_brightness=0, 
        mode="gaussian", 
        linear_decay_rate=None, 
        transparency=None
        p=0.5
    )

  position: tuple of integers (x, y) defining
  direction: integer from 0 to 360 to indicate the rotation degree of light strip
  max_brightness: integer that max brightness in the mask
  min_brightness: integer that min brightness in the mask
  mode: the way that brightness decay from max to min: linear or gaussian
  linear_decay_rate: only valid in linear_static mode. Suggested value is within [0.2, 2]

Example:

The Subtle Noise augmentation emulates the imperfections in scanning solid colors due to subtle lighting differences.

Usage:

augmentation = SubtleNoiseAugmentation(
        range=5,
        p=0.5
    )

Example:

Note: Example below created with a range of 25.

The gamma augmentation randomly applies gamma correction (from a range of value) on the overall image. Gamma value of 1 will not effect the image.

Usage:

augmentation = GammaAugmentation(
        range=(0.5,1.5),
        p=0.5
    )

Example:

In Example image below : lefmost picture is original, middle image is obtained with value=1.4, and rightmost image is obtained with value = 0.6

The JPEG augmentation uses JPEG encoding to create JPEG compression artifacts in the image.

Usage:

augmentation = JpegAugmentation(
        quality_range=(50, 95),
        p=0.5
    )

Example:

Encoded with quality range of (10, 15)

You may wish to use Augraphy with other projects, such as Albumentations, imgaug, or imagecorruptions.

We support wrapping Albumentations augmentations and imgaug augmenters with the ForeignAugmentation class, defined in src/augraphy/utilities/foreign.py. imgaug already provides wrapper classes for imagecorruptions transforms, so transitively we support those too.

This library uses the Python standard library's random module to generate pseudorandom numbers. If you want to limit nondeterministic results, consider setting the random seed in programs that use the Augraphy library, by including random.seed(42) in your scripts. (The number 42 is not required; feel free to choose your own memorable seed when requiring deterministic RNG).

Depending on your application, you may also need to set the numpy random seed or the seeds for other sources of randomness (for a brief overview, see here), but Augraphy does not depend on these for random number generation.

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

MIT

Copyright 2021 Sparkfish LLC

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.