Twitter | WACV | arXiv | Explore the Dataset | Hugging Face Spaces | eval.ai | dacl.ai Workshop Page

ℹ️ This module should act as an entry point for simple data usage for the dacl10k dataset.

📛 dacl10k stands for damage classification 10k images and is a multi-label semantic segmentation dataset for 19 classes (13 damages and 6 objects) present on bridges.

🏆 This dataset is used in the challenge associated with the "1st Workshop on Vision-Based Structural Inspections in Civil Engineering" at WACV2024.

python -m pip install git+https://github.com/phiyodr/dacl10k-toolkit

• Requirements: See requirements.txt
• Tested on Ubuntu 20.04 LTS with Python 3.10

📃 Data licence: (CC BY-NC 4.0)

📝 Please cite the corresponding paper: Johannes Flotzinger, Philipp J. Rösch and Thomas Braml, "dacl10k: Benchmark for Semantic Bridge Damage Segmentation," arXiv:2309.00460 [cs.CV], Sep. 2023."

(new!) Visualize dacl10k with the Voxel51 demo:

¹ Hosted at RWTH Aachen University.

The data should be in folder path_to_data and in the following format:

├── annotations
│   ├── train (n=6.935)
│   └── validation (n=975)
└── images
    ├── train (n=6.935)
    ├── validation (n=975)
    ├── testdev (n=1.012)
    └── testchallenge (will be made available on 27 October 2023)

Multi-label semantic segmentation

• Multi-label: Each pixel can be associated with several classes, e.g. a surface can have Rust and Crack (2 damages) or Rust on Drainage (1 damage and 1 object).
• Semantic segmentation: Pixel detailed annotation of damages and objects.
• We do not care about "instances". If several polygons of the same class overlap, they are merged into one mask.

For evaluation we use mean Intersection over Union (mIoU).

Annotation files originate from labelme-format and are slightly adjusted. One JSON file looks like this:

{'imageWidth': 1280,
 'imageHeight': 960,
 'imageName': 'dacl10k_v2_validation_0012.jpg',
 'imagePath': 'images/train/dacl10k_v2_validation_0012.jpg', 
 'split': 'validation',
 'dacl10k_version': 'v2',
 'shapes': [{'label': 'Rust',
   'points': [...],
   'shape_type': 'polygon'},
  {'label': 'Rust',
   'points': [...],
   'shape_type': 'polygon'},
  {'label': 'Drainage',
   'points': [[581.5714285714286, 410.2857142857142],
    [555.8571428571428, 407.4285714285714],
    [524.4285714285713, 435.99999999999994],
    [507.2857142857142, 486.0],
    [502.99999999999994, 531.7142857142857],
    [550.1428571428571, 574.5714285714286],
    [578.7142857142857, 593.1428571428571],
    [612.9999999999999, 560.2857142857142],
    [625.8571428571428, 508.8571428571428],
    [622.9999999999999, 447.42857142857133],
    [605.8571428571428, 420.2857142857142]],
   'shape_type': 'polygon'}]}

Explanation:

• imageWidth, imageHeight: Image width and height (many different image sizes exists).
• imageName, imagePath, split: Corresponding image name and full path, as well the corresponding dataset split.
• dacl10k_version: The first version of the arXiv paper uses v1 and the challenge uses v2.
• shapes (list of dictionaries): Each dictionary contains
  • label (str): Name of the class
  • points (list of lists): List of edge points of the polygone (x, y). Origin in top-left corner.
  • shape_type: Always 'polygone' (originates from labelme)

Each polygone can have one of 19 classes. For detailed explanation please see Table 7 through 9 in the Appendix of Flotzinger et al., 2023.

• 13 damage classes: Crack, Alligator Crack (ACrack), Wetspot, Efflorescence, Rust, Rockpocket, Hollowareas, Cavity, Spalling, Graffiti, Weathering, Restformwork, Exposed Rebars (ExposedRebars),
• 6 object classes: Bearing, Expansion Joint (EJoint), Drainage, Protective Equipment (PEquipment), Joint Tape (JTape), Washouts/Concrete corrosion (WConccor)

• Dacl10kDataset a torch Dataset class for the dacl10k dataset.
• utils with labelme2mask, resize_images, and resize_annotations

ℹ️ This functionality should be a starting point for simple Torch Dataset usage. It is not a comprehensive dataset module for all SOTA work (e.g. no augmentation, etc.).

What is does:

• Loades images and resize it to a given shape
• Loades annotation files (labelme-like format) and creates a target mask (torch.tensor) for the 19 classes from the Polygones provided.

from dacl10k.dacl10kdataset import Dacl10kDataset

split = "validation" # "train", "testdev" (later also "testchallenge")
path_to_data = "my/path/to/data"
dataset = Dacl10kDataset(split, path_to_data, resize_mask=(512, 512), resize_img=(512, 512), normalize_img=True)
img, mask = dataset[100]
img.shape, mask.shape
#> (torch.Size([3, 512, 512]), torch.Size([19, 512, 512]))

• The mask has 19 channels in the following order:

dataset.TARGET_LIST
#> ['Crack', 'ACrack', 'Wetspot', 'Efflorescence', 'Rust', 'Rockpocket', 'Hollowareas', 'Cavity', 'Spalling', 'Graffiti', 'Weathering', 'Restformwork', 'ExposedRebars', 'Bearing', 'EJoint', 'Drainage', 'PEquipment', 'JTape', 'WConccor']

Advantage:

• Easy to start implementation for loading images (.jpg) and annotations (.json).

Disadvantage:

• Images are large, hence loading takes a lot of time. Annotations are available in Polygone format and need to be transformed to masks, this takes a lot of time. To overcome this problem use prefetching.

What is does:

• All images and annotations in the current split are loaded (in a distributed fashion using n_jobs workers) and stored in a dictionary. Loading is done once at the beginning and not during training.

split = "validation" # "train", "testdev" (later also "testchallenge")
path_to_data = "my/path/to/data"
dataset = Dacl10kDataset(split, path_to_data, resize_mask=(512, 512), resize_img=(512, 512), normalize_img=True)

dataset.run_prefetching(n_jobs=10)
#> Prefetching started
#> Prefetchting: 100%|████████████████████████| 975/975 [04:49<00:00,  3.36it/s]
#> Prefeching done. Data stored in `self.prefetched_data`.

len(dataset.prefetched_data), type(dataset.prefetched_data)
#> (975, dict)

sample = dataset.prefetched_data["dacl10k_v2_validation_0000.jpg"]
sample.keys()
#> dict_keys(['image', 'mask'])

• It is best to save the prefetched data, so you can reuse it:

path_to_store_prefechted_data = "my/path/to/store/prefetched/data"
dataset.save_prefetched_data(path_to_store_prefechted_data) # you can also pass a filename if you want otherwise the split name ("validation.pkl") is used.
#> Start saving prefeched data.
#> Prefetched data saved in my/path/to/store/prefeched/data/validation.pkl.
del dataset 

• Load data using load_prefetched_data. Since images and masks are already in requested format you should set resize_mask and resize_img to None:

filename_of_prefechted_data= "validation.pkl"
dataset = Dacl10kDataset(split, path_to_data, resize_mask=None, resize_img=None, normalize_img=True)
dataset.load_prefetched_data(path_to_store_prefechted_data, filename_of_prefechted_data)
img, mask = dataset[100]
img.shape, mask.shape
#> (torch.Size([3, 512, 512]), torch.Size([19, 512, 512]))

Advantage:

• Loading images and annotations in Dacl10kDataset and DataLoader is super quick.
• Prefetched data is already in requested image and mask size (saves memory).

Disadvantage:

• You need to run prefetching once in the beginning, which takes some time (approx. 10 min for 1k images using 10 workers).

from dacl10k import utils

Annotations are provided in labelme-like format. With utils.labelme2mask transform from labelme to dense array format is made easy.

• Load annotation file:

import matplotlib.pyplot as plt
from PIL import Image

image_file = "my/path/to/data/images/validation/dacl10k_validation_0012.jpg"
annotation_file = "my/path/to/data/annotations/validation/dacl10k_validation_0012.json"

utils.open_json(annotation_file)

• The annotation in labelme-like format looks like this:

{'imageWidth': 1280,
 'imageHeight': 960,
 'imageName': 'dacl10k_v2_validation_0012.jpg',
 'imagePath': 'images/train/dacl10k_v2_validation_0012.jpg', 
 'split': 'validation',
 'dacl10k_version': 'v2',
 'shapes': [{'label': 'Rust',
   'points': [...],
   'shape_type': 'polygon'},
  {'label': 'Rust',
   'points': [...],
   'shape_type': 'polygon'},
  {'label': 'Drainage',
   'points': [[581.5714285714286, 410.2857142857142],
    [555.8571428571428, 407.4285714285714],
    [524.4285714285713, 435.99999999999994],
    [507.2857142857142, 486.0],
    [502.99999999999994, 531.7142857142857],
    [550.1428571428571, 574.5714285714286],
    [578.7142857142857, 593.1428571428571],
    [612.9999999999999, 560.2857142857142],
    [625.8571428571428, 508.8571428571428],
    [622.9999999999999, 447.42857142857133],
    [605.8571428571428, 420.2857142857142]],
   'shape_type': 'polygon'}]}

• In the following we are interested in Drainage, which has index 15 in the TARGET_LIST.
• We display the example image img.

# Create map from target name to index. In this example we are interested in `Drainage`.
target_dict = dict(zip(utils.TARGET_LIST, range(len(utils.TARGET_LIST))))
target_of_interest = target_dict["Drainage"]

# Display image
img = Image.open(image_file)
img.close()
plt.imshow(img)
plt.show()

• We transform the labelme-like dictionary to a dense mask format (numpy) and plot it:

# Apply labelme2mask transforms
mask = utils.labelme2mask(annotation_file)
mask.shape, type(mask)
#> ((960, 1280, 19), numpy.ndarray)
plt.imshow(mask[:, :, target_of_interest])
plt.show()

If you want to resize images and all values in the annotations files at once, you can use following utilities:

# Resize images
source_folder = "my/path/to/data/images/validation"
target_folder = "my/path/to/resized/data/images/validation"
utils.resize_images(source_folder, target_folder, size=(512,512))

# Resize annotations (in JSON files)
source_folder = "my/path/to/data/annotations/validation"
target_folder = "my/path/to/resized/data/annotations/validation"
utils.resize_annotations(source_folder, target_folder, size=(512,512))

Example file from source folder:

{'imageWidth': 1280,
 'imageHeight': 960,
 'imageName': 'dacl10k_v2_validation_0012.jpg',
 'imagePath': 'images/train/dacl10k_v2_validation_0012.jpg', 
 'split': 'validation',
 'dacl10k_version': 'v2',
 'shapes': [{'label': 'Rust',
   'points': [...],
   'shape_type': 'polygon'},
  {'label': 'Rust',
   'points': [...],
   'shape_type': 'polygon'},
  {'label': 'Drainage',
   'points': [[581.5714285714286, 410.2857142857142],
    [555.8571428571428, 407.4285714285714],
    [524.4285714285713, 435.99999999999994],
    [507.2857142857142, 486.0],
    [502.99999999999994, 531.7142857142857],
    [550.1428571428571, 574.5714285714286],
    [578.7142857142857, 593.1428571428571],
    [612.9999999999999, 560.2857142857142],
    [625.8571428571428, 508.8571428571428],
    [622.9999999999999, 447.42857142857133],
    [605.8571428571428, 420.2857142857142]],
   'shape_type': 'polygon'}]}

Corresponding example in target folder (with new imageWidth, imageHeight, and points):

{'imageWidth': 512,
 'imageHeight': 512,
 'imageName': 'dacl10k_v2_validation_0012.jpg',
 'imagePath': 'images/train/dacl10k_v2_validation_0012.jpg', 
 'split': 'validation',
 'dacl10k_version': 'v2',
 'shapes': [{'label': 'Rust',
   'points': [...]
   'shape_type': 'polygon'},
  {'label': 'Rust',
   'points': [...],
   'shape_type': 'polygon'},
  {'label': 'Drainage',
   'points': [[232.62857142857143, 218.81904761904758],
    [222.3428571428571, 217.29523809523806],
    [209.77142857142854, 232.5333333333333],
    [202.91428571428568, 259.2],
    [201.2, 283.58095238095234],
    [220.05714285714285, 306.43809523809523],
    [231.48571428571427, 316.3428571428571],
    [245.19999999999996, 298.8190476190476],
    [250.3428571428571, 271.39047619047614],
    [249.19999999999996, 238.62857142857138],
    [242.3428571428571, 224.15238095238092]],
   'shape_type': 'polygon'}]}