This open-source project aims to provide a standardized benchmark dataset for Khmer Optical Character Recognition (OCR) engine. It consists of different difficulty levels where achieving good results on each level provides insights into specific capabilities of the OCR engine.

In Cambodia, an area of active development in AI technology is the Khmer OCR. However, it is currently difficult to determine the capabilities of Khmer OCR engines in the market. This is due to two reasons. First, those engines tend to benchmark performance on internal testing datasets, which often leads to bias in favor of the creator. Second, some engines ran performance tests on computer-generated opensource datasets provided by the Tessaract project, which does not reflect performance on real-world images. Hence, this project aims to solve this problem by providing a standardized benchmarking dataset for text recognition in the Khmer language.

Each level aims to test the specific capabilities of the OCR engine. The levels are arranged in increasing difficulties and consist of subtasks. Each task consists of images and corresponding labels in JSON format. The labels are created using LabelMe, an open-source Image Polygonal Annotation Tool.

This level provides testing samples for clean digital images in the form of 15 Government Official Documents. There are four tasks at this level that provides variations of the same 15 documents. The table below outlines the description of each task.

1. Khmer printed text in straight lines.
2. Khmer printed text in bold letters.
3. Khmer printed text with a clean background.

OCR engine that performs well on this level can read Khmer text from Government Official Posts.

This level provides testing samples for scene text images which are text in real-world scenery. The images are taken through mobile devices. There are two tasks at this level:

1. Khmer printed text that is curved.
2. Khmer printed text with occlusions/noise.
3. Khmer printed text under different lighting conditions.
4. Khmer printed text with reflection.

OCR engine that performs well on this level can read Khmer printed text in the real world where the environment can’t be controlled.

This level includes testing samples for handwritten text images. There are two tasks at this level:

This is the biggest level of OCR engine that was able to reconginize and understand the human written text that was present along side many of the real world document and text. This require the OCR engine to be capable of handling:

• The inconsistency of human written size, stroke, style, aligment and padding
• Accountable for human error in writing
• The randomness of text location

OCR engine that performs well on this level can read Khmer hand-written text in the real world where the environment can’t be controlled.

(back to top)

1. Download the dataset from one of the levels here.

2. Test the dataset with your own OCR engine and provide the output in one text file where each line in the text file consists of a pair of prediction and label separated by a tab.

ការដើរលេង  ការដើរលេង

3. Install the neccessary python packages from requirements.txt

pip install -r requirements.txt

4. Start the evaluation using scripts/evaluate.py.

python scripts/evaluate.py --input PATH_TO_OCR_OUTPUT_FILE

• Level 1: Clean Digital Images
• Level 2: Scene Text Images
• Level 3: Hand Written Images

This open project is initiated in collaboration with institutions from different industries in Cambodia. We are grateful for their continuing support of this project and our mission to help improve the state of the Khmer OCR engine.

EKYC Solutions	Prudential Life Assurance PLC	Paragon International University

Distributed under the MIT License. See LICENSE for more information.

Email: [email protected]

(back to top)