This repository contains a series of simple scripts for testing three different methods of parsing binary files in Python. The goal of this project was to compare speeds of two libraries for parsing binary files (Kaitai and Construct) as well as a pure Python method.

This project was deveeloped in Python 3.6.9. Please refer to requirements.txt for package information.

See below for information on generating test data and parsing.

###Generating Test Data

After cloning the repository, you should run the script test_data_generator.py in order to create binary files for parsing testing. Modify the following lines to change file sizes:

# desired file sizes
file_size_small = 100*1024 # 100 kib
file_size_medium = 100*1024**2 # 100 mib
file_size_big = 1024**3 # 1 gib

To test parsing, run the script test_suite.py. Currently the largest file size is skipped, this can be modified in the followign lines of code my modifying the list after "for file_size in":

# benchmark how long it takes to parse each file in various parsing types
    for parse_type in ["construct",  "kaitai", "homebrew"]:
        for file_type in [data_simple_fn, data_varying_fn, data_complex_fn]:
            for file_size in [small_f, med_f]: #[small_f, med_f, big_f]:

If desired, the structure of generated files can be easily modified by changing the format definitions in construct_format_defs.py. Note that parsers will have to be updated accordingly. Please refer to Kaitai documentation as well as Construct documentation for more information on that.

• Hayden Riddiford - haydenridd

This project is licensed under the MIT License - see the LICENSE.md file for details