Yara rule aggregation and modification tool.
Inspired by tools used to manage open source NIDS signatures such as pulledpork and suricata-update
- Aggregate Yara rules from multiple remote git repositories into a single file
- Raw
.yara
and compiled.yarac
- Note: The compiled yara rule can only be used on a system with the same Yara version
- Raw
- Add
meta
key-value pairs to each Yara rule - Ability to modify specific rules via regex
- Ability to disable specific rules via rule name
The script will output several files
- Files
${rule_path}/signatures.yara
andsignatures.yarac
- All signatures compiled into a single file${rule_path}/[RULESET].yara
and[RULESET].yarac
- All the signatures from each ruleset compiled into a single file
- Directories
${rule_path}/compendium/
- Holds all the modified Yara rule files in a single directory${rule_path}/[RULESET]/
- Raw Yara rule files from each ruleset
- Install OS dependencies (Ubuntu 20.04). Install python3-yara and libyara3 here to get the proper linking.
apt install git python3 python3-pip python3-yara libyara3 yara
- Create and activate virtual environment
python3 -m venv venv source venv/bin/activate
- Install
pip3 install .
docker build . -t yara-compendium:latest
usage: yara-compendium [-h] [-c CONFIG] [-v]
Yara Compendium
optional arguments:
-h, --help show this help message and exit
-c CONFIG, --config CONFIG
Configuration file path (default: etc/config.yml)
-v, --verbose Verbose (DEBUG level logs) (default: False)
This configuration file has example values and must be edited before usage.
git_repos
(list of dictionaries) - Configuration details for each remote git repository to pull Yara signatures fromname
(str) - Arbitrary string of what to name this repositoryurl
(str) - Link to the remote git repositorybranch
(str) - Branch of the git repository to usesub_dirs
(Boolean) - Whether to look for Yara rules in all subdirectoriesroot_dir
(Boolean) - Whether to look in the root directory for Yara rulesinclude_dirs
(list of str)- Specific subdirectories to search for Yara rulesexclude_dirs
(list of str)- Specific subdirectories to exclude searching for Yara rules (requiressub_dirs
to betrue
)metadata
(dict) - Arbitrary key-value pairs to add to the metadata of each Yara rule- If a key already exists, this will overwrite the value
- Examples:
license
,source_repo
,tags
indent
(int) - Number of spaces to use when indentingvalid_ext
(list) - Yara rule file extensions to search forrule_path
(str) - Path to save output (will be created if it does not exist)configs
(dict) - Path to the alteration configuration filesmodify
(str) - Path tomodify.conf
disable
(str) - Path todisable.conf
cleanup_raw_rules
(Boolean) - Whether to delete the raw (git clone) rules when completekeep_uncompiled
(Boolean) - Whether to keep the non.yarac
files when complete
Modify specific rules via regular expression
Each line of this file uses the following format:
RULENAME "FIND REGEX" "REPLACE STRING"
Example
is_pe "MZ" "{ 4D 5A }"
Disable specific rules by name. If a disabled rule is referenced in another rule, it will cause a syntax error in the other rule, which will then be skipped from being included.
Each line of this file uses the following format:
RULENAME
Example
is_pe
Local example
$ yara-compendium -v
2022-07-08 17:07:04,390 yara-compendium DEBUG: Git repo Example cloned to ./rules/Example
2022-07-08 17:07:04,395 yara-compendium DEBUG: Directories from rule set './rules/Example' to gather Yara rules: ['./rules/Example/malware', './rules/Example/webshells']
2022-07-08 17:07:04,592 yara-compendium WARNING: Syntax Error - bad_example.yar in Example ruleset
2022-07-08 17:07:04,674 yara-compendium INFO: Wrote compiled Yara file: ./rules/Example.yarac
2022-07-08 17:07:05,023 yara-compendium DEBUG: Git repo Example2 cloned to ./rules/Example2
2022-07-08 17:07:05,045 yara-compendium DEBUG: Directories from rule set './rules/Example2' to gather Yara rules: ['./rules/Example']
2022-07-08 17:07:05,100 yara-compendium INFO: Wrote compiled Yara file: ./rules/Example2.yarac
2022-07-08 17:07:55,258 yara-compendium INFO: Wrote compiled Yara file: ./rules/signatures.yarac
Docker example
$ mkdir rules_out
$ docker run --rm -v $(pwd)/etc:/app/etc -v $(pwd)/rules_out:/app/rules yara-compendium:latest -v
- Allow for regular expressions in the
modify.conf
rule name