I could use a little help with today’s packaging of Python module. I did what I could before the new Python distribution method came out a few years back.

One can build by doing

$ python3 -mbuild

view "trusted" {

match-clients { 192.168.23.0/24; };
recursion yes;
zone "example.com" {
type master;
file "internal/master.example.com";
};
zone "example22.com" {
type master;
file "internal/master.example22.com";
};
};
view "badguys" {
match-clients {"any"; };
recursion no;
zone "exampleaa.com" {
type master;
file "external/master.exampleaa.com";
};
};

Even though all the statements and clauses passed their own individual unit test, this is the super unit test, short of integration test where all the settings come into play.

It takes time to revalidate each line in the configuration file and ensuring that the Pythonized variable is properly nested with the correct list/dict in accordance with its configuration setup by repeating clauses, repeating statements, repeating rows, repeating elements ...

One of the many complaints is the error output and how vague it can be.

I discovered that if I change '+' into '-' in the following:

optview_stmt_response_policy_element_break_dnssec = (
    Keyword('break-dnssec').suppress()
    - isc_boolean('break_dnssec')   # <-- note the change of polarity of operator
    )

Do that throughout all the code, as needed.

Background

The error message changes from an obtuse "optview_stmt_response_policy_global_element_set" (which is the one syntax layer higher) into a more concise error message:

    } add-soa no break-dnssec max-policy-ttl 30S min-update-interval 4w min-ns-dots 2 nsip-wait-recurse yes nsdname-wait-recurse yes qname-wait-recurse yes recursive-only yes nsip-enable yes nsdname-enable yes dnsrps-enable yes dnsrps-options unspecified_options;
                              ^(FATAL)
FAIL: Expected <boolean>, found 'm'  (at char 836), (line:7, col:31)

Now you can see exactly what we did wrong.

The correct sequence should be ... break-dnssec yes max-policy-ttl ...

The above output is part of the test_isc_optview_stmt_response_policy_passing test in tests/test_optview.py.

I just noticed that the TSIG key_id is quoted, although identifiers should not:

 'key': [[{'algorithm': 'hmac-sha256',
           'key_id': '"ansible"',
           'secret': 'redacted_key_hash_1'}],
         [{'algorithm': 'hmac-sha256',
           'key_id': '"externaldns"',
           'secret': 'redacted_key_hash_2'}]],

Looks like something went wrong with the issue mentioned in #53 and the fix in dd9fe1a: The fix adds the dot to charset_tls_id_base, while it should go to charset_key_id_base as proposed in my PR #54.

@egberts Do you want me to submit a new PR?

Just faced pyparsing.exceptions.ParseException for option filter-aaaa-on-v4:

pyparsing.exceptions.ParseException: Expected '}', found 'filter'  (at char 2902), (line:98, col:9)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "library/bind9_info.py", line 227, in <module>
    main()
  File "library/bind9_info.py", line 223, in main
    run_module()
  File "library/bind9_info.py", line 208, in run_module
    result = my_clauses.parseString(toplevel_config, parseAll=True)
  File "/usr/local/lib/python3.8/ansible-2.12/lib/python3.8/site-packages/pyparsing/core.py", line 1141, in parse_string
    raise exc.with_traceback(None)
pyparsing.exceptions.ParseSyntaxException: Expected '}', found 'filter'  (at char 2902), (line:98, col:9)

Config:

options {
  […]
  filter-aaaa-on-v4 yes;
  […]
};

Hi @ptmcg ,

Was wondering if there are things that I should check for (a checklist before release of GitHub).

Yeah, I know I released it already in PyPi, but I didn't planned this rollout too well.

Some things that I am thinking are:

• Leverage new pyparsing unittest methods (major rework, impacts tests/ only)
• Renaming of methods to better reflect its methods
• Renaming of this package?
• More example in README

When more than 1 acl exists in the named.conf file, only the last one gets processed - example data:
acl "a3internet" { 10.1.12.25; 10.1.12.26; };
acl "my" { 127.0.0.1; };

the result is that only the last ACL is included in the output json file

Somehow, I missed that one.

Example config:

catalog-zones { 
    zone "cat_zone1" 
        zone-directory "dir/cat_zone_file1" 
        in-memory no
        min-update-interval 1H; 
    zone "cat_zone2" 
        zone-directory "dir/cat_zone_file2" 
        in-memory yes
        min-update-interval 2H; 
    };

EBNF is:

        catalog-zones { zone <string> [ default-primaries [ port <integer>
            ] [ dscp <integer> ] { ( <remote-servers> | <ipv4_address> [
            port <integer> ] | <ipv6_address> [ port <integer> ] ) [ key
            <string> ] [ tls <string> ]; ... } ] [ zone-directory
            <quoted_string> ] [ in-memory <boolean> ] [ min-update-interval
            <duration> ]; ... };

Adjust more unit test to support dequotable key_id

As reported in #4, after fixing #5, the top-level zone statements disappeared from its dictionary result after a zone appeared inside the view statement.

Uses the same named.conf as #4/#5.

If I make this one line, it seems to work, but not like this:
zone "umichtest.net" {
type slave;
file "oncampus/net.umichtest";
masters {
"DNS123" ;
};
};

This dnsrps-options is hard, very hard, and quite irksome.

Give me time, but for 99.98% of you system programmers, it should be fine without this.

Really, this is a ISC quirk. Not going to fix this because it does not make sense.

According to the BIND9 docs the TSIG key name can contain any character that is valid for a DNS domain name, and it looks like the docs encourage using names with trailing dots. However, bind9_parser does not accept dots currently.

pyparsing.exceptions.ParseException: Expected <key_id>, found '.'  (at char 69), (line:4, col:13)

For the following server-addresses in zone clause:

server-addresses { fb03::7 port 9553; 9.9.9.9; };

Was expecting

 'server_addresses': [ {'addr': 'fb03::7', 'ip_port': '9553'},
                        {'addr': '9.9.9.9'}] 

Actually getting:

  'server_addresses': [ { 'addr': 'fb03::7',
                          'ip46_addr_port': ['fb03::7', '9553'],
                          'ip_port': '9553'},
                        { 'addr': '9.9.9.9',
                          'ip46_addr_port': ['9.9.9.9']}],

I think this was something I walked away during last Jan 2021.

bind9_parser seems to not parse multiple TSIG keys correctly. The configuration

key "ansible" {
        algorithm hmac-sha256;
        secret "redacted_key_1";
};

key "externaldns" {
        algorithm hmac-sha256;
        secret "redacted_key_2";
};

results in

  "key": [
    {
      "algorithm": "hmac-sha256",
      "key_id": "\"externaldns\"",
      "secret": "redacted_key_2"
    }
  ],

for result.asDict()

So, the first key statement is missing.

Need to correct the syntax for mastesr option in zone statement.

It is incorrectly re-using the same syntax as masters (top-level) statement.

Oddly, it seems to fail on this part:
logging {
channel "general_file" {
file "/var/log/named/general.log" versions 10 size 104857600;
severity dynamic;
print-time yes;
print-severity yes;
print-category yes;
};
category "general" {
"general_file";
"notice-alert_file";
};

Thanks, this looks useful. Testing with my config,
Parser choked on:

options {
	tcp-clients 1000;
	use-v4-udp-ports {
		1024 65535;
	};
};

Hi,

It seems there is a parsing error on masters statement in zone declaration.
following this config : http://www.zytrax.com/books/dns/ch7/zone.html#masters

from bind9_parser import *
zone = """
zone "example.com" in {
    type slave;
    file "slave/example.com";
    masters mymaster {192.168.2.7;};
};
"""
result = clause_statements.parseString(zone)
print(result)

result is, as expected :
[['"example.com"', 'in', 'slave', '"slave/example.com"', ['mymaster', [['192.168.2.7']]]]]

but with

zone = """
zone "example.com" in {
    type slave;
    file "slave/example.com";
    masters {192.168.2.7;};
};
"""

I got a parsing exception :
pyparsing.ParseException: Expected {W:(ABCD...) | W:(ABCD...) | <master_name>}, found '{' (at char 83), (line:5, col:13)

Also with

zone = """
zone "example.com" in {
    type slave;
    file "slave/example.com";
    masters port 1127 {192.168.2.7; 10.2.3.15 key zt-key; 2001:db8:0:1::15 port 1128;};
};
"""

that is the example from zytrax, I got
pyparsing.ParseException: Expected "{", found '1' (at char 88), (line:5, col:18)

It appears that, after a masters statement, the parsers looks for a word, and a {, but description is optional, and other statements can be present.
Both 3 examples here are valid with named-checkconf bind tool.

There seem to be no support for the keyword masters in the slave zone definitions.

Hi, this is not really an issue but a feature request.

It would be really nice if after parsing the configuration, one could dump it back out, possibly after altering the data structure as needed.

Thanks for this nice project!

_965,_tcp.example.net is a failing test, not a passing test for domain_fqdn function.
]

Add unit tests to support multiple key clause especially for TSIG-related keys.

Bug fixed in #48
Pull-request approved in #49 (in response to #48).

Hello,

First of all, thanks a lot for your work !

I'm looking a way to make a network graph representation of the bind configuration.
I began to make it using python regex it's not good enough.
Searching for existing bind9 parser, I've found yours and I'm testing it now.

I know the code is beta, I've found something looking like a bug.

$ cat test_view.conf

view "red" {
    zone "z1.com" { };
    zone "z2.com" { };
};

view "green" {
    zone "z3.com" { };
    zone "z4.com" { };
};

Running parser $ ~/src/bind9_parser/examples/parse_bind9.py test_view.conf

Start: Is the library quiet?
End: Is the library quiet?
len(result): 2

Plain print(result):
[['"red"', [['"z1.com"'], ['"z2.com"']]],
 ['"green"', [['"z3.com"'], ['"z4.com"']]]]
result: {'view': [{'view_name': '"green"', 'configs': {'zone': {'zone_name': '"z4.com"'}}}]}

print(result.asDict()):
{   'view': [   {   'configs': {   'zone': {   'zone_name': '"z4.com"'}},
                    'view_name': '"green"'}]}
end of result.

The red view is present in result but absent in the dict format.

Thanks in advance for informations.

-- François

Hi,

I was trying to open a configuration file, from a bind installed on Cent-OS, encountered such an error:
pyparsing.ParseSyntaxException: Expected <boolean>, found '-'

I checked the file; Error hit on this property:
memstatistics-file

Decided to run the provided example parse_bind9, to be sure my code is right, but even with example give same error.

To check nothing is wrong with my named.conf, i used this named.conf which come along the example. It raises same error on same property
pyparsing.ParseSyntaxException: Expected <boolean>, found '-'
memstatistics-file

Can you please, tell me what am i missing?

Thank you and much appreciate

Hello,

I recently found your module to try and programmatically examine the BIND config files. This environment specifics are below.

OS - CentOS 8
BIND - 9.11.13-6
Python - 3.6.8
bind9-parser - 0.9.8 (installed via pip3)
parse_bind9.py from the master branch on github.

The parse_bind9.py example script is throwing errors when trying to process the the named.conf file. In particular it is complaining about:

options {
...
memstatistics-file "/var/named/data/named_mem_stats.txt";
...
include "/etc/crypto-policies/back-ends/bind.config";
...
}

The named_mem_stats.txt file does not exist, but shows this exception

Opening /etc/named.rfc1912.zones
Opening /etc/named.root.key
Start: Is the library quiet?
pyparsing.ParseException: Expected , found '-' (at char 576), (line:20, col:22)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/tsl/github/bind9_parser/examples/parse_bind9.py", line 269, in
result = my_clauses.parseString(toplevel_config, parseAll=True)
File "/usr/local/lib/python3.6/site-packages/pyparsing.py", line 1955, in parseString
raise exc
File "/usr/local/lib/python3.6/site-packages/pyparsing.py", line 4065, in parseImpl
raise ParseSyntaxException._from_exception(pe)
pyparsing.ParseSyntaxException: Expected , found '-' (at char 576), (line:20, col:22)

The bind.config does exist and has the following contents:

disable-algorithms "." {
RSAMD5;
DSA;
};
disable-ds-digests "." {
GOST;
};

Attempting to parse that file gives:

Opening /etc/crypto-policies/back-ends/bind.config
Opening /etc/named.rfc1912.zones
Opening /etc/named.root.key
Start: Is the library quiet?
pyparsing.ParseException: Expected "}", found 'd' (at char 1841), (line:49, col:1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/tsl/github/bind9_parser/examples/parse_bind9.py", line 269, in
result = my_clauses.parseString(toplevel_config, parseAll=True)
File "/usr/local/lib/python3.6/site-packages/pyparsing.py", line 1955, in parseString
raise exc
File "/usr/local/lib/python3.6/site-packages/pyparsing.py", line 4065, in parseImpl
raise ParseSyntaxException._from_exception(pe)
pyparsing.ParseSyntaxException: Expected "}", found 'd' (at char 1841), (line:49, col:1)

Commenting both those lines out of the configuration allows the script to parse the the main config files.

Hi egberts,

Sorry to bother, but I spent some time trying to debug my issue with no luck.
When I give the top of my (old BIND 9.6.-ESV-R3) named.conf, I get the error bellow.

I tried to clean the file form special / invisible chars. But it looks like it is a syntax issue ...
If I remove the ' acl "ripe-hostcount"' line, it breaks on 'acl "bogon" {'
Maybe you have an idea ?

Thanks a lot.
F

log from parse_bind9.py

root@39448b75c10c:/bind9_parser/examples# ./parse_bind9.py /data/production-data/namedb/named.conf
Start: Is the library quiet?
Traceback (most recent call last):
  File "./parse_bind9.py", line 268, in <module>
    result = my_clauses.parseString(toplevel_config, parseAll=True)
  File "/usr/local/lib/python3.7/site-packages/pyparsing.py", line 1955, in parseString
    raise exc
  File "/usr/local/lib/python3.7/site-packages/pyparsing.py", line 3814, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pyparsing.ParseException: Expected end of text, found '{'  (at char 862), (line:17, col:22)

Extract from my named.conf :

key "rndc-key" { algorithm hmac-sha256; secret "iQxBKDHuO62ECsjuWkrKzfC1FFmYF3C/5Wb8sR/MSZU="; };controls { inet 127.0.0.1 port 953 allow { 127.0.0.1; } keys { "rndc-key"; };}; logging { channel query_log  { file "/etc/namedb/log/named.log" versions 12 size 100m; print-time yes; };
category default { query_log; default_syslog; default_debug;};
category lame-servers { query_log; default_syslog; default_debug;};
category config { query_log; default_syslog; default_debug;};
category update { query_log; default_syslog; default_debug;};
category update-security { query_log; default_syslog; default_debug;};
category notify { query_log; default_syslog; default_debug;};
category queries { query_log; };
};



acl "ok" { any; };
acl "nok" { none; };
acl "auth-temp-pub" { 192.168.254.40; 192.168.254.41; };
acl "auth-temp-priv" { 192.168.3.171;  192.168.3.172; };
acl "ripe-hostcount" { 193.0.0.0/22; 91.121.158.151; 2001:610:240::/48; 2001:67c:2e8::/48; };

acl "bogon" {
// Filter out the bogon networks.  These are networks
// listed by IANA as test, RFC1918, Multicast, experi-
// mental, etc.  If you see DNS queries or updates with
// a source address within these networks, this is likely
// of malicious origin. CAUTION: If you are using RFC1918
// netblocks on your network, remove those netblocks from
// this list of blackhole ACLs!
0.0.0.0/8;
169.254.0.0/16;
192.0.0.0/24;
192.0.2.0/24;
198.18.0.0/15;
198.51.100.0/24;
203.0.113.0/24;
240.0.0.0/4;
//10.0.0.0/8;
//192.168.0.0/16;
//172.16.0.0/12;
};

options {
// Relative to the chroot directory, if any
directory       "/etc/namedb";
pid-file        "/var/run/named/pid";
dump-file       "/etc/namedb/log/named_dump.db";
statistics-file "/etc/namedb/log/named.stats";
memstatistics-file "/etc/namedb/log/named-mem.stats";
zone-statistics yes;
listen-on       { 127.0.0.1; 192.168.254.1; };
//      listen-on-v6    { ::1; };
version	"none";
interface-interval 0;
transfers-in 15;
transfers-out 15;
transfers-per-ns 15;
recursive-clients 20000;

This issue does not affect general use cases by end-users. It is a focus on helping end-user identifying their parse error scenarios ... related to AML syntaxes.

Whenever a syntax error occurs within the address_match_list (aka <aml>) syntax, it always reports the location of the text offset as the beginning of AML and not where the actual offending character occurs.

For example,

deny-answer-address { 127.0.0.1 }; is missing a semicolon after 127.0.0.1 but error code will report error offset location as 0 (zero), when it should be 32.

Until this is fixed, many development of new keywords using AML syntax will be long and audurously difficult to create those newly definitions of PyParsing logic/rule set ... that are re-using <aml> syntax.

It should be noted that <aml> is a recursive syntax.

The idea is to pass the content of the value to another statement. Closing this as a non-issue.

The python variable should not be encumbered with surrounding quotes of domain name, hostname, RR data, and pubkeys.

This is a massive undertaking.

@ptmcg

Here's the first cut, complete with the most comprehensive test cases I've got (because I absolutely needed it to ensure nothing got broken during development).

I use the JetBrain PyCharm IDE so the (.idea hidden dotfile should be there).

Enjoy.

Steve

FAILED (errors=2)
ERR_COUNT: 6
FAILED_MODULES: , test_clauses, test_clause_trusted_keys, test_clause_view, test_options, test_optviewzone, test_zone

Might be more useful if the unstated settings (of named.conf) has its default values and seeded in the final PyParsing object tree.

Missing a whole lot of unit test for IPv6, IPv6 with prefix, IPv6 with IPv6, IPv6 with network device name, et. al.

• 1:: 1:2:3:4:5:6:7::
• 1::8 1:2:3:4:5:6::8 1:2:3:4:5:6::8
• 1::7:8 1:2:3:4:5::7:8 1:2:3:4:5::8
• 1::6:7:8 1:2:3:4::6:7:8 1:2:3:4::8
• 1::5:6:7:8 1:2:3::5:6:7:8 1:2:3::8
• 1::4:5:6:7:8 1:2::4:5:6:7:8 1:2::8
• 1::3:4:5:6:7:8 1::3:4:5:6:7:8 1::8
• fe80::7:8%eth0 (link-local IPv6 addresses with zone index)
• fe80::7:8%1 (link-local IPv6 addresses with zone index)
• ::2:3:4:5:6:7:8 ::2:3:4:5:6:7:8 ::8 ::
• ::ffff:0:255.255.255.255 (IPv4-mapped IPv6 addresses and IPv4-translated addresses)
• ::ffff:255.255.255.255 (IPv4-mapped IPv6 addresses and IPv4-translated addresses)
• 2001:db8:3:4::192.0.2.33 64:ff9b::192.0.2.33 (IPv4-Embedded IPv6 Address)
• ::255.255.255.255 (IPv4-mapped IPv6 addresses and IPv4-translated addresses)

Hello! I'm working on a tool that runs alongside BIND and has a need for reading BIND configuration. I found this repo through the corresponding PyPI project page and it looks like it could work for what I need! However, it seems the PyPI project is a bit behind in terms of published versions available. Are there any plans to publish the newer releases to PyPI?

FAILED (errors=2)
ERR_COUNT: 6
FAILED_MODULES: , test_clauses, test_clause_trusted_keys, test_clause_view, test_options, test_optviewzone, test_zone

Commit c957ab6