handshake-org / hnsd Goto Github PK

Handshake SPV name resolver

License: Other

Makefile 0.54% M4 0.50% C 96.63% Shell 0.19% Batchfile 0.26% Python 0.79% JavaScript 0.57% Assembly 0.51% Dockerfile 0.01%

hnsd's People

Contributors

Stargazers

Watchers

hnsd's Issues

Persistent Headers

We should be saving the last 1152 headers to disk. Flat file storage can work for this. Every hour or so, we can flush the current state to a single file and read it again on boot. It's a couple hundred kb to write every hour, which shouldn't be a big deal.

Having this ready by v1 is ideal, but realistically, this only needs to be ready a couple months after the chain is launched. Headers are fast to sync. It will only become a pain later on.

Is it compatible with the latest HNS protocol?

⚠️ Currently, hnsd is not compatible with the latest, Handshake p2p network protocol. We are exploring a potential rewrite of the entire codebase. Please watch this repo for development updates.

Has this been fixed yet?

Inconsistent response around ttl expiry

With hnsd running locally: dig @127.0.0.1 -t txt f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu

The domain has one TXT record with a TTL of 5 seconds.

The dig command usually produces a result with that TXT record in the answer section. Occasionally, as the ttl expires, it produces a result with no answers and just an authority section.

I've copied hnsd logs for two consecutive executions of that dig command, the first succeeding, the second failing:

rs: query
rs:   id=15530
rs:   labels=4
rs:   name=f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu.
rs:   type=16
rs:   class=1
rs:   edns=1
rs:   dnssec=0
rs:   tld=johnxu
rs:   addr=127.0.0.1:56465
rs_worker: request 191: f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu.
rs: received answer for: f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu.
rs:   rcode: 0
rs:   havedata: 1
rs:   nxdomain: 0
rs:   secure: 0
rs:   bogus: 1
rs:   why_bogus: validation failure <f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu. TXT IN>: key for validation johnxu. is marked as invalid because of a previous validation failure <f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu. TXT IN>: no DNSSEC records from 44.231.6.183 for DS johnxu. while building chain of trust
rs: query
rs:   id=10958
rs:   labels=4
rs:   name=f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu.
rs:   type=16
rs:   class=1
rs:   edns=1
rs:   dnssec=0
rs:   tld=johnxu
rs:   addr=127.0.0.1:40859
rs_worker: request 192: f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu.
rs: received answer for: f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu.
rs:   rcode: 3
rs:   havedata: 0
rs:   nxdomain: 1
rs:   secure: 0
rs:   bogus: 1
rs:   why_bogus: validation failure <f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu. TXT IN>: key for validation johnxu. is marked as invalid because of a previous validation failure <f44ca263-6f26-410d-a08a-ac3bb0f7c480._auth.jee.johnxu. TXT IN>: no DNSSEC records from 44.231.6.183 for DS johnxu. while building chain of trust

Please provide clear installation instructions for Linux

Seeds provided in command line not preferred by addrmgr.

Right now when you add seeds via -s or --seeds the addresses provided are dumped into address manager without any kind of priority. There is no guarantee that they will be tried and in fact after getting an addr message from a peer with 2000 addrs, its very likely the seeds provided by the user are never picked.

*.co domains do not resolve, dig times out.

List of TLDs that don't resolve as of now (not complete):

co (t.co, go.co)
horse (https://www.xrays.horse/examinations/)
cloud (https://console.hetzner.cloud/)

Domains under the co TLD do not resolve. Ex: t.co, go.co

Built from master branch, updated tld.h generated with hs-names. Still no luck.

This is what happens in hnsd:

Request comes in:

rs: query
rs:   id=600
rs:   labels=2
rs:   name=go.co.
rs:   type=1
rs:   class=1
rs:   edns=1
rs:   dnssec=0
rs:   tld=co
rs:   addr=127.0.0.1:58007

No response, dig times out
After 30-60 secs, receive answer in hnsd:

rs: received answer for: go.co.
rs:   rcode: 0
rs:   havedata: 1
rs:   nxdomain: 0
rs:   secure: 0
rs:   bogus: 1
rs:   why_bogus: validation failure <go.co. A IN>: key for validation co. is marked as invalid because of a previous validation failure <t.co. A IN>: No DNSKEY record for key co. while building chain of trust

Next query with dig to the same domain works (gets IP, fast, mostly from cache)

Question: Where is the data folder?

I am using hnsd as docker container and everytime I restart the container, the blocks are downloaded again.

I wonder where it downloads the data of blocks and if it is possible to bind it as volume?

Thanks.

config file is not being parsed

The config file as set by -c, --config <config> doesn't look like it's being parsed. I couldn't find the corresponding code either.

In case this is TODO, we can consider dropping the config file option altogether and just use environment variables to provide the command line parameters (as in #18 via systemd directive EnvironmentFile)

Can this forward legacy TLD requests to a particular DNS server?

I run my own DNS server for legacy TLD access, which includes some features I want (including access to some other alt roots and also ad blocking) -- what is the best way to mix in access to handshake names as well?

Internal public resolve

I'm working on a proof of concept: running hnsd on a Raspberry Pi Zero W.
Ok, it's running.

Could I use this Pi as DNS for the other devices on local network?

I'm able to resolve names from inside the Pi but not from other devices.

I tried to run hnsd with a different variety of arguments/values but it never seems to be LISTENing:

sudo netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      352/sshd: /usr/sbin
tcp6       0      0 :::22                   :::*                    LISTEN      352/sshd: /usr/sbin
udp        0      0 0.0.0.0:5353            0.0.0.0:*                           244/avahi-daemon: r
udp        0      0 0.0.0.0:32795           0.0.0.0:*                           244/avahi-daemon: r
udp        0      0 0.0.0.0:68              0.0.0.0:*                           500/dhcpcd
udp6       0      0 :::5353                 :::*                                244/avahi-daemon: r
udp6       0      0 :::546                  :::*                                500/dhcpcd
udp6       0      0 :::56254                :::*                                244/avahi-daemon: r

How to allow hnsd to receive and respond to external DNS queries?

Dynamically allocate rrsets

RRSets currently use far too much memory due to preallocation of pointers. This may lead to issues with cache size in the future. We should dynamically allocate them ldns-style.

hnsd should ignore ipv6 addresses if network is not available

I don't have ipv6 on my machine, the query for icann.org wastes so much time on this, that it times out. If I request again, the ipv4 will have been cached (by then) and respond immediately.

ns: query
ns:   id=39901
ns:   labels=0
ns:   name=.
ns:   type=48
ns:   class=1
ns:   edns=1
ns:   dnssec=1
ns:   tld=
ns:   addr=127.0.0.1:20683
ns: sending root soa (39901): 280
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:8f::53 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:8d::53 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:8f::53 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:8f::53 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:89::53 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:1a::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:c::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:1b::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:1a::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:1c::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:1b::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:1c::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:e::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:c::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:b::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:b::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:c::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:c::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:b::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:f::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:b::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:e::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:c::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:e::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:b::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:c::1 port 53
[1615899353] libunbound[69307:0] error: udp connect failed: No route to host for 2001:500:e::1 port 53

Short of public full nodes?

With the growing amount of apps which uses hnsd it seems we need more public full nodes to get the data from. I run one myself with max-inbound=500 and it comes to this limit soon.

I'm not sure if I understand this correctly but if I say ideally an spv node should have 4 full nodes to connect with, then the sum of max-inbound of all public full nodes should be 1/4 of all running spv nodes out there, right?

Docker build fails at RUN ./autogen.sh

Attempting to build on Ubuntu, I received the following error:

ERROR [build 2/2] RUN ./autogen.sh && ./configure && make 0.3s

[build 2/2] RUN ./autogen.sh && ./configure && make:                                                                                      
#0 0.288 /bin/sh: ./autogen.sh: not found                                                                                                                                                                                                                                       
Dockerfile:12

10 |       libtool \
11 |       unbound-dev
12 | >>> RUN ./autogen.sh && ./configure && make
13 |     
14 |     FROM base

error: failed to solve: process "/bin/sh -c ./autogen.sh && ./configure && make" did not complete successfully: exit code: 127

Failed to set capabilities

when i run this command :

setcap 'cap_net_bind_service=+ep' /hnsd

i have this message :

Failed to set capabilities on file `/hnsd' (No such file or directory)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file

types should always be 16 bits

There's several uint8_t type around the code we need to fix... what if someone tries to pass in a URI record?! (type 256)

unknown command: 6

This message can be safely ignored.

What's happening is hnsd is connecting to hsd full nodes, which are being very nice and offering to send unconfirmed transaction messages (INV) they don't realize hnsd does not have a mempool and so these messages are just ignored.

We can probably fix this on either hnsd or hsd by enabling a blocks-only mode (Bitcoin Core has this)

The actual number 6 comes from the hsd protocol message list:

/**
 * Packet types.
 * @enum {Number}
 * @default
 */

exports.types = {
  VERSION: 0,
  VERACK: 1,
  PING: 2,
  PONG: 3,
  GETADDR: 4,
  ADDR: 5,
  INV: 6,
  GETDATA: 7,
  NOTFOUND: 8,
  GETBLOCKS: 9,
  GETHEADERS: 10,
  HEADERS: 11,
  SENDHEADERS: 12,
  BLOCK: 13,
  TX: 14,
  REJECT: 15,
  MEMPOOL: 16,
  FILTERLOAD: 17,
  FILTERADD: 18,
  FILTERCLEAR: 19,
  MERKLEBLOCK: 20,
  FEEFILTER: 21,
  SENDCMPCT: 22,
  CMPCTBLOCK: 23,
  GETBLOCKTXN: 24,
  BLOCKTXN: 25,
  GETPROOF: 26,
  PROOF: 27,
  CLAIM: 28,
  AIRDROP: 29,
  UNKNOWN: 30,
  // Internal
  INTERNAL: 31,
  DATA: 32
};

By contrast, the hnsd protocol message list is much shorter:

hnsd/src/msg.h

Lines 13 to 24 in 1b47849

 #define HSK_MSG_VERSION 0 

 #define HSK_MSG_VERACK 1 

 #define HSK_MSG_PING 2 

 #define HSK_MSG_PONG 3 

 #define HSK_MSG_GETADDR 4 

 #define HSK_MSG_ADDR 5 

 #define HSK_MSG_GETHEADERS 10 

 #define HSK_MSG_HEADERS 11 

 #define HSK_MSG_SENDHEADERS 12 

 #define HSK_MSG_GETPROOF 26 

 #define HSK_MSG_PROOF 27 

 #define HSK_MSG_UNKNOWN 255

Add a root server message cache

Although the recursive server caches our responses (which means we're not hitting the p2p network every single second), the root server still has to translate the resource data to a DNS message, create DNSSEC signatures (slow), and add a SIG0 record (slow, but not as slow as DNSSEC sigs).

A simple cache for the root server can mitigate this problem a lot. This is actually a very strange situation since, in traditional DNS, authoritative servers don't tend to need caches. We're different in that we don't have a static zone file (our zone file is a blockchain).

Adding this to the todo.

Proposal: Create an OpenWrt package for hnsd

"Currently OpenWrt is used as a base for about 10% to 30% of all home routers and CPE device sold world wide." [source]

Therefore, there's a tremendous opportunity to expand the Handshake community by integrating with OpenWrt.

According to some of the OpenWrt guys, the easiest integration route would be to convert hnsd into a package:

https://openwrt.org/docs/guide-developer/packages

That would allow existing OpenWrt users to install the package (and use hnsd), and it would allow us to create custom firmware images that have the package preinstalled (we could then talk to router manufacturers about using those images and ultimately deploy Handshake-enabled devices to the general public).

Mainnet

There's a lot to review:

#29 #30 #31 #32

If you're feeling crazy and just want to compile the whole mess and get started using hnsd on mainnet, you can attempt to deploy my combined branch: https://github.com/pinheadmz/hnsd/tree/maintest1

I'm running it right now as my computer's only resolver! It's holding up great, and I'm able to browse to all the dope HNS websites users have developed so far ;-)

Update ICANN root zone file tld.h

This looks like something we can do with each release if necessary.

Was just brought to my attention that esurance is not in the root any more.

Proposal: Build Instructions for *BSD

HNSD does not include build instructions for FreeBSD and other BSD variants. Although MacOS is Unix-like (precisely Darwin) like the other BSD variants, its build instructions would not work for other BSD variants. Most handshake projects and subprojects include MacOS build instructions but not other BSD variants. It is now being proposed that future updates on these projects should include the missing build instructions.

Test Framework

It would be ideal to add unit tests to hnsd so that its easier to develop without regressions. There are many options for test frameworks in C, many projects opt to roll their own framework. I found what I believe to be good options.

Options

cmocka
- Used by bind, libssh
- Custom assertions
- Simple to use, create an array of CMUnitTests using cmocka_unit_test which wrap describe like functions
check
- Designed to work with autotools
- Simple, but a lot of boilerplate
CGreen
- Large, has many features
- TDD/BDD support, along with Before, After, etc macros
Custom
- No extra dependencies
- As simple as it needs to be
- Must implement it, must be bugfree

I'd personally opt for using cmocka since it looks simple to use and is also used by other applications that are similar (DNS + security). It would be easier than creating our own custom test library.

Malformed packet returned when requested type does not exist

Using hnsd as my DNS resolver, I found that it will return a malformed packet when there are no RR of the requested type. Synced with testnet4.

Testing with easyhandshake, it is also possible to see the name here: https://hnscan.com/name/easyhandshake

$ hsd-cli rpc getnameresource

{
  "version": 0,
  "compat": false,
  "ttl": 64,
  "hosts": [
    "165.22.151.242"
  ],
  "text": [
    "Run a Handshake SPV node in your web browser!"
  ]
}

Sending a DNS request to hnsd for an A record, which will be synthesized from the INET4 address in the hosts list:

$ dig +dnssec easyhandshake A

; <<>> DiG 9.14.5 <<>> @127.0.0.1 -p 15360 +dnssec easyhandshake A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65007
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;easyhandshake.			IN	A

;; ANSWER SECTION:
easyhandshake.		64	IN	A	165.22.151.242
easyhandshake.		10800	IN	RRSIG	A 13 1 64 20191023003544 20190925003544 60944 . yUqi5xY28TVlYCxUKDHz5piMPDMUMLdZsnkPUoeVwKs+MurfG47No/nI 45d8/BABqmkSaiKWvfvPj60OtqMuKw==

;; Query time: 0 msec
;; SERVER: 127.0.0.1#15360(127.0.0.1)
;; WHEN: Tue Oct 08 17:39:24 PDT 2019
;; MSG SIZE  rcvd: 153

Corresponding hnsd logs:

rs: query
rs:   id=40258
rs:   labels=1
rs:   name=easyhandshake.
rs:   type=1
rs:   class=1
rs:   edns=1
rs:   dnssec=1
rs:   tld=easyhandshake
rs:   addr=127.0.0.1:39171
rs: received answer for: easyhandshake.
rs:   rcode: 0
rs:   havedata: 1
rs:   nxdomain: 0
rs:   secure: 1
rs:   bogus: 0

Now sending a request for a AAAA record:

$ dig +dnssec easyhandshake AAAA

;; Got bad packet: FORMERR
277 bytes
1a 3c 81 a0 00 01 00 00 00 04 00 01 0d 65 61 73          .<...........eas
79 68 61 6e 64 73 68 61 6b 65 00 00 1c 00 01 c0          yhandshake......
0c 00 2f 00 01 00 01 50 d3 00 01 00 c0 0c 00 2e          ../....P........
00 01 00 00 29 83 00 53 00 2f 0d 01 00 01 51 80          ....)..S./....Q.
5d af a0 65 5d 8a b6 65 ee 10 00 09 d1 c3 0a d7          ]..e]..e........
d6 cc d4 e2 4f 75 75 87 7e 62 63 a4 18 38 0e 18          ....Ouu..bc..8..
69 0c 01 d7 34 c5 b0 86 95 e7 9a 23 66 eb 55 28          i...4......#f.U(
c1 f9 a3 c3 7e db 1a 7b 3a 4f 24 60 c6 df 02 7c          .......{:O$`...|
c1 9c 97 33 f6 ca f9 93 3d af b5 00 00 06 00 01          ...3....=.......
00 00 0d 63 00 16 00 00 78 59 08 e4 00 00 07 08          ...c....xY......
00 00 03 84 00 09 3a 80 00 01 51 80 00 00 2e 00          ......:...Q.....
01 00 00 29 83 00 53 00 06 0d 00 00 01 51 80 5d          ...)..S......Q.]
af a0 65 5d 8a b6 65 ee 10 00 5b c9 53 63 d7 ad          ..e]..e...[.Sc..
46 df 6c 74 b5 4a 39 31 25 e3 8e a2 ac 80 47 43          F.lt.J91%.....GC
08 e6 28 d5 41 f7 b4 8b 12 ac 32 7a 5c 9b e9 dd          ..(.A.....2z\...
d7 7b 12 39 1c 40 ac 83 c0 23 00 08 5e 23 90 02          .{.9.@...#..^#..
3c 2d cd 48 18 e7 75 bc 5c 49 00 00 29 10 00 00          <-.H..u.\I..)...
00 80 00 00 00                                           .....

Corresponding hnsd logs:

rs: query
rs:   id=15234
rs:   labels=1
rs:   name=easyhandshake.
rs:   type=28
rs:   class=1
rs:   edns=1
rs:   dnssec=1
rs:   tld=easyhandshake
rs:   addr=127.0.0.1:60003
ns: query
ns:   id=54357
ns:   labels=1
ns:   name=easyhandshake.
ns:   type=1
ns:   class=1
ns:   edns=1
ns:   dnssec=1
ns:   tld=easyhandshake
ns:   addr=127.0.0.1:42670
cache: cache hit for: easyhandshake.
ns: sending cached msg (54357): 153
ns: query
ns:   id=12191
ns:   labels=1
ns:   name=easyhandshake.
ns:   type=28
ns:   class=1
ns:   edns=1
ns:   dnssec=1
ns:   tld=easyhandshake
ns:   addr=127.0.0.1:62400
cache: cache hit for: easyhandshake.
ns: sending cached msg (12191): 277
rs: received answer for: easyhandshake.
rs:   rcode: 0
rs:   havedata: 0
rs:   nxdomain: 0
rs:   secure: 1
rs:   bogus: 0

Sending a request for a TXT record returns the data in Resource.text correctly. Querying for a NS record causes it to return a malformed response.

Fails build on MacOS 10.15.4

When following the build process on my mac it stops on an error:

configure: error: could not determine ar interface

Full output here:

+ pushd uv
~/hnsd/uv ~/hnsd
+ test -f configure
+ ./autogen.sh
+ glibtoolize --copy
glibtoolize: putting auxiliary files in '.'.
glibtoolize: copying file './ltmain.sh'
glibtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
glibtoolize: copying file 'm4/libtool.m4'
glibtoolize: copying file 'm4/ltoptions.m4'
glibtoolize: copying file 'm4/ltsugar.m4'
glibtoolize: copying file 'm4/ltversion.m4'
glibtoolize: copying file 'm4/lt~obsolete.m4'
+ aclocal -I m4
+ autoconf
+ automake --add-missing --copy
configure.ac:38: installing './ar-lib'
configure.ac:25: installing './compile'
configure.ac:22: installing './config.guess'
configure.ac:22: installing './config.sub'
configure.ac:21: installing './install-sh'
configure.ac:21: installing './missing'
Makefile.am: installing './depcomp'
+ popd
~/hnsd
+ glibtoolize --copy
glibtoolize: putting auxiliary files in '.'.
glibtoolize: copying file './ltmain.sh'
glibtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
glibtoolize: copying file 'm4/libtool.m4'
glibtoolize: copying file 'm4/ltoptions.m4'
glibtoolize: copying file 'm4/ltsugar.m4'
glibtoolize: copying file 'm4/ltversion.m4'
glibtoolize: copying file 'm4/lt~obsolete.m4'
+ aclocal -I m4
+ autoconf
+ automake --add-missing --copy
configure.ac:12: installing './ar-lib'
configure.ac:10: installing './compile'
configure.ac:6: installing './config.guess'
configure.ac:6: installing './config.sub'
configure.ac:4: installing './install-sh'
configure.ac:4: installing './missing'
Makefile.am: installing './depcomp'
checking for a BSD-compatible install... /usr/local/bin/ginstall -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/local/bin/gmkdir -p
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking build system type... x86_64-apple-darwin19.4.0
checking host system type... x86_64-apple-darwin19.4.0
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of g++... gcc3
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... unknown
configure: error: could not determine ar interface

Can't connect to peers

Not sure if did something wrong but after following instructions this is not working:

$ ./hnsd -p 4 -r 127.0.0.1:5333
chain (0): chain is fully synced
addrman: added addr: 172.104.214.189:13038
addrman: added addr: 173.255.209.126:13038
addrman: added addr: 172.104.177.177:13038
addrman: added addr: 139.162.183.168:13038
pool: pool opened (size=4)
ns: root nameserver listening on: 127.0.0.1:15359
rs: recursive nameserver pointing to: 127.0.0.1@15359
rs: recursive nameserver listening on: 127.0.0.1:5333
starting event loop...
peer 3 (173.255.209.126:13038): failed connecting: connection refused
peer 3 (173.255.209.126:13038): closing peer
peer 3 (173.255.209.126:13038): closed peer

OS: Linux Debian 10

Suffix user agent with slash

Not a big deal, but hnsd user agents don't end with a slash. This was added in hnsd (#81) last year (before hsd handshake-org/hsd#710 a few months ago) and hence the difference.

Taken from a node peer list:

{
  "/hnsd:1.0.0/": 474,
  "/hnsd:1.0.0/beacon": 2,
  "/hsd:4.0.0/": 4,
  "/hsd:4.0.1/bob-wallet:1.0.0/": 1,
  "/hsd:4.0.2/": 7,
}

I think this needs to change?

hnsd/src/pool.c

Lines 305 to 312 in 856ee22

 size_t len = strlen(pool->user_agent); 

 len += strlen(user_agent); 

 // Agent size in p2p version message is 1 byte 

 if (len > 0xff) 

 return false; 

 pool->user_agent = strcat(pool->user_agent, user_agent);

hnsd on iOS

Hi there -- I'd like to build a handshake dns resolver on iOS. I am wondering if someone here could guide me in the right direction:
(1) are there instructions somewhere for building this (or a library version of hnsd) for an iOS target?
(2) is there a function that takes an input query (in DNS-wire format) and returns a response (also in DNS-wire format)?

Thanks!

DNSSEC seems broken when using custom forwarding

Hi,

My setup is the following:
unbound <-> hnsd <-> pi-hole <- queries

Sample of unbound.conf used in HNSD:

server:
  trust-anchor-file: "/usr/share/dnssec-root/trusted-key.key"
  harden-dnssec-stripped: yes
  harden-short-bufsize: yes
  harden-below-nxdomain: yes
  harden-referral-path: yes
  hide-version: yes
  prefetch-key: yes
  use-caps-for-id: no
forward-zone:
  name: "COM."
  forward-addr: unbound@xxxx

Using "cloudflare.com" as example.

When Pi-hole validate the DNSSEC answer it tries to resolve cloudflare.com then com then .
Everything is going fine until Pi-hole reaches .

Here is dig answer from Pi-hole querying HNSD:

# dig @x.x.x.x +dnssec . -p xxxx

; <<>> DiG 9.11.5-P4-5.1+deb10u2-Debian <<>> @x.x.x.x +dnssec . -p xxxx
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5348
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;.                              IN      A

;; AUTHORITY SECTION:
.                       84068   IN      NSEC    . NS SOA RRSIG NSEC DNSKEY
.                       8468    IN      RRSIG   NSEC 13 0 86400 20210227010230 20210130010230 60944 . 70/mMGxapZuWAsA80+iWYwrtWTv7O6Y4DRSP5CySZbsIJ11mKR95XaMj c/+GGJHVWx4DxKQXraq7wQiDe/7AcQ==
.                       1268    IN      SOA     . . 2021021301 1800 900 604800 86400
.                       8468    IN      RRSIG   SOA 13 0 86400 20210227010230 20210130010230 60944 . oAwRaQGfb6K7Tcp5M76QsCJaZY7HjOUTFIqrgmjRkD2Qy3QCLJ5LO5KH x+60BFpJhdW+QbpwaPufLZ+DXqNOuA==

;; Query time: 0 msec
;; WHEN: Sat Feb 13 02:41:22 CET 2021
;; MSG SIZE  rcvd: 270

Here is the dig answer from Pi-hole when querying unbound;

# dig @x.x.x.x +dnssec . -p xxxx

; <<>> DiG 9.11.5-P4-5.1+deb10u2-Debian <<>> @x.x.x.x +dnssec . -p xxxx
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45590
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1472
;; QUESTION SECTION:
;.                              IN      A

;; AUTHORITY SECTION:
.                       3600    IN      SOA     a.root-servers.net. nstld.verisign-grs.com. 2021021201 1800 900 604800 86400
.                       86400   IN      RRSIG   SOA 8 0 86400 20210225170000 20210212160000 42351 . kSNTcz4yre3DZ3Vs85f5sKD7ZkwCfak1bqazvRTpWuaYYyKDiJLKwrRO HqNfWFzTpAMMCHdovaLaOVCg6z4XrIMM0/+lSsA88DMwjp+cL+XJlusy Ga83MurWb66fkIErC41cIuU1NJysCfxEozTid+WeuXvkw/WARdWzwHBA Z9oCxoSWWEgbEExMUqKxjff24M0D3422G6jCXEckkdNm7iMXKmBJQFVe o2bhlUy9El76nhAE4WQ91c+biyvQa7ge499eva5a2uGa9Ssl/HISBhZ5 pDIlTgL/oyPHRF5nGAFEqDOZCKjasCv8bHyh+Hp8fluO2sHyzlNEhueC 2e8Iaw==
.                       79300   IN      NSEC    aaa. NS SOA RRSIG NSEC DNSKEY
.                       79300   IN      RRSIG   NSEC 8 0 86400 20210225170000 20210212160000 42351 . PHM3hQ4kOUiscAfg7OBXBpgYQyLf/Waq6F1gSt33sy/n1V+gcgoneaP4 68Rsf0nFna2JH5EXMYfMoCnkIZsMVX21eOioVs5U7pwrjEim1Zuc7gQ3 bFWy80k+kY9PbPYRMxzNC6+fW2oXNsOXPmMxSrQgCcEXi4udX3hYNemg TRCNreEvkArKIVAgFzj8sD4pEmIrgWHVWIs9VZv3U8Gz0c6VJPrFHRcX lqXIZHyODlAW2AqS6B7X0EH0rGl6lVhrb6GVChgpbqSikPR33Hga6wbC OoePXHWh4QQvqGX2WDPEqfIBzA7tl1IgMWlZ1v/nfkIY91Emb7+BLLcH 0clyVA==

;; Query time: 10 msec
;; WHEN: Sat Feb 13 02:41:36 CET 2021
;; MSG SIZE  rcvd: 700

Hope that'll help.

Proposal for --upstream flag

This issue is a proposal for a new flag which would control the behavior of hnsd for queries that were not resolvable by handshake.

The current behavior of hnsd is to attempt to resolve a domain using handshake, and then use a recursive resolver as a fallback to handle ICANN domains.

Motivations:

The recursive resolver fails if local DNS traffic (all outbound udp port 53) is redirected to a specific DNS server. This could be the case on public wifi.
The user may want to use a specific DNS server for ICANN domains
The network may not have internet access (using handshake in an intranet or off-grid setting) and may need to configure a local upstream DNS for local queries

Alternate solutions:

#46 suggests using unbound config, which unfortunately seems to take ultimate precedence and redirect everything. Using that method would require maintaining a list of TLDs
The behavior of the unbound config could be changed to only apply to queries destined for ICANN, at the cost of any current use case that relies on the unbound config

Proposal:
Adding an --upstream flag to change the destination of queries that failed to be resolved by handshake.

--upstream recursive the current would be default
--upstream x.x.x.x would use the specified DNS server instead of the built in recursive resolver
--upstream 2001::0 should accept ipv6 as well

This would avoid interfering with any existing unbound config use case and also not require maintaining a dump of ICANN TLDs while making it easier to handle different network environments.

Unable to build on Apple M-series chips

I installed the prerequisites but got this message at the end of the build command:

configure: error: Unable to find libunbound

Full output:

./autogen.sh && ./configure && make
+ pushd uv
~/Development/hnsd-2.0.0/uv ~/Development/hnsd-2.0.0
+ test -f configure
+ popd
~/Development/hnsd-2.0.0
+ glibtoolize --copy
+ aclocal -I m4
+ autoconf
configure.ac:26: warning: The macro 'AC_PROG_LIBTOOL' is obsolete.
configure.ac:26: You should run autoupdate.
m4/libtool.m4:100: AC_PROG_LIBTOOL is expanded from...
configure.ac:26: the top level
configure.ac:122: warning: The macro 'AC_PROG_CC_C89' is obsolete.
configure.ac:122: You should run autoupdate.
./lib/autoconf/c.m4:1655: AC_PROG_CC_C89 is expanded from...
configure.ac:122: the top level
+ autoheader
+ automake --add-missing --copy
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... mkdir -p
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking build system type... aarch64-apple-darwin23.3.0
checking host system type... aarch64-apple-darwin23.3.0
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C++... yes
checking whether g++ accepts -g... yes
checking for g++ option to enable C++11 features... -std=gnu++11
checking whether make supports the include directive... yes (GNU style)
checking dependency style of g++ -std=gnu++11... gcc3
checking for gcc... gcc
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /Library/Developer/CommandLineTools/usr/bin/ld
checking if the linker (/Library/Developer/CommandLineTools/usr/bin/ld) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 786432
checking how to convert aarch64-apple-darwin23.3.0 file names to aarch64-apple-darwin23.3.0 format... func_convert_file_noop
checking how to convert aarch64-apple-darwin23.3.0 file names to toolchain format... func_convert_file_noop
checking for /Library/Developer/CommandLineTools/usr/bin/ld option to reload object files... -r
checking for file... file
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for archiver @FILE support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for lipo... lipo
checking for otool... otool
checking for otool64... no
checking for -single_module linker flag... ld: warning: -single_module is obsolete
no
checking for -exported_symbols_list linker flag... yes
checking for -force_load linker flag... yes
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... yes
checking for gcc option to produce PIC... -fno-common -DPIC
checking if gcc PIC flag -fno-common -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin23.3.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -std=gnu++11 -E
checking for ld used by g++ -std=gnu++11... /Library/Developer/CommandLineTools/usr/bin/ld
checking if the linker (/Library/Developer/CommandLineTools/usr/bin/ld) is GNU ld... no
checking whether the g++ -std=gnu++11 linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking for g++ -std=gnu++11 option to produce PIC... -fno-common -DPIC
checking if g++ -std=gnu++11 PIC flag -fno-common -DPIC works... yes
checking if g++ -std=gnu++11 static flag -static works... no
checking if g++ -std=gnu++11 supports -c -o file.o... yes
checking if g++ -std=gnu++11 supports -c -o file.o... (cached) yes
checking whether the g++ -std=gnu++11 linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin23.3.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether make supports nested variables... (cached) yes
checking whether byte ordering is bigendian... no
checking for _Bool... yes
checking for stdbool.h that conforms to C99 or later... yes
checking for inline... inline
checking for size_t... yes
checking for uint8_t... yes
checking for uint16_t... yes
checking for uint32_t... yes
checking for uint64_t... yes
checking for int8_t... yes
checking for int16_t... yes
checking for int32_t... yes
checking for int64_t... yes
checking for GNU libc compatible malloc... yes
checking for GNU libc compatible realloc... yes
checking for gcc option to enable large file support... none needed
checking for memset... yes
checking for strcasecmp... yes
checking for strdup... yes
checking for strchr... yes
checking for getrandom... no
checking for arc4random... yes
checking for random... yes
checking for arpa/inet.h... yes
checking for fcntl.h... yes
checking for limits.h... yes
checking for netinet/in.h... yes
checking for stddef.h... yes
checking for stdint.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for strings.h... (cached) yes
checking for sys/random.h... yes
checking for unistd.h... (cached) yes
checking how to run the C preprocessor... gcc -E
checking for gcc... gcc
checking whether the compiler supports GNU C... (cached) yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking how to run the C preprocessor... gcc -E
checking dependency style of gcc... gcc3
checking for __int128... yes
checking for __builtin_expect... yes
checking for x86_64 assembly availability... no
checking for ub_ctx_create in -lunbound... no
configure: error: Unable to find libunbound

My Intel Mac had no problem building. Of course. SIGH

Get build working on windows

I don't even know where to start with this one. Hoping someone with more windows knowledge than me can help out.

libunbound error: can't create socket: Too many open files

After running on master + #38 for several hours. I was watching memory and I never saw it break 25/26MB.

log message (several hundred of these:)

 libunbound[12624:0] error: can't create socket: Too many open files

Implement DNS label compression

We should really be compressing DNS labels, as there's no telling what people may put in the root zone. To make things worse, we don't currently listen on TCP (since it's a lot of crazy micromanagement in the code). If a response is greater than 4096 bytes, it will not fit in the UDP send buffer.

We switched away from LDNS, which did have compression code included in it, but for some reason not exposed in the public API (perhaps they're still testing it?).

On the bright side, we switched from using RSA sigs to ECDSA sigs for DNSSEC which does save a lot of space.

As for the recursive server, I suspect unbound is returning the raw wire data to us in a compressed form. The only problem: we need to reserialize everything to change the ID, remove DNSSEC data (which helps with the size), and change EDNS parameters. If we could do this in a trickier way, we may not even need a compression implementation.

The JS implementation can currently do compression, but it's really non-trivial and I can't imagine what a pain it would be to do in C.

Failed to set capabilities on file

When attempting the following segment of a Linux install:
sudo setcap 'cap_net_bind_service=+ep' /path/to/hnsd

The responding error is delivered:
Failed to set capabilities on file ` /path/to/hnsd' (Invalid argument)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file

It could be that not all linux kernals (I'm running 4.4.0-57-generic) support setcap https://serverfault.com/a/874053

Memory leak

Running master branch at 95da9e3

Command: sudo valgrind -s --track-origins=yes --leak-check=full ./hnsd -r 127.0.0.1:53
Run for a few hours, normal system DNS along with two threads of this

valgrind output:

==262390== 
==262390== HEAP SUMMARY:
==262390==     in use at exit: 4,111,477 bytes in 16,868 blocks
==262390==   total heap usage: 3,638,404 allocs, 3,621,536 frees, 1,501,481,665 bytes allocated
==262390== 
==262390== 192 bytes in 3 blocks are definitely lost in loss record 5 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x4C3759D: __gmp_default_allocate (in /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0)
==262390==    by 0x4BC9FE7: _nettle_gmp_alloc_limbs (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x4BCD9B8: nettle_ecc_point_init (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x48D3591: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D3A6F: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D5099: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D5E77: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D6565: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D7576: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D7B97: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48C81BC: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390== 
==262390== 192 bytes in 3 blocks are definitely lost in loss record 6 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x4C3759D: __gmp_default_allocate (in /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0)
==262390==    by 0x4BC9FE7: _nettle_gmp_alloc_limbs (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x4BCD9B8: nettle_ecc_point_init (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x48D3591: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D3A6F: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D5099: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D60A3: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D62C9: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D6E6B: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D70BD: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48C9FAE: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390== 
==262390== 1,536 bytes in 24 blocks are definitely lost in loss record 10 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x4C3759D: __gmp_default_allocate (in /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0)
==262390==    by 0x4BC9FE7: _nettle_gmp_alloc_limbs (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x4BCD9B8: nettle_ecc_point_init (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x48D3591: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D3A6F: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D5099: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D60A3: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D62C9: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D6E6B: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D70BD: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48C9898: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390== 
==262390== 1,600 bytes in 25 blocks are definitely lost in loss record 11 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x4C3759D: __gmp_default_allocate (in /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0)
==262390==    by 0x4BC9FE7: _nettle_gmp_alloc_limbs (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x4BCD9B8: nettle_ecc_point_init (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x48D3591: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D3A6F: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D5099: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D60A3: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D62C9: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D6E6B: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D70BD: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D1CDC: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390== 
==262390== 1,640 bytes in 1 blocks are definitely lost in loss record 12 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x134A49: hsk_peer_init (pool.c:804)
==262390==    by 0x134A49: hsk_peer_alloc (pool.c:877)
==262390==    by 0x134A49: hsk_pool_refill.part.0 (pool.c:399)
==262390==    by 0x135457: hsk_pool_refill (pool.c:337)
==262390==    by 0x135457: hsk_pool_open (pool.c:334)
==262390==    by 0x121166: hsk_daemon_open (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390==    by 0x11FECE: main (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390== 
==262390== 1,856 bytes in 29 blocks are definitely lost in loss record 13 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x4C3759D: __gmp_default_allocate (in /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0)
==262390==    by 0x4BC9FE7: _nettle_gmp_alloc_limbs (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x4BCD9B8: nettle_ecc_point_init (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x48D3591: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D3A6F: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D5099: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D60A3: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D62C9: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D6E6B: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D70BD: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D21FB: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390== 
==262390== 2,752 bytes in 43 blocks are definitely lost in loss record 14 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x4C3759D: __gmp_default_allocate (in /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0)
==262390==    by 0x4BC9FE7: _nettle_gmp_alloc_limbs (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x4BCD9B8: nettle_ecc_point_init (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x48D3591: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D3A6F: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D5099: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D5E77: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D6565: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D72D8: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D7998: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48C803E: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390== 
==262390== 5,696 bytes in 89 blocks are definitely lost in loss record 15 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x4C3759D: __gmp_default_allocate (in /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0)
==262390==    by 0x4BC9FE7: _nettle_gmp_alloc_limbs (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x4BCD9B8: nettle_ecc_point_init (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x48D3591: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D3A6F: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D5099: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D60A3: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D62C9: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D6E6B: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D70BD: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48C7A29: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390== 
==262390== 13,120 bytes in 8 blocks are definitely lost in loss record 16 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x134A49: hsk_peer_init (pool.c:804)
==262390==    by 0x134A49: hsk_peer_alloc (pool.c:877)
==262390==    by 0x134A49: hsk_pool_refill.part.0 (pool.c:399)
==262390==    by 0x134F07: hsk_pool_refill (pool.c:770)
==262390==    by 0x134F07: hsk_pool_timer (pool.c:783)
==262390==    by 0x134F07: after_timer (pool.c:1847)
==262390==    by 0x164F24: uv__run_timers (timer.c:165)
==262390==    by 0x15C589: uv_run (core.c:359)
==262390==    by 0x11FEEF: main (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390== 
==262390== 16,320 bytes in 255 blocks are definitely lost in loss record 17 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x4C3759D: __gmp_default_allocate (in /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0)
==262390==    by 0x4BC9FE7: _nettle_gmp_alloc_limbs (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x4BCD9B8: nettle_ecc_point_init (in /usr/lib/x86_64-linux-gnu/libhogweed.so.5.0)
==262390==    by 0x48D3591: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D3A6F: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D5099: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D60A3: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D62C9: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D6E6B: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48D70BD: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390==    by 0x48C9B78: ??? (in /usr/lib/x86_64-linux-gnu/libunbound.so.8.1.4)
==262390== 
==262390== 73,597 bytes in 294 blocks are definitely lost in loss record 18 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x12B315: hsk_dns_msg_encode (dns.c:178)
==262390==    by 0x120CC3: hsk_cache_insert (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390==    by 0x1222E8: after_resolve (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390==    by 0x133FEB: hsk_peer_handle_proof (pool.c:1525)
==262390==    by 0x133FEB: hsk_peer_handle_msg (pool.c:1584)
==262390==    by 0x133FEB: hsk_peer_parse (pool.c:1696)
==262390==    by 0x133FEB: hsk_peer_on_read.part.0 (pool.c:1606)
==262390==    by 0x134563: hsk_peer_on_read (pool.c:1595)
==262390==    by 0x134563: after_read (pool.c:1818)
==262390==    by 0x161B80: uv__read (stream.c:1249)
==262390==    by 0x162527: uv__stream_io (stream.c:1316)
==262390==    by 0x1672F7: uv__io_poll (linux-core.c:400)
==262390==    by 0x15C61B: uv_run (core.c:368)
==262390==    by 0x11FEEF: main (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390== 
==262390== 334,592 bytes in 1,307 blocks are definitely lost in loss record 19 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x12876F: hsk_dns_txt_alloc (dns.c:2075)
==262390==    by 0x137B73: hsk_txt_record_read (resource.c:144)
==262390==    by 0x137F27: hsk_record_read (resource.c:376)
==262390==    by 0x1380BB: hsk_resource_decode (resource.c:436)
==262390==    by 0x1221C0: after_resolve (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390==    by 0x133FEB: hsk_peer_handle_proof (pool.c:1525)
==262390==    by 0x133FEB: hsk_peer_handle_msg (pool.c:1584)
==262390==    by 0x133FEB: hsk_peer_parse (pool.c:1696)
==262390==    by 0x133FEB: hsk_peer_on_read.part.0 (pool.c:1606)
==262390==    by 0x134563: hsk_peer_on_read (pool.c:1595)
==262390==    by 0x134563: after_read (pool.c:1818)
==262390==    by 0x161B80: uv__read (stream.c:1249)
==262390==    by 0x162527: uv__stream_io (stream.c:1316)
==262390==    by 0x1672F7: uv__io_poll (linux-core.c:400)
==262390==    by 0x15C61B: uv_run (core.c:368)
==262390== 
==262390== 1,029,312 (1,029,056 direct, 256 indirect) bytes in 4,513 blocks are definitely lost in loss record 20 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x12B315: hsk_dns_msg_encode (dns.c:178)
==262390==    by 0x120CC3: hsk_cache_insert (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390==    by 0x1222E8: after_resolve (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390==    by 0x133FEB: hsk_peer_handle_proof (pool.c:1525)
==262390==    by 0x133FEB: hsk_peer_handle_msg (pool.c:1584)
==262390==    by 0x133FEB: hsk_peer_parse (pool.c:1696)
==262390==    by 0x133FEB: hsk_peer_on_read.part.0 (pool.c:1606)
==262390==    by 0x1534AC: hsk_brontide_parse (brontide.c:776)
==262390==    by 0x153709: hsk_brontide_on_read (brontide.c:629)
==262390==    by 0x13453B: after_read (pool.c:1807)
==262390==    by 0x161B80: uv__read (stream.c:1249)
==262390==    by 0x162527: uv__stream_io (stream.c:1316)
==262390==    by 0x1672F7: uv__io_poll (linux-core.c:400)
==262390==    by 0x15C61B: uv_run (core.c:368)
==262390== 
==262390== 2,628,352 bytes in 10,267 blocks are definitely lost in loss record 21 of 21
==262390==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==262390==    by 0x12876F: hsk_dns_txt_alloc (dns.c:2075)
==262390==    by 0x137B73: hsk_txt_record_read (resource.c:144)
==262390==    by 0x137F27: hsk_record_read (resource.c:376)
==262390==    by 0x1380BB: hsk_resource_decode (resource.c:436)
==262390==    by 0x1221C0: after_resolve (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390==    by 0x133FEB: hsk_peer_handle_proof (pool.c:1525)
==262390==    by 0x133FEB: hsk_peer_handle_msg (pool.c:1584)
==262390==    by 0x133FEB: hsk_peer_parse (pool.c:1696)
==262390==    by 0x133FEB: hsk_peer_on_read.part.0 (pool.c:1606)
==262390==    by 0x1534AC: hsk_brontide_parse (brontide.c:776)
==262390==    by 0x153709: hsk_brontide_on_read (brontide.c:629)
==262390==    by 0x13453B: after_read (pool.c:1807)
==262390==    by 0x161B80: uv__read (stream.c:1249)
==262390==    by 0x162527: uv__stream_io (stream.c:1316)
==262390== 
==262390== LEAK SUMMARY:
==262390==    definitely lost: 4,110,501 bytes in 16,861 blocks
==262390==    indirectly lost: 256 bytes in 1 blocks
==262390==      possibly lost: 0 bytes in 0 blocks
==262390==    still reachable: 720 bytes in 6 blocks
==262390==         suppressed: 0 bytes in 0 blocks
==262390== Reachable blocks (those to which a pointer was found) are not shown.
==262390== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==262390== 
==262390== ERROR SUMMARY: 28 errors from 16 contexts (suppressed: 0 from 0)
==262390== 
==262390== 7 errors in context 1 of 16:
==262390== Syscall param socketcall.connect(serv_addr.sin6_scope_id) points to uninitialised byte(s)
==262390==    at 0x496253B: connect (connect.c:26)
==262390==    by 0x163CE3: uv__tcp_connect (tcp.c:229)
==262390==    by 0x134900: hsk_peer_open (pool.c:927)
==262390==    by 0x134900: hsk_pool_refill.part.0 (pool.c:408)
==262390==    by 0x134F07: hsk_pool_refill (pool.c:770)
==262390==    by 0x134F07: hsk_pool_timer (pool.c:783)
==262390==    by 0x134F07: after_timer (pool.c:1847)
==262390==    by 0x164F24: uv__run_timers (timer.c:165)
==262390==    by 0x15C589: uv_run (core.c:359)
==262390==    by 0x11FEEF: main (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390==  Address 0x1fff000008 is on thread 1's stack
==262390==  in frame #2, created by hsk_pool_refill.part.0 (pool.c:385)
==262390==  Uninitialised value was created by a stack allocation
==262390==    at 0x1345D6: hsk_pool_refill.part.0 (pool.c:385)
==262390== 
==262390== 
==262390== 7 errors in context 2 of 16:
==262390== Syscall param socketcall.connect(serv_addr.sin6_flowinfo) points to uninitialised byte(s)
==262390==    at 0x496253B: connect (connect.c:26)
==262390==    by 0x163CE3: uv__tcp_connect (tcp.c:229)
==262390==    by 0x134900: hsk_peer_open (pool.c:927)
==262390==    by 0x134900: hsk_pool_refill.part.0 (pool.c:408)
==262390==    by 0x134F07: hsk_pool_refill (pool.c:770)
==262390==    by 0x134F07: hsk_pool_timer (pool.c:783)
==262390==    by 0x134F07: after_timer (pool.c:1847)
==262390==    by 0x164F24: uv__run_timers (timer.c:165)
==262390==    by 0x15C589: uv_run (core.c:359)
==262390==    by 0x11FEEF: main (in /home/pinheadmz/Desktop/work/hnsd/hnsd)
==262390==  Address 0x1ffefffff4 is on thread 1's stack
==262390==  in frame #2, created by hsk_pool_refill.part.0 (pool.c:385)
==262390==  Uninitialised value was created by a stack allocation
==262390==    at 0x1345E1: hsk_pool_refill.part.0 (pool.c:385)
==262390== 
==262390== ERROR SUMMARY: 28 errors from 16 contexts (suppressed: 0 from 0)

Clearnet connections leak privacy: allow extra settings to limit hnsd to encrypted connections only

Before #38 is merged, hnsd only connects to a curated list of 8 seed nodes. This is a centralization issue. However, all 8 of those connections are encrypted.

With #38 we can now discover peers by requesting addr messages from the network, but these messages only contain clearnet addresses (no public key for brontide encryption). In hsd, we disabled gossiping brontide addresses (see handshake-org/hsd#315 and handshake-org/hsd@a333a24) but we may potentially be able to add them back, or use connection upgrades in the future (handshake-org/hsd#487). Once that is figured out in hsd we can circle back here and add the upgrade protocol to hnsd, and try to keep everything encrypted. For now, #38 compromises privacy for decentrazliation.

Clearnet connections leak privacy because they rely on getproof messages from a light client to a full node server which contain the unencrypted nameHash of the TLD with which to query the root zone. Note that if the TLD is not in the HNS root zone, a proof of non-inclusion is returned and the light client attempts to resolve the query using legacy DNS. But even in this case, let's say we are requesting google.com, the TLD com is sent in the clear to the hsd full node -- well, it's hash.

So we've made the tradeoff in #38 that decentralization is more important than privacy. hnsd will connect to multiple peers instead of just the seed nodes, but many of these peer connections will not be encrypted, meaning that network snoops can see what domains you are looking up. Note that with encrypted connections, snoops won't know what you are looking up, but the node operator on the other end will. So, privacy for hnsd is not great already, same as Bitcoin SPV.

Before we figure out connection upgrades in hsd there are a few things we can add to hnsd:

restrict connections only to peers added with --seeds (see #39)
allow configuring hnsd to only connect to encrypted peers, which at this point is just the centralized seed nodes

Arm segfault (strlen)

(gdb) r -p 4 -r 127.0.0.1:5300
Starting program: /home/pi/hnsd/hnsd -p 4 -r 127.0.0.1:5300
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
chain (0): chain is fully synced
addrman: added addr: 172.104.214.189:13038
addrman: added addr: 173.255.209.126:13038
addrman: added addr: 172.104.177.177:13038
addrman: added addr: 139.162.183.168:13038
pool: pool opened (size=4)
ns: root nameserver listening on: 127.0.0.1:15359
rs: recursive nameserver pointing to: 127.0.0.1@15359
rs: recursive nameserver listening on: 127.0.0.1:5300
starting event loop...

Program received signal SIGSEGV, Segmentation fault.
strlen () at ../sysdeps/arm/armv6/strlen.S:26
26	../sysdeps/arm/armv6/strlen.S: No such file or directory.
(gdb) bt
#0  strlen () at ../sysdeps/arm/armv6/strlen.S:26
#1  0x76d97690 in _IO_vfprintf_internal (s=0x76e91d50 <_IO_2_1_stdout_>, format=0xb4c58 "peer %lu (%s): ", format@entry=0x7effb110 "\004", ap=..., ap@entry=...) at vfprintf.c:1637
#2  0x76d9cae0 in __printf (format=0xb4c58 "peer %lu (%s): ") at printf.c:33
#3  0x00022f64 in hsk_peer_log (peer=peer@entry=0xf93a8, fmt=0xb4d68 "connected\n") at src/pool.c:949
#4  0x0002361c in on_connect (conn=<optimized out>, status=0) at src/pool.c:1662
#5  0x0005a840 in uv__stream_connect (stream=0xf93b4) at src/unix/stream.c:1391
#6  uv__stream_io (loop=<optimized out>, w=0xf93f8, events=<optimized out>) at src/unix/stream.c:1308
#7  0x0005ec00 in uv__io_poll (loop=loop@entry=0xda610 <default_loop_struct>, timeout=2502) at src/unix/linux-core.c:400
#8  0x0005559c in uv_run (loop=0xda610 <default_loop_struct>, mode=UV_RUN_DEFAULT) at src/unix/core.c:368
#9  0x00013b64 in main ()

Heap corruption during authoritative resolution

Spotted this today when clicking an obfuscated twitter link. The daemon had been running locally on my laptop for a few days.

rs: query
rs:   id=27745
rs:   labels=2
rs:   name=t.co.
rs:   type=1
rs:   class=1
rs:   edns=0
rs:   dnssec=0
rs:   tld=co
rs:   addr=127.0.0.1:51557
rs: udp nodata
ns: query
ns:   id=14389
ns:   labels=2
ns:   name=t.co.
ns:   type=1
ns:   class=1
ns:   edns=1
ns:   dnssec=1
ns:   tld=co
ns:   addr=127.0.0.1:52340
ns: udp nodata
corrupted double-linked list (not small)
Aborted

I've been unable to reproduce it, so no chance of using valgrind to track this down. The ns: udp nodata log implies that hsk_ns_onrecv() successfully executed otherwise I would suspect this of being an issue with the authoritative cache that was just added. It's also not sending a message in response, so there's no cache hit there.

My best guess is something funky happened in the P2P pool which corrupted the heap. I'm starting to add more debug logs so we can narrow this down when it happens again.

Use dynamic root zone fallback

hnsd uses a hardcoded ICANN root zone. From the whitepaper:

Hardcoded fallback involves hard-coding the existing ICANN zonefile into the
SPV software. When an absence proof is received by the authoritative server for
a pre-existing TLD, the hardcoded fallback is checked for records and returned
as a DNS response.

This doesn't work because it assumes the ICANN root zone never changes. We had to update the zone file in the past to fix the .org TLD already I believe. The .org TLD is now broken again. hnsd serves the following .org DS records:

org.                    20416   IN      DS      17883 7 1 38C5CF93B369C7557E0515FAAA57060F1BFB12C1
org.                    20416   IN      DS      17883 7 2 D889CAD790F01979E860D6627B58F85AB554E0E491FE06515F35548D 1EB4E6EE

but .org now has the following:

org.                    76777   IN      DS      26974 8 2 4FEDE294C53F438A158C41D39489CD78A86BEB0D8A0AEAFF14745C0D 16E1DE32

This issue will become more apparent once the DNSSEC PR is merged. The .org TLD will turn bogus. hnsd should use a dynamic fallback like hsd perhaps with the names reserved like in this issue. The root zone DOES change frequently.

ICANN doesn't even recommend keeping their root keys hardcoded and recommends some automated mechanisms like RFC5011 to keep the root keys up to date. Hardcoding the entire root zone is much more problematic.

Here's how I'm testing hnsd right now

@buffrr told me about this tool: https://www.dns-oarc.net/tools/dnsperf which seems like it might be overkill, but it includes a file with TEN MILLION DNS QUERIES: https://www.dns-oarc.net/files/dnsperf/data/queryfile-example-10million-201202.gz

I have a short bash script in a neighboring file that randomly makes queries from this list, I run it simultaneously in a couple of tmux panes to get multithreaded bombardment of hnsd:

t=0.1

while true
do
	dig @127.0.0.1 -p 5350 $(shuf -n 1 ./queryfile-example-10million-201202.txt)
	sleep $t
done

I also made a list of HNS root zone names extracted from dns.live's json data and run a thread like this but checking HNS names as well.

Using this to beat on #38

Access class HS from docker image

It seems like the class HS added in #102 can't be access when running on docker:

Docker log:

ns: ignoring non-local HS class request

Which comes from

hnsd/src/ns.c

Lines 339 to 342 in 42aa53d

 if (!hsk_addr_is_local(&address)) { 

 hsk_ns_log(ns, "ignoring non-local HS class request\n"); 

 goto done; 

 }

It would be great to have this feature on docker as well, so it could be used for readiness and liveness when using Kubernetes.

Error on startup

With hskd up and running on defaults $ ./bin/hskd

I start hns with
$ ./hnsd --pool-size=1 --rs-host=127.0.0.1:53

and get

chain (0): chain is fully synced

addrman: added addr: 127.0.0.1:15038

pool: pool opened (size=1)

ns: root nameserver listening on: 127.0.0.1:35359

rs: recursive nameserver pointing to: 127.0.0.1@35359

failed opening rns: EFAILURE

peer 0 (127.0.0.1:15038): closing peer

Add man pages

Uncaught Exception while receiving invalid headers from a node.

While Running the master (c4d0a9f) branch on a device today, I encountered this error:

chain (15998): adding block: 00000000000001620090cea3b412d19512989cadfc13643d424fc5608cdb4034
chain (15999):   added to main chain
chain (15999):   new height: 15999
chain (15999): adding block: 000000000000041a9be251ab14c46e664e7ded73253449167e631b101c1883aa
chain (16000):   added to main chain
chain (16000):   new height: 16000
peer 0 (139.162.183.168:44806): requesting more headers
peer 0 (139.162.183.168:44806): sending getheaders
pool: size: 2 active: 1
peer 3 (165.22.151.242:44806): connected
[libsecp256k1] illegal argument: !hsk_secp256k1_fe_is_zero(&ge->x)
Aborted

Which exited the process. I was able to reproduce it on the subsequent attempts where the crash happened when connecting to this node 165.22.151.242:44806.

Forwarding to hnsd from knot-resolver

I am running like this:

hnsd -r 127.0.0.1:5353

And then trying to setup knot-resolver in the obvious way:

policy.add(policy.all(policy.FORWARD({ '127.0.0.1@5353' })))

This results in knot being unable to resolve any name. I see no logs coming out of hnsd, and when I turn on debug logs in knot-resolver it says connection refused. Dig to 5353 directly works fine, but for some reason knot cannot connect. Any guesses what could cause a connection refused like that?

Optimize header chain sync using minimal disk space

hnsd currently downloads the entire header chain every time it is launched. While this takes less than one minute currently, it is still an unexpected delay for web-browsing users. #61 attempts to speed this up by storing all downloaded headers on disk, but hnsd still has to verify each of these from the genesis block to tip on every boot. This also means hnsd will require unlimited disk space as time goes on, consuming an additional ~12MB per year (headers are 236 bytes * 144 per day * 365 per year).

One thing that is an important difference between hnsd and the Bitcoin SPV model is that a Handshake light name resolver does not need to know every block header in the chain -- we only need to know the current Urkel Tree root! This means for the purpose of verifying Urkel proofs we only need to store one header: the chain tip.

However we obviously can't just throw away the entire rest of the header chain -- what if there is a chain reorg? If a reorg happens either during operation or while hnsd is shutdown and then restarted, we need to be able to verify the competing chain from the new tip down to wherever it connects to some other header we've already verified. This means for the purpose of following the best chain, we need store some amount of historical block headers to stay in sync with the network.

Solutions:

Checkpoints

There is an open PR to add checkpoints to hsd. It makes sense to copy these points over to hnsd. At the very least, we don't need to start sync from genesis every boot, we can just start from the highest checkpoint.

Chain Locator

Here's where it gets interesting. In Bitcoin and Handshake we already have this data structure called a chain locator. It is a list of some hashes in our best chain that we send to peers when we request blocks. Those peers will respond with blocks in their best chain that connect to one of the hashes in the locator. Usually those new blocks connect directly to our chain tip but sometimes they may connect to an earlier hash if there has been a reorg recently.

I grabbed a chain locator from a fully synced bcoin node starting up and syncing to peers. These are the chain heights in the locator:

[
  697216, 697215, 697214, 697213,
  697212, 697211, 697210, 697209,
  697208, 697207, 697206, 697205,
  697203, 697199, 697191, 697175,
  697143, 697079, 696951, 696695,
  696183, 695159, 693111, 689015,
  680823, 664439, 631671, 566135,
  435063, 172919,      0
]

31 hashes. Note the list is the last 10 blocks, then skip 2, then skip 4, then 8, 16... down to genesis. And this is a locator from a 10-year old chain.

So here's my proposal: hnsd stores enough headers on disk to put together a chain locator, that should be all it needs at all times, after any amount of down time, to reconnect to the network.

Implicitly declaring library function memcpy... on MacOS 10.15.7

I'm tried to run gpk rebuild rebuild from the /build of the bcrypto because bcrypto.node is missing in the build directory. Here is the error I'm getting.

Related to #28

Rust rewrite?

Hi I was curious if the Handshake org ever considered rebuilding this in Rust? Performance wise, they're virtually identical, with C only be slightly faster due to lack of bounds checking. Rust however has many safety guarantees that C doesn't have: all compiled safe Rust code is guaranteed to be memory & thread safe, and devoid of seg faults. Given this emphasis correctness & security, it seems like a more suitable language choice. Rust also makes it easier to optimize concurrency, often making it viable to build concurrent solutions that are more performant than a C/C++ counterpart lacking concurrency optimizations (Read up on the story of Stylo/Firefox Quantum). Has the team considered making a Rust version of Handshake?

	#define HSK_MSG_VERSION 0
	#define HSK_MSG_VERACK 1
	#define HSK_MSG_PING 2
	#define HSK_MSG_PONG 3
	#define HSK_MSG_GETADDR 4
	#define HSK_MSG_ADDR 5
	#define HSK_MSG_GETHEADERS 10
	#define HSK_MSG_HEADERS 11
	#define HSK_MSG_SENDHEADERS 12
	#define HSK_MSG_GETPROOF 26
	#define HSK_MSG_PROOF 27
	#define HSK_MSG_UNKNOWN 255

	size_t len = strlen(pool->user_agent);
	len += strlen(user_agent);

	// Agent size in p2p version message is 1 byte
	if (len > 0xff)
	return false;

	pool->user_agent = strcat(pool->user_agent, user_agent);

	if (!hsk_addr_is_local(&address)) {
	hsk_ns_log(ns, "ignoring non-local HS class request\n");
	goto done;
	}

handshake-org / hnsd Goto Github PK

hnsd's People

Contributors

Stargazers

Watchers

Forkers

hnsd's Issues

Options

Recommend Projects

Recommend Topics

Recommend Org