DNS tunnel that can do DoH and DoT about bbs HOT 13 OPEN

This utility works during internet shutdown in Turkmenistan. It successfully establishes direct UDP connection to the destination server (without using any public resolver) and transfers up to 2 mbit/s of download.
Iodine is very unstable in these conditions, while dnstt is stable and fast.

from bbs.

For technical reasons, the tunnel requires the resolver to support a UDP payload size of at least 1232 bytes, which is bigger than the minimum of 512 guaranteed by DNS. I suspect that most public DoH or DoT servers meet this requirement, but I haven't done a survey or anything.

As of tag v0.20200426.0 in the source code, dnstt-server lets you control the maximum UDP payload size with the -mtu option. The default is -mtu 1232. You can use -mtu 512 for maximum compatibility with resolvers, at the expense of bandwidth. If you know the resolver you are using supports larger UDP payloads, you can increase the value. The Cloudflare resolver supports -mtu 1452, for example; but I don't recommend going higher than that because you start to risk IP fragmentation. If you use an -mtu value that is larger than what the resolver supports, the tunnel won't work at all and you will get error messages in the server log that tell you what value to use.

from bbs.

Download speed tests

I did some experiments of download performance of the DNS tunnel. tl;dr a DNS tunnel can go faster than you may think, but the choice of resolver matters a lot.

I tried downloading a 10 MB file through the tunnel, using a selection of resolvers and DNS transports. I cut off the download after 10 minutes. "none" is the special case of no intermediate recursive resolver (the tunnel client sends queries directly to the tunnel server). The server was located in Fremont, US and the client in Tokyo, JP. There was about 100 ms of latency between the two hosts. Download rates are the median of 5 trials. The dnstt tag was v0.20200430.0. See below for source code, data, pcaps, etc.

Cloudflare's DoH and DoT resolvers are both fast. Google's DoH resolvers is much faster than its DoT server (I noticed the DoT server terminating TCP connections every 200 KB or so). Comcast's DoH and DoT resolvers have about the same middling performance. Quad9's DoT resolver is notably slow; there's clearly something wrong there, whether it's the resolver or how the tunnel uses it. For comparison, the download rate of an untunneled, direct TCP transfer was 4666.3 KB/s.

I repeated the experiment with iodine, an existing DNS tunnel. iodine works over plaintext UDP only. dnstt is faster than iodine in every case, except for the Quad9 DoT resolver. It is possible to run iodine over a DoH proxy; I didn't try that myself but Sebastian Neef reports 4–12 KB/s when tunneling iodine through dnscrypt-proxy.

This graph shows the 5 trials under each experimental condition and gives an idea of the variance. Steeper lines are better.

The source code for these experiments is available in the following repo. I used git-annex to store the data files (there are over 3 GB of pcap files). You will have to git annex get the data files you want after git clone. The .csv files are sufficient to reproduce the graph. See procedure.txt for the commands to run to reproduce the experiment. .keylog files are TLS secrets in NSS Key Log Format; you can use these in Wireshark to decrypt the DoH and DoT streams.

git clone https://www.bamsoftware.com/git/dnstt-tests.git
cd dnstt-tests
git annex get 2020-04-30/*.csv
Rscript graphs.R

Also posted at https://www.bamsoftware.com/software/dnstt/performance.html.

Update 2020-05-05: I updated the tables and figure to exclude a preliminary test run that I did not intend to include in the first place. The change did not affect any of the qualitative observations. The Cloudflare/DoH case increased by about 7 KB/s from 126.7 KB/s to 133.5 KB/s; none of the other cases changed by more than 3 KB/s.

from bbs.

Web page

I've set up a web page for dnstt:
https://www.bamsoftware.com/software/dnstt/

And I wrote notes on the protocol:
https://www.bamsoftware.com/software/dnstt/protocol.html

from bbs.

Shadowsocks plugin (proof of concept)

It would not be hard to adapt the dnstt code for a Shadowsocks plugin. Here I show Bash scripts that wrap dnstt-client/dnstt-server in a Shadowsocks plugin interface:
https://gist.github.com/wkrp/0712b87ab095dd0f77c56b02646060b7

A more portable/permanent solution would be to fork the dnstt code and swap the command-line interface for a Shadowsocks plugin environment variable interface.

from bbs.

A more portable/permanent solution would be to fork the dnstt code and swap the command-line interface for a Shadowsocks plugin environment variable interface.

IMO fork is even unnecessary, we can add SIP003 support without breaking command line interface. Just check if SIP003 environment variable exist when start, if they exist, then dnstt is running as plugin, command line can be ignored, else it's running independent, read command line normally. Same for other tunnel program.

from bbs.

Performance tuning, v1.20210803.0

I just released v1.20210803.0 of dnstt.

• https://www.bamsoftware.com/software/dnstt/#download
• git clone https://www.bamsoftware.com/git/dnstt.git

The main feature of this release is some parameter tuning for a small improvement in performance in some configurations. See the full post.

I'm working on Champa, a circumvention tunnel based on AMP cache. Like dnstt, Champa uses a Turbo Tunnel model, with KCP and smux as an inner session layer. While working on Champa, I discovered that adjusting some buffer and window sizes could greatly improve download performance. I suggested that the same idea might improve performance in Snowflake, and I spent some time experimenting to see if it could help dnstt as well.

In summary, I was able to improve download speeds, but only in some configurations, and only a little bit. I was encouraged in initial tests with plaintext UDP and without a recursive resolver, which I was able to make go quite fast, even over 1 MB/s. But this is a configuration we don't care about, because it's not covert. In a recommended configuration with a recursive resolver and an encrypted transport, I was really only able to speed up Cloudflare/DoT, by about 25%.

I started by re-running the experiment with v0.20200430.0, the version used in the previous round of tests, in order to have a fresh basis of comparison. Since then, the Comcast/DoT server ceased operation, and Cloudflare/UDP went from one of the fastest configurations to the slowest. I repeated the experiment with v1.20210803.0, which has the performance tweaks.

The Google/DoT, Quad9/DoH, Quad9/UDP rows need some comment. In looking at the second-by-second download rates, we see that in 2 out of 3 trials, Google/DoT was initially going somewhat faster in the new version than in the old version, but then stalled and made no further progress. This was caused by a TCP disconnection (which itself is not unusual when using the Google DoT resolver) followed by a failure to reestablish the connection due to a name lookup error. This could be made more robust, but it does not really bear on bandwidth measurements. In the old Quad9/DoH and the new Quad9/UDP graphs, in 2 of the 3 trials there is a pattern of the download making progress, then stalling, then making progress, then stalling, and so on. I don't know what may be causing this phenomenon, except to guess that it may be rate limiting on a subset of backend server. In both cases, the 1 trial without the stop-and-start pattern has similar performance as in the corresponding graph.

As before, I've made the test code and raw data available, so you should be able to reproduce the table and graph, or run your own experiments. You will need git-annex to download a subset of the data files.

git clone https://www.bamsoftware.com/git/dnstt-tests.git
cd dnstt-tests/2021-08-02
git annex get data/*/*.csv
Rscript graphs.R

from bbs.

@wkrp Thanks for your excellent work on this. While testing I ran into a very annoying problem on the client side: if connectivity is lost when using DoH, the tunnel dies but does not exit:

sendLoop: Post "https://cloudflare-dns.com/dns-query": context deadline exceeded
sendLoop: Post "https://cloudflare-dns.com/dns-query": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
sendLoop: Post "https://cloudflare-dns.com/dns-query": net/http: request canceled (Client.Timeout exceeded while awaiting headers)

All subsequent connections through the tunnel hang. Tried Google and Cloudflare DoH.

The tunnel does recover when using DoT or UDP, so I'm assuming this is some limitation of DoH. Unfortunately DoT is very slow/unreliable for me, and UDP is "not recommended" for obvious reasons.

Is it possible to add some network resilience to dnstt-client? Just a graceful exit would be nice. Any plans to accept pull requests?

from bbs.

@alexandervlpl Thanks for testing and for the useful report.

My guess is that it's not the sendLoop errors that are the source of the failure to reconnect. That's expected when there's a lack of connectivity.

Offhand, I don't have a idea as to why DoT recovers and DoH does not. In both cases, the local session should time out all streams after idleTimeout (2 minutes) without receiving any data from the server.

My first guess about what might be going wrong was that the server is timing out the client's session during the loss of connectivity, while the client still thinks it has a session and continues uselessly sending packets that the server thinks are not associated with any existing session. But the server's idleTimeout is 2 minutes as well, so the client should have stopped trying by that time.

How long did you wait after restoring the network connection? Try waiting 5 minutes. That should be enough to exceed any possible timeout. (The idleTimeout is nominally 2 minutes, but it can actually be up to 4 − ε minutes because of how smux checks the timeout only periodically.) If 5 minutes works, then we'll know what's going wrong and can try adjusting some parameters; if not, we'll know to look elsewhere. I encountered some pathology with DoT reconnection during the most recent round of big performance tests, and I investigated it a bit and made a note:

In -dot mode, if, after the TLS connection may become disconnected, the redial fails to connect, it results in "operation on closed connection" errors and a useless connection up until idleTimeout (2 to 4 minutes later), when the stream ends.

There is a separate subforum for dnstt. That's currently the best place to send patches. You're also welcome to keep posting on this thread if that's more convenient.

https://ntc.party/c/community-software/dnstt/33

from bbs.

I tried waiting 5, 10 minutes and longer. It looks like dnstt-client keeps trying to reuse the old connection(s) forever:

handle: session e261cadc opening stream: io: read/write on closed pipe

On Linux, switching off my network connection for a minute (or even less) is enough to reproduce this.

My workaround is a little bash function watchdog that restarts dnstt-client when it logs "Client.Timeout" or "closed pipe". That seems to work reliably, I'm currently using this every day on all my devices for small amounts of traffic.

Happy to post instructions and results on the forum when they're ready.

from bbs.

@wkrp
@ValdikSS

I wonder if there is a simple guide for non-specialist people in order to implement DNSTT?
And I also wonder if DNSTT needs any specific client on Windows and Android OS?

from bbs.

@alidxdydz on Android you can install the Termux app from Play or F-Droid and follow the same Linux instructions, this worked very well for me. Since this is Go it should compile and "just work" on Windows as well, can anyone confirm? There are no nice client apps I'm afraid.

from bbs.

And I also wonder if DNSTT needs any specific client on Windows and Android OS?

Unfortunately there is no ready-to-use official mobile client app. If you search the Play Store for "dnstt" you will see some VPN apps come up. I cannot comment on their quality or trustworthiness, but I reverse engineered one once and found that they were using the upstream dnstt code, without any changes, such that an unmodified client could connect to the VPN (if you added in the extra layer of SSH authentication they were using).

from bbs.