Giter Club home page Giter Club logo

shovel's Introduction

shovel

Make a lot of DNS requests and count the results! Useful for testing complex dynamic DNS records.

Pass multiple qnames, nameservers, record types, and client subnets, either via command line flags, a config, or a combo of both. shovel will dig all combinations of those and show you the results.

Use

Also see examples.md

With different client subnets

./demo.gif

With different record types

This uses the same config as the above gif. No subnets passed, so that column is excluded from the output.

$ shovel dig --qname linkedin.com --rtype A --rtype AAAA
╭──────────────┬───────┬──────────────────┬─────────────────┬───────╮
│ QNAME         │ RTYPE │ NAMESERVER       │ ANS/ERR         │ COUNT │
├──────────────┼───────┼──────────────────┼─────────────────┼───────┤
│ linkedin.com │ A     │ # ns1            │ 13.107.42.14    │    10 │
│              │       │ 198.51.45.9:53   │                 │       │
│              │       ├──────────────────┼─────────────────┼───────┤
│              │       │ # dyn            │ 13.107.42.14    │    10 │
│              │       │ 108.59.161.43:53 │                 │       │
│              ├───────┼──────────────────┼─────────────────┼───────┤
│              │ AAAA  │ # ns1            │ 2620:1ec:21::14 │    10 │
│              │       │ 198.51.45.9:53   │                 │       │
│              │       ├──────────────────┼─────────────────┼───────┤
│              │       │ # dyn            │ 2620:1ec:21::14 │    10 │
│              │       │ 108.59.161.43:53 │                 │       │
╰──────────────┴───────┴──────────────────┴─────────────────┴───────╯

Install

scoop bucket add bbkane https://github.com/bbkane/scoop-bucket
scoop install bbkane/shovel
  • Download Mac/Linux/Windows executable: GitHub releases
  • Go: go install go.bbkane.com/shovel@latest
  • Build with goreleaser after cloning: goreleaser --snapshot --skip-publish --clean

Notes

See Go Developer Tooling for notes on development tooling.

Run the webapp locally with OpenObserve

Export env vars:

export SHOVEL_SERVE_OPENOBSERVE_PASS='...';
export SHOVEL_SERVE_OPENOBSERVE_USER='...';
export ZO_ROOT_USER_EMAIL='...';
export ZO_ROOT_USER_PASSWORD='...';

Run OpenObserve (in another terminal) after downloading:

./openobserve

Open OpenObserve at: http://localhost:5080/web/traces?period=15m&query=&org_identifier=default

Run shovel. Check go run . serve --help to see all flags available. Also see [format_jsonl.py]https://github.com/bbkane/dotfiles/blob/master/bin_common/bin_common/format_jsonl.py)

go run . serve | format_jsonl.py fmt

Open shovel at: http://127.0.0.1:8080/?count=1&nameservers=dns3.p09.nsone.net%3A53&protocol=udp&qnames=linkedin.com+www.linkedin.com&rtypes=A&subnetMap=&subnets=

Install shovel + OpenObserve as systemd services, on a local dev VM or production VM with shovel_ansible

Dev Notes

shovel's People

Contributors

bbkane avatar bkane-msft avatar dependabot[bot] avatar

Stargazers

Julian Derungs avatar Lisa Hagemann avatar Tom Hummel avatar Jarrian Gojar avatar  avatar  avatar

Watchers

 avatar  avatar

shovel's Issues

Update warg

including:

  • breaking changes
  • tests with the new context
  • updating the serve command

add link to trace url, rm origin flag

Once I confirm that I can use "request.Host" as an origin flag, hardcore that and remove the flag

Add a flag that lets me specify a traceid url as an html template so I can inject that rendered template into submit and click on a link to see the http trace instead of copy-pasting the traceid into the portal manually

Support IPv4 only switch

This should be easy and allow easier DNS proxying

Code from DNS package to Exchange Context:

// ExchangeContext performs a synchronous UDP query, like Exchange. It
// additionally obeys deadlines from the passed Context.
func ExchangeContext(ctx context.Context, m *Msg, a string) (r *Msg, err error) {
	client := Client{Net: "udp"}
	r, _, err = client.ExchangeContext(ctx, m, a)
	// ignoring rtt to leave the original ExchangeContext API unchanged, but
	// this function will go away
	return r, err
}

Then I can set that to any of the options in https://github.com/miekg/exdns/blob/d851fa434ad51cb84500b3e18b8aa7d3bead2c51/q/q.go#L161

Send library logs to server logger

$ go run . serve | format_jsonl.py fmt

---
line: ⇨ http server started on 127.0.0.1:8080
2023/12/30 14:11:39 traces export: Post "http://127.0.0.1:5080/api/default/traces": dial tcp 127.0.0.1:5080: connect: connection refused

That second line comes from the otel libraries logging using the log library. I need to redirect that to the server JSON logs, exactly like https://caddyserver.com/docs/json/logging/sink/ . I could look up how that works or how https://pkg.go.dev/golang.org/x/exp/slog#SetDefault works

Re-org embedded directory

embedded directory

right now its:

static/
    3p/
        ...
    templates/
        ...
    index.css
    loading-spinner.svg

What if I did instead:

embedded/
    templates/
        ...
    static/
        3p/
            ...
        index.css
        loading-spinner.svg

Then I could use an embedded FS for serving instead of manually listing paths... Something to think about if the app grows

serve updates

  • server hardening middleware
  • traces
  • metadata- traceid, date, version
  • gh login
  • playwright tests
  • dashboards
    • req count
    • dns query count

Ensure panics end up in otel traces

The easiest way to trigger this is by trying to render a page with an invalid template. The code panics and I need to make sure the panic ends up in the trace and the user gets a 5xx

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.