haveibeensquatted / twistrs Goto Github PK

View Code? Open in Web Editor NEW

90.0 3.0 5.0 49.58 MB

A domain name permutation and enumeration library powered by Rust.

Home Page: https://haveibeensquatted.com/

License: MIT License

Rust 100.00%

security security-tools rust dns-lookup permutation typosquatting

twistrs's People

Contributors

Stargazers

Watchers

Forkers

sd2k rushmorem bonedaddy tompscanlan firaenix

twistrs's Issues

Infinite permutation generation on three-dot domains

When generating permutations for domains such as ox.ac.uk, we are entering into a run-off state where the number of permutations keeps cycling -- which is a bug.

Punycode domains are not being generated as expected

If we generate permutations for cool.com, we should expect to find çool.com (i.e., https://xn--ool-1la.com/) in the list of domains.

Current tracing into the internals of the library is non-existent, making it very difficult to understand what is happening internally when using the library. This should change and include some instrumentation to help with this.

Permutation library does not apply IDNA filtering

Currently the permutation library does not perform any sort of IDNA domain filtering which hinders performance, particularly for lookups. This method is implemented by dnstwist and should be included into all permutation methods.

DNS lookup yielding poor results for linkedin.com

Description

Currently when performing DNS resolution for linkedin.com, twistrs is yielding 158 domains, whereas dnstwist.py is yielding 178.

Steps to reproduce

dnstwist

python3 dnstwist.py linkedin.com --registered

twistrs-cli

cd examples/twistrs-cli
cargo r -- linkedin.com

Expected behavior

twistrs yields as many, or more, domains as dnstwist (currently 178).

Actual behavior

twistrs yields 158 domains behind dnstwist's 158 domains.

Implement GeoIP lookup Enrichment

Using the rust-geopip library (or something similar), use the cached GeoIP lookup table to perform lookups on resolved IPs.

Note that message signature may vary as IPs need to pushed as opposed to FQDNs.

Add example using WebSocket client implementation

Enhance the current example suite by including a new client implementation using WebSockets.

For .cz TLD, there is missing valid typosquat attack using .cy TLD

For .cz TLDs there are widely used .cy TLDs as another type of typosquat attack.

Because people here in CZ very often make a mistake to type "y" instead of "z" (Czech keyboard has switched position of these two letters in comparison to ENG keyboard.)

For example: seznam.cz (legit) vs seznam.cy (fake)

Consider deprecating or removing the enrichment module entirely

Currently the library is a mix between async and non-async modules. The permutation engine is trivial to use and integrate, making it pleasant to work with. The enrichment module however locks into a specified implementation which is (a) inflexible and (b) a pain to maintain.

After some thoughts, it seems that enrichment does not necessarily belong in the library at all, besides perhaps some traits to help library users extend the functionality to their liking.

Errors on build main branch

Wanted to import and run

    let domain = Domain::new("google.com").unwrap();
    let permutations = domain.all().unwrap();

Getting errors at bottom.

❯ cargo --version
cargo 1.72.0 (103a7ff2e 2023-08-15)

great-squat on  main [!] is 📦 v0.1.0 via 🦀 v1.72.0 on ☁️  (us-east-1) 
❯ rustc --version
rustc 1.72.0 (5680fa18f 2023-08-23)

errors:

❯ cargo build
   Compiling twistrs v0.5.2 (https://github.com/haveibeensquatted/twistrs.git?branch=main#5ebc1d17)
error[E0277]: the trait bound `String: Serialize` is not satisfied
    --> /home/tscanlan/.cargo/git/checkouts/twistrs-ab64395ec7e7a9bf/5ebc1d1/twistrs/src/enrich.rs:90:24
     |
90   | #[derive(Debug, Clone, Serialize, Default)]
     |                        ^^^^^^^^^ the trait `Serialize` is not implemented for `String`
91   | pub struct DomainMetadata {
92   |     /// The domain that is being enriched.
     |     -------------------------------------- required by a bound introduced by this call
     |
     = help: the trait `Serialize` is implemented for `str`
note: required by a bound in `enrich::_::_serde::ser::SerializeStruct::serialize_field`
    --> /home/tscanlan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.188/src/ser/mod.rs:1865:12
     |
1859 |     fn serialize_field<T: ?Sized>(
     |        --------------- required by a bound in this associated function
...
1865 |         T: Serialize;
     |            ^^^^^^^^^ required by this bound in `SerializeStruct::serialize_field`

error[E0277]: the trait bound `Vec<std::net::IpAddr>: Serialize` is not satisfied
    --> /home/tscanlan/.cargo/git/checkouts/twistrs-ab64395ec7e7a9bf/5ebc1d1/twistrs/src/enrich.rs:90:24
     |
90   | #[derive(Debug, Clone, Serialize, Default)]
     |                        ^^^^^^^^^ the trait `Serialize` is not implemented for `Vec<std::net::IpAddr>`
...
95   |     /// Any IPv4 and IPv6 ips that were discovered during
     |     ----------------------------------------------------- required by a bound introduced by this call
     |
     = help: the following other types implement trait `Serialize`:
               bool
               char
               isize
               i8
               i16
               i32
               i64
               i128
             and 93 others
     = note: required for `std::option::Option<Vec<std::net::IpAddr>>` to implement `Serialize`
note: required by a bound in `enrich::_::_serde::ser::SerializeStruct::serialize_field`
    --> /home/tscanlan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.188/src/ser/mod.rs:1865:12
     |
1859 |     fn serialize_field<T: ?Sized>(
     |        --------------- required by a bound in this associated function
...
1865 |         T: Serialize;
     |            ^^^^^^^^^ required by this bound in `SerializeStruct::serialize_field`

error[E0277]: the trait bound `Vec<(std::net::IpAddr, String)>: Serialize` is not satisfied
    --> /home/tscanlan/.cargo/git/checkouts/twistrs-ab64395ec7e7a9bf/5ebc1d1/twistrs/src/enrich.rs:90:24
     |
90   | #[derive(Debug, Clone, Serialize, Default)]
     |                        ^^^^^^^^^ the trait `Serialize` is not implemented for `Vec<(std::net::IpAddr, String)>`
...
106  |     /// IP addresses resolved through GeoIP lookup to City, Country, Continent.
     |     --------------------------------------------------------------------------- required by a bound introduced by this call
     |
     = help: the following other types implement trait `Serialize`:
               bool
               char
               isize
               i8
               i16
               i32
               i64
               i128
             and 93 others
     = note: required for `std::option::Option<Vec<(std::net::IpAddr, String)>>` to implement `Serialize`
note: required by a bound in `enrich::_::_serde::ser::SerializeStruct::serialize_field`
    --> /home/tscanlan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.188/src/ser/mod.rs:1865:12
     |
1859 |     fn serialize_field<T: ?Sized>(
     |        --------------- required by a bound in this associated function
...
1865 |         T: Serialize;
     |            ^^^^^^^^^ required by this bound in `SerializeStruct::serialize_field`

error[E0277]: the trait bound `String: Serialize` is not satisfied
    --> /home/tscanlan/.cargo/git/checkouts/twistrs-ab64395ec7e7a9bf/5ebc1d1/twistrs/src/enrich.rs:115:24
     |
115  | #[derive(Debug, Clone, Serialize)]
     |                        ^^^^^^^^^ the trait `Serialize` is not implemented for `String`
...
120  |     /// Message received back from the SMTP server
     |     ---------------------------------------------- required by a bound introduced by this call
     |
     = help: the trait `Serialize` is implemented for `str`
note: required by a bound in `enrich::_::_serde::ser::SerializeStruct::serialize_field`
    --> /home/tscanlan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.188/src/ser/mod.rs:1865:12
     |
1859 |     fn serialize_field<T: ?Sized>(
     |        --------------- required by a bound in this associated function
...
1865 |         T: Serialize;
     |            ^^^^^^^^^ required by this bound in `SerializeStruct::serialize_field`

error[E0277]: the trait bound `String: Serialize` is not satisfied
    --> /home/tscanlan/.cargo/git/checkouts/twistrs-ab64395ec7e7a9bf/5ebc1d1/twistrs/src/permutate.rs:36:26
     |
36   | #[derive(Default, Debug, Serialize)]
     |                          ^^^^^^^^^ the trait `Serialize` is not implemented for `String`
...
41   |     /// The top-level domain of the FQDN (e.g. `.com`).
     |     --------------------------------------------------- required by a bound introduced by this call
     |
     = help: the trait `Serialize` is implemented for `str`
note: required by a bound in `enrich::_::_serde::ser::SerializeStruct::serialize_field`
    --> /home/tscanlan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.188/src/ser/mod.rs:1865:12
     |
1859 |     fn serialize_field<T: ?Sized>(
     |        --------------- required by a bound in this associated function
...
1865 |         T: Serialize;
     |            ^^^^^^^^^ required by this bound in `SerializeStruct::serialize_field`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `twistrs` (lib) due to 5 previous errors

Should take a look at the whois_rust crate and see if it's possible to refactor this to an async call - https://docs.rs/whois-rust/latest/whois_rust/#asynchronous-apis

Missing new permutations modes added to dnstwist

Since release of the library, there have been a couple of permutations modes that need to be implemented. These currently include:

These need to be implemented and benchmarked to make sure they operate properly

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.