Giter Club home page Giter Club logo

xn-twist's Introduction

XN Twist

Find Unicode domain squats

image

image

image

Codacy Badge

Updates

NOTE: This project will be deprecated in early 2021. The functionality will still exist in another project (yet to be released).

Installation

Stable release

To install XN Twist, run this command in your terminal:

pip install xn-twist

This is the preferred method to install XN Twist, as it will always install the most recent stable release.

If you don't have pip installed, this Python installation guide can guide you through the process.

From sources

The sources for XN Twist can be downloaded from the Github repo.

You can either clone the public repository:

$ git clone git://github.com/xn-twist/xn-twist

Or download the tarball:

$ curl  -OL https://github.com/xn-twist/xn-twist/tarball/master

Once you have a copy of the source, you can install it with:

$ python setup.py install

Usage

Via Python

You can use XN-Twist in a script as follows:

from xn_twist import XNTwist
xn = XNTwist()
twist_results = xn.twist('example.com')

The following parameters can be passed into the initialization of the XNTwist object:

  • query_dns: Specify whether or not you would like to query dns values for the domains. Defaults to False. Be aware that setting this value to ``True`` will slow the twisting process down significantly.
  • output: Specify an output file to which the json output of the algorithm will be written.

When calling the twist() function, there is an optional limit value which limits the number of characters used as a spoofs. This cuts down on the processing load and makes the results manageable. An example usage is shown below.

from xn_twist import XNTwist
xn = XNTwist()
twist_results = xn.twist('example.com', limit=2)

When calling the twist() function, there is an optional simple value which only returns possible domain squats with one character changed rather than all of the possible permutations (the default is FALSE). This cuts down on the processing load and makes the results manageable. An example usage is shown below.

from xn_twist import XNTwist
xn = XNTwist()
twist_results = xn.twist('example.com', simple=True)

The response will be json that looks something like:

{
    "datetime": "2018-04-06 05:33:45.889048",
    "possible_squats": [
        {
            "displayed": "\u0430.com",
            "punycode": "xn--80a.com"
        },
        {
            "displayed": "\u0105.com",
            "punycode": "xn--2da.com"
        },
        {
            "displayed": "\u03ac.com",
            "punycode": "xn--hxa.com"
        },
        {
            "displayed": "\u00e1.com",
            "punycode": "xn--1ca.com"
        },
        {
            "displayed": "\u01ce.com",
            "punycode": "xn--tja.com"
        },
        {
            "displayed": "\u1f05.com",
            "punycode": "xn--ing.com"
        },
        {
            "displayed": "\u01df.com",
            "punycode": "xn--bka.com"
        },
        {
            "displayed": "\u1f00.com",
            "punycode": "xn--dng.com"
        },
        {
            "displayed": "\u1f01.com",
            "punycode": "xn--eng.com"
        },
        {
            "displayed": "\u00e4.com",
            "punycode": "xn--4ca.com"
        }
    ],
    "count": 10,
    "xn_twist_version": "1.2.0"
}

Via Command Line

You can use XN-Twist from the command line as follows:

xntwist example.com

The usage for the command line form of XN-Twist is as follows:

XN Twist.

Usage:
    xntwist <domain> [--limit=LIMIT] [--dns] [--output=OUTPUT]
    xntwist (-h | --help)
    xntwist --version

Options:
    -h --help     Show this screen.
    --version     Show version.
    -l=LIMIT --limit=LIMIT    Limit the number of characters used as spoofs [default: 5].
    -d --dns  Query DNS for each domain.
    -o=OUTPUT --output=OUTPUT  Specify an output file.

Run Tests

After cloning the repo, you can test it using the following commands from the base directory of this repository:

make test

Data, Data, Everywhere...

This project relies on a dataset. More details on how to access the dataset and even how to help us build it coming soon!

Credits

This package was created with Cookiecutter and the fhightower/python-project-template project template.

xn-twist's People

Contributors

codacy-badger avatar fhightower avatar pyup-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

xn-twist's Issues

Change the indexing of the output json

Right now, the output json looks something like:

{
    "2017-09-23 18:24:53.710278": {
        "a.com": {
            "possible_squats": [
                {
                    "displayed": "\u0430.com",
                    "punycode": "xn--80a.com"
                },
                ...
            ],
            "results": 17
        },
        "character_set_version": "0.1",
        "fuzzer_version": "1.0.1"
    }
}

This is stupid as it is very hard to work with this output unless you know the exact time stamp when the script was run. The time should be recorded, but should not be the root index.

Initial Update

Hi ๐Ÿ‘Š

This is my first visit to this fine repo, but it seems you have been working hard to keep all dependencies updated so far.

Once you have closed this issue, I'll create seperate pull requests for every update as soon as I find one.

That's it for now!

Happy merging! ๐Ÿค–

Initial Update

Hi ๐Ÿ‘Š

This is my first visit to this fine repo, but it seems you have been working hard to keep all dependencies updated so far.

Once you have closed this issue, I'll create separate pull requests for every update as soon as I find one.

That's it for now!

Happy merging! ๐Ÿค–

Write a plug-in that uses the xn-twist-python-sdk to pull updated content from the API

Instead of hard-coding character sets into this script, the character sets are being served on an API. There is a python SDK here that can retrieve the characters from the API (refer to the function here). We need to finalize the SDK, get it on pypi, and then configure this script to use that SDK.

With this change, I'm removing the ability for users to specify which character sets they would like to use when looking for domain squats (e.g. they can no longer specify that they would like to use only Cyrillic characters). I don't think this is a big problem as it is possible to register unicode domain squats that use characters from diverse character sets, but I'm curious if you agree @cezhunter . Don't feel like you have to start working on this issue yet. I'm just curious if you agree that we can get rid of this functionality.

Simplify the Datasets

I feel I need to distill all of the current datasets into two for each character set: complete and simplified. This will make the datasets easier to understand/work with. I at least need to simplify how the datasets are communicated.

Simplify the handling of character sets

Two thoughts:

  • Programatically create the arguments based on the available character sets

  • Allow user to specify a 'level' of character sets to use (e.g. 'simplified', 'advanced', or 'complete')

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.