Giter Club home page Giter Club logo

onionscraper's Introduction

onionScraper

onionScraper is a Python script for .onion web site scraping.

Installation

Assuming that you have Python and pip already installed, you can then install the necessary dependencies:

pip install requests beautifulsoup4 pandas

Usage

python3 onionScraper.py

Troubleshooting

Typically, there could be a problem with connecting to the SOCKS5 proxy server at localhost:9051, for ex. the "connection is being refused", which means that the proxy server may not be running, or it may be running on a different port. To troubleshoot such network issues, you can try the following:

see which ports are being used by Tor

sudo netstat -tlnp | grep tor
cat /etc/tor/torrc | grep ControlPort

Verify that the Tor service is running

sudo systemctl status tor

Check/add your user to the debian-tor

sudo adduser <your-username> debian-tor

Check that the Tor proxy settings are correct. By default, the Tor control port is not enabled. To enable it, you need to add/uncomment the following lines to your torrc file and restart the Tor service

ControlPort 9051
CookieAuthentication 1

If the Tor service is running and the proxy settings are correct, try restarting the Tor service

sudo systemctl restart tor

If none of the above steps work, try connecting to the Tor network using a different Tor client

onionscraper's People

Contributors

semelnyk avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.