Giter Club home page Giter Club logo

ripmegw's Introduction

ripmegw

Found a wrapper for ripme a while ago, have expanded on it a bit

  • subreddits.txt holds a list of subreddits to rip, one per line
  • /authors will hold lists of reddit-accounts to rip, one file per subreddit
  • /rips will hold the ripped files

Note: check filepaths in all the scripts. The current one is customized to run in /var/www

First, run get_authors_all.pl. This will get the authors of the top posts in each of the subreddits you have in subreddits.txt The file will be checked for duplicates and all existing users in rips/ will also be added to the list Usernames added to banned_authors.txt will not be added to the list of accounts to be ripped

If you want to manually add a reddit-account to be ripped, just add the userid on its own row to the textfile in /authors. The next time you run get_authors.pl it will be ripped

Next, run rip_authors.pl, this will start the actual ripping.

After each author has been ripped, it will be removed from the textfile in /authors, so if you stop and start the ripping-process, it will resume from the last account it ripped.

Eventually, the textfile will be empty. When that is the case, re-run get_authors.pl and start over.

The script checkripping.sh is a script created to run as a cron-job. It checks the modified-date on the authors.txt files. If the age of the file is older than 15 minutes, it will restart the ripping-process. This is a workaround for ripme which has a tendency to hang sometimes.

ripmegw's People

Contributors

scuttlese avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.