Giter Club home page Giter Club logo

hackjohn's Introduction

Bot to monitor for southbound permit spaces on the John Muir Trail

Hackjohn CI

This repository contains a Python script for monitoring Yosemite's permit availability report. Vacancies in this table indicate the availability of a southbound John Muir Trail permit, available for immediate reservation by filling out the online permit request form. This strategy enables securing a soutbound JMT permit after the lottery drawing for a given date.

Learn more about using hackjohn and its history of success in the blog post titled Introducing the hackjohn bot for southbound John Muir Trail permits and The Mercury News article titled Here’s how to get a coveted reservation to hike the John Muir Trail.

Usage

Follow these steps to run hackjohn:

  1. Clone this repository.
  2. Modify config.py with with the parameters of your permit search. This includes your 2Captcha API key (see the Captcha solving service section) and your tokens for Telegram, IFTTT, and/or Twilio (see the Notifications section).
  3. Then simply run the hackjohn Python script in the hackjohn appropriate environment (see Environment section) with:
python hackjohn.py

The script will print some progress messages and at the end it will print a message containing available permit details. The same message will be sent via whichever notification services you set up. An example of the report is:

2021-08-20:
5 permits for Happy Isles->Little Yosemite Valley
2 permits for Lyell Canyon

2021-09-03:
1 permit for Happy Isles->Little Yosemite Valley

Report last updated 2021-03-20 10:34:32 AM PDT.

Permit request form: https://yosemite.org/yosemite-wilderness-permit-request-form/
Permit office phone: 209-372-0826

By default, hackjohn writes the output to the file hackjohn-output.txt (as specified by the OUTPUT_PATH variable in config.py). To avoid repeated notification, hackjohn skips sending notifications if its output matches the pre-existing output.

Captcha solving service

As of June 22, 2020, the trailhead report switched to a new website that moved the permit report behind a Recaptcha (the "I am not a robot" thing). This made it more difficult for bots to access the report.

hackjohn uses 2Captcha to solve the Recaptcha and get to the permit report. You will need to create a 2Captcha account and upload some funds, then copy your 2Captcha API key into the CAPTCHA_API_KEY variable in config.py.

This costs money, but it is very cheap (a few dollars will get you 1000 Recaptchas). hackjohn reuses the same Recaptcha authorization as long as it is still valid, so it only calls the 2Captcha service when necessary (currently the authentication lasts about a day, but that is subject to change). Realistically, it will cost you at most a couple dollars to run hackjohn multiple times per day for an entire year.

Note: There are many alternatives to 2Captcha and they all work pretty much the same way. I didn't have any particular reason for choosing 2Captcha. Feel free to use a different Captcha solving service, but you will need to make modifications to the code.

Notifications

hackjohn supports the following services to provide notifications:

  • The Telegram messaging app. Refer to the webhook2telegram repo for more details.

    1. Download the Telegram app at https://telegram.org. Mobile and desktop apps are available.
    2. Message "\start" to @MiddleManBot. It will send back a token.
    3. In config.py, set ENABLE_TELEGRAM to True and set TELEGRAM_TOKEN to the token you received in the previous step.
  • If This Then That (IFTTT). (Thanks Markus Neuhoff for contributing this feature!)

    1. Create an applet at https://ifttt.com/create. (You will also need to create an IFTTT account.)
    2. In the "If This" section of your applet, click "Webhooks" and then "Receive a web request". Give it a name (e.g., "hackjohn"). Add this name to the IFTTT_EVENT_NAME variable in config.py.
    3. In the "Then That" section of your applet, select the type of notification you would like to receive. For example, "Notifications -> Send a rich notification from the IFTTT app" or "Email -> Send me an email".
    4. Go to https://ifttt.com/maker_webhooks, click on documentation, and copy your key at the top of the screen into the IFTTT_KEY variable and set ENABLE_IFTTT to True in config.py.
  • SMS text messages with Twilio.

    1. Create an account at https://www.twilio.com.
    2. In the "Project Info" section of your Twilio dashboard, copy your Account SID and Auth Token values into the TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN variables in config.py, respectively.
    3. Click "Get a trial phone number" in your Twilio dashboard. Twilio will generate a phone number for your account. Copy this phone number into the TWILIO_PHONE_NUMBER variable in config.py.
    4. Set the TWILIO_TO_PHONE variable to the phone number you want to send notifications to (i.e., your phone number) and setENABLE_TWILIO to True in config.py.
    5. Note: It costs a small amount of money to operate your Twilio account (currently $1 per month to maintain your phone number plus $0.0075 per message sent). It is often possible to find promos that will add money to your account. For example, there is currently a promo code for $50 in free credits in the "Basic Training" section in Twilio Quest (working as of March 2021).

hackjohn can be run without enabling notifications, which is useful for prototyping and development, but less useful for automated monitoring.

Environment

The recommended installation method is to create a virtual environment for just hackjohn. This ensures installation does not modify dependencies for other projects. To install a virtual environment, run the following:

# Create a virtual environment in the env directory
python3 -m venv env

# Activate the virtual environment
source env/bin/activate

# Install the required dependencies into the virtual env
pip install --requirement requirements.txt

# Now you can run hackjohn
python hackjohn.py

Automation

You can automate the running of hackjohn.py at scheduled times every day. According to the Yosemite permit office, the trailhead quotas are often updated around 11 AM Pacific Time.

I used cron, which despite its awful interface got the job done. Therefore, I added the following lines to my crontab configuration (replacing ABOSOLUTE_REPO_PATH with the absolute path to the directory containing hackjohn.py):

# hackjohn bot scheduling (configured for a system on Eastern Time)
0 13 * * * ABOSOLUTE_REPO_PATH/hackjohn.py >> ABOSOLUTE_REPO_PATH/hackjohn.log
44 13 * * * ABOSOLUTE_REPO_PATH/hackjohn.py >> ABOSOLUTE_REPO_PATH/hackjohn.log
59 13 * * * ABOSOLUTE_REPO_PATH/hackjohn.py >> ABOSOLUTE_REPO_PATH/hackjohn.log
14 14 * * * ABOSOLUTE_REPO_PATH/hackjohn.py >> ABOSOLUTE_REPO_PATH/hackjohn.log
30 14 * * * ABOSOLUTE_REPO_PATH/hackjohn.py >> ABOSOLUTE_REPO_PATH/hackjohn.log
40 14 * * * ABOSOLUTE_REPO_PATH/hackjohn.py >> ABOSOLUTE_REPO_PATH/hackjohn.log
00 15 * * * ABOSOLUTE_REPO_PATH/hackjohn.py >> ABOSOLUTE_REPO_PATH/hackjohn.log
19 15 * * * ABOSOLUTE_REPO_PATH/hackjohn.py >> ABOSOLUTE_REPO_PATH/hackjohn.log
00 19 * * * ABOSOLUTE_REPO_PATH/hackjohn.py >> ABOSOLUTE_REPO_PATH/hackjohn.log

Run crontab -e to edit your cron configuration. In order to for the cron-scheduled script to run in the proper environment, you must add a shebang pointing to which Python to use. For example, replace the following with the output of which python when you have activated the right environment:

#!/home/username/path/to/hackjohn/env/bin/python

hackjohn's People

Contributors

dhimmel avatar djcunningham0 avatar eckmlje avatar neuhoffm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

hackjohn's Issues

Permit application moved to recreation.gov

Looks like permits have moved to recreation.gov this year so hackjohn is broken again. I assume the permit application process has otherwise remained the same, but I haven't looked into it in detail. Either way, it could be a substantial amount of work to update hackjohn.

I don't have the time to work on this at the moment but it's possible I will later this spring or summer. If anyone else wants to take a stab at it I can offer some help as needed.

Including report date / avoiding repeated notification

Is there a way to include the report date in the message that gets sent out? It looks like the report isn't updated very consistently and I'd like to include the report date in the message. There doesn't seem to be a specific css class associated with the report date span but it is the only occurrence of the "Report Date:" string in the file. Python isn't my strong suit or I'd add a PR for this feature.

New Yosemite Planning Your Wilderness Permit page prevents automated access to the trailhead report

Currently, the trailhead report webpage that hackjohn uses at https://www.nps.gov/yose/planyourvisit/fulltrailheads.htm shows the following:

image

The "improved version" at https://yosemite.org/planning-your-wilderness-permit/ places the trailhead report behind a capcha:

image

So it looks like users will no longer be able to perform automated checks of the trailhead report, like Hackjohn used to enable. This is a bit of a bummer, but perhaps Hackjohn was a victim of its own success!

If anyone is aware of a workaround or finds the trailhead available from a public API endpoint, let us know below and Hackjohn might be able to resurrect itself.

Running hackjohn from cron

Hi there--very cool project. I can run hackjohn from the shell, but not from cron--when it runs from cron it does not seem to be able to successfully load the anaconda environment. Do you mind sharing more details on how you managed to configure it with cron?
I have created a shell script to be executed by cron. I have tried every possible permutation of:
cd /Users/friedman/hackjohn
conda activate hackjohn or source /anaconda2/bin/activate hackjohn
/anaconda2/envs/hackjohn/bin/python hackjohn.py or python hackjohn.py or
hackjohn.py (with the proper shebang line at the top)

Could you share insight about how you got cron to properly activate the anaconda environment and use the proper version of python? thanks so much:)

ValueError "Date" is not in list

I'm a beginner at python, but I did manage to get this bot to work earlier today, 3 pm PDT. A little later (7 pm PDT), I tried to run it again and it gave me the ValueError "Date" is not in list message. I checked the trailhead website and the table looks really different than it did earlier today. I think Yosemite NP changed the format of the table at some point this afternoon, and that's what is messing up the code.

Confusion about how to read the reservation vacancy table

Hi - Sorry for posting my off topic question in the other issue. I've spent a ton of time in the high sierra but never really bothered with the Yosemite system since it always confused me.

I was wondering if you could help me understand how to read the table of vacancies. Mainly I was wondering, are dates other than the 2-week-out date subject to lottery? For instance, if I wanted to submit a reservation for August 13 (1 available) or September 5 (4 available), are these bookable today (Aug 4)?

Apologies if my question is confusing. I'm trying to determine if it's worth it to work on a scraper to bypass the captcha or if it's the chances of getting any sort of permit through this system is low.

image

Use GitHub Actions for notifications

It should be possible to do everything via GitHub actions, so users don't have to run anything locally.

Instead, a user would fork dhimmel/hackjohn and edit the parameters. Then scheduled cron actions would take care of the rest.

Empty DataFrame

Any reason I would be getting this message?

Empty DataFrame
Columns: [Date, Trailhead, Spaces]
Index: []

IS this still working?

I am getting lots of error. I solved some of them but their is no Trailhead column in the df.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.