Inspired by this medium post but uses Pillow for screenshotting and Google Vision API for OCR parsing instead.

Usage:

workon venv 
pip install -r requirements.txt

// download a credentials.json service credentials file from a Google Vision-enabled Cloud Platform project (see OCR section below)
// then create config.json following config.json.sample

// connect iphone to Mac
// position quicktime player at top left -> select New Movie Recording
// open recording button dropdown -> switch Movie recording to iphone

// run the full script:
// - will launch browser to google search of the question
// - in terminal, prints out the search results count for each q+a search (usually, take the  one with largest count)
python main.py

USAGE: from screengrab import screenshot

• Quicktime player must be positioned at top left of screen, on iPhone recording
• Uses PILLOW imagegrab - bounding box grabs only question + the multiple choice answers

USAGE: from detect_text import parse_screenshot

• Processes screenshot into question + answers
• Launches browser to google search of the question

This uses Google Cloud Vision api, so make sure you've downloaded a service account file from a Google Cloud project that has enabled Google Cloud Vision. See "using a service account": https://cloud.google.com/vision/docs/auth

Then add the path of that service account json file config.json as CREDENTIALS_PATH.

py detect_text.py returns a dict:

{
    question: 'After Texas , what U . S . state produces the most crude oil ?'
    answers: ['Oklahoma', 'North Dakota', 'Alaska']
}

USAGE: from google_search import run_query_all

https://developers.google.com/custom-search/json-api/v1/overview

Compare the total num results for each answer:

answer: Oklahoma === TOTAL: 1,180,000
answer: North Dakota === TOTAL: 1,360,000
answer: Alaska === TOTAL: 1,330,000

Notes:

• if the question is a "not" question ("which of these is NOT...") take the one with the lowest count
• it's not unusual for the answer to be the second largest