alexburkey / mfaimagebot Goto Github PK

View Code? Open in Web Editor NEW

3.0 2.0 0.0 71 KB

A bot which direct-links numbered images in an album

License: GNU General Public License v3.0

Python 93.36% Shell 6.64%

bot reddit-bot reddit redditbot imgur imgur-links python python3 python-3 python-bot

mfaimagebot's People

Contributors

Stargazers

Watchers

mfaimagebot's Issues

Better deployment setup

Is your feature request related to a problem? Please describe.
The nested deployment scripts are (probably) useless. I can most likely remove the shell-script retry stuff and just use the cron job.

Describe the solution you'd like
Use just a cron job to check if the bot is running and start it if it isn't running. This would simplify the deployment a lot and make redeploys and new releases easier.

Process lists of single images

Is your feature request related to a problem? Please describe.
Often people will reference multiple numbers in an album. For example, "I like how 6, 7, 12, and 34 look." Currently, each image request would have to be done in a separate comment. Processing them like this would be nicer: !MFAImagebot op 6, 7, 12, 34

Notes:
I'll have to figure out how to process them easily and without errors/exceptions. Failure should be graceful for cases where an individual number from the list is bad. For example, !MFAImagebot op 6, 7, 12, 3400 (index out of bounds) or !MFAImagebot op 6, 7, !@, 34 (!@ not a number).

Script redeploying bot

Is your feature request related to a problem? Please describe.
QOL improvement. redeploying the bot takes a few steps. Would be nice to script this and add some differentiation between runs of the bot in the output/log

Describe the solution you'd like
To start a basic shell script which just kills existing processes, moves the current nohup.out file to some other location, and restarts the bot.

Describe alternatives you've considered
Haven't considered any yet. Will scale the solution as needed.

Additional context
NA

Bot should pull imgur album/galler link from reddit post link

Example reddit post:
https://www.reddit.com/r/malefashionadvice/comments/gm4wrx/top_of_waywt_april_2020/

Example comment:
!MFAImageBot link 16

Desired behavior/response:
`#16 \nImage number 16 from album https://imgur.com/a/T7LITMx'
Reddit post is a link post to an imgur album/gallery. Bot should get that link when no album link is specified.
Failure path(s): If there is no album link supplied and the post is not a link post the bot should respond with a message asking for the link to be included.
Future feature: Bot searches reddit self-post source for the first imgur album then uses that instead.

Bot missed a comment, unsure of cause

Describe the bug
Someone made a comment calling the bot but the bot didn't see the comment and didn't respond. Without a restart the bot responded to a similar comment that I constructed. But there seems to be weird whitespace characters in the comment which might be the cause.

To Reproduce
Currently unsure of exact cause. See link to comment below.

Expected behavior
The bot should see and respond to the comment.

Comment Link
https://www.reddit.com/r/malefashionadvice/comments/gupgck/hello_i_am_mfaimagebot_a_bot_built_to_help/fsqm25v/

Comment text taken from source

 !mfaimagebot op 12

&#x200B;

Anyone know what that jacket is?

Additional context
I believe it might be a zero-width whitespace in the character which is messing with it. Perhaps there's a leading one that I can see for some reason.

Bot nickname

Is your feature request related to a problem? Please describe.
I love this bot but my pampered thumbs would love it even more if there was an option to summon it with fewer characters, eg !mfa or !mfai. Kind of like a nickname. Just a thought, cheers!

Image caption searching

Match image captions to keywords using Acora (or something similar).

Then you could call the bot like !MFAImageBot search <keywords> and it would return a list (maybe 5 max) of the images whose captions match the keywords

Implement new Reddit Galleries

Is your feature request related to a problem? Please describe.
Reddit introduced a "gallery" feature now on posts. Investigate and implement the same functionality for reddit galleries if possible.

Describe the solution you'd like
Galleries are only link posts so if someone comments !MFAImageBot 12 on a post the bot should look in the reddit gallery and attempt to get a direct link to image 12.

Describe alternatives you've considered
N/A

Additional context
Announcement

New Reddit sucks.

Use RegEx to parse commands

Is your feature request related to a problem? Please describe.
Refactor code to use regex to parse sections of the command: command, imgur link, number(s)

Exceptions from Praw comments stream crashes bot

Describe the bug
Currently using Praw's reddit comments stream to monitor comments in the subreddit. When there is an issue with the stream or reddit like a 503 error the program just raises the exception and drops out

To Reproduce
Take reddit down I guess? I'm not interested in mocking a reddit client.

Expected behavior
The bot should be able to deal with reddit being down just like it deals with issues with making requests to Imgur.

Additional context
nohup.out log:

Setting up DB...
Done!
Looking for comments...
Traceback (most recent call last):
  File "bot.py", line 250, in <module>
    for comment in r.subreddit(SUBREDDIT_NAME).stream.comments():
  File "/home/pi/.local/lib/python3.7/site-packages/praw/models/util.py", line 186, in stream_generator
    for item in reversed(list(function(limit=limit, **function_kwargs))):
  File "/home/pi/.local/lib/python3.7/site-packages/praw/models/listing/generator.py", line 61, in __next__
    self._next_batch()
  File "/home/pi/.local/lib/python3.7/site-packages/praw/models/listing/generator.py", line 71, in _next_batch
    self._listing = self._reddit.get(self.url, params=self.params)
  File "/home/pi/.local/lib/python3.7/site-packages/praw/reddit.py", line 490, in get
    return self._objectify_request(method="GET", params=params, path=path)
  File "/home/pi/.local/lib/python3.7/site-packages/praw/reddit.py", line 574, in _objectify_request
    data=data, files=files, method=method, params=params, path=path
  File "/home/pi/.local/lib/python3.7/site-packages/praw/reddit.py", line 732, in request
    timeout=self.config.timeout,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 336, in request
    url=url,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 235, in _request_with_retries
    url,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 195, in _make_request
    timeout=timeout,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/rate_limit.py", line 35, in call
    kwargs["headers"] = set_header_callback()
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 282, in _set_header_callback
    self._authorizer.refresh()
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/auth.py", line 352, in refresh
    password=self._password,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/auth.py", line 149, in _request_token
    response = self._authenticator._post(url, **data)
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/auth.py", line 32, in _post
    raise ResponseException(response)
prawcore.exceptions.ResponseException: received 503 HTTP response
[Sun 24 May 14:47:38 BST 2020] bot exited with code 1. restarting ...

Add testing pipeline for branches

Investigate Python testing tools like

pytest: https://docs.pytest.org/en/latest/
doctest: https://docs.python.org/3/library/doctest.html

Need to investigate and see what others are there and available for GH Actions.

Implement proper logging

Is your feature request related to a problem? Please describe.
Currently all "logs" are just print statements to stdout. Use a proper logger so that crashes and exceptions get sent to one spot while tracking of responses goes to a different log.

Describe the solution you'd like
A separation of crashes like:

  File "bot.py", line 259, in <module>
    for comment in r.subreddit(SUBREDDIT_NAME).stream.comments():
  File "/home/pi/.local/lib/python3.7/site-packages/praw/models/util.py", line 186, in stream_generator
    for item in reversed(list(function(limit=limit, **function_kwargs))):
  File "/home/pi/.local/lib/python3.7/site-packages/praw/models/listing/generator.py", line 61, in __next__
    self._next_batch()
  File "/home/pi/.local/lib/python3.7/site-packages/praw/models/listing/generator.py", line 71, in _next_batch
    self._listing = self._reddit.get(self.url, params=self.params)
  File "/home/pi/.local/lib/python3.7/site-packages/praw/reddit.py", line 490, in get
    return self._objectify_request(method="GET", params=params, path=path)
  File "/home/pi/.local/lib/python3.7/site-packages/praw/reddit.py", line 574, in _objectify_request
    data=data, files=files, method=method, params=params, path=path
  File "/home/pi/.local/lib/python3.7/site-packages/praw/reddit.py", line 732, in request
    timeout=self.config.timeout,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 336, in request
    url=url,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 262, in _request_with_retries
    url,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 170, in _do_retry
    retry_strategy_state=retry_strategy_state.consume_available_retry(),  # noqa: E501
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 262, in _request_with_retries
    url,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 170, in _do_retry
    retry_strategy_state=retry_strategy_state.consume_available_retry(),  # noqa: E501
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 265, in _request_with_retries
    raise self.STATUS_EXCEPTIONS[response.status_code](response)
prawcore.exceptions.ServerError: received 503 HTTP response

From normal log messages like:

Setting up DB...
Done!
Looking for comments...
Comment hash: fscc3yb
Request url: https://api.imgur.com/3/album/MqT99lp/images
Image link: https://i.imgur.com/OoXu0jY.jpg
Hash: fscc3yb
Has responded: True
Response type: op
Comment Body: 
b'!MFAImageBot op 33\n\nLets see if this works...'
Tokens: [b'!MFAImageBot', b'op', b'33', b'Lets', b'see', b'if', b'this', b'works...']
Response comment body: 
b'[#33](https://i.imgur.com/OoXu0jY.jpg)  \nImage number 33 from album https://imgur.com/a/MqT99lp'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

All that's probably needed is just to change the print statements into logger stuff but I want to make sure I understand where the output and stuff is going.

Refactor to get rid of "commands"

Is your feature request related to a problem? Please describe.
Current state with two separate commands is confusing and tough for people to use/remember. Better to consolidate to be able to process these three without any fuss:
!MFAImageBot <number>
!MFAImageBot <link> <number>
!MFAImageBot <number> <link>

This makes it way easier for people to use rather than remembering the difference between link and op commands

Describe the solution you'd like
Bot is able to process just arbitrary inputs of a link and a number (or set of numbers). Can just like split on the first line and then pattern-match the "bat-signal", number, and link in that order.

Describe alternatives you've considered
Could just leave it the way it is and let people learn how to use the bot. But I think this is a better choice.

Another alternative is to rename the commands so they're easier to remember but I think this is the least desirable especially since the bot is deployed and in-use right now.

Additional context
Credit for part of the solution to @hussainjaffery

Monitor Bot and auto-redeploy if it crashes

Is your feature request related to a problem? Please describe.
I need to know if the bot dies somehow, or keep it deployed near constantly.

Describe the solution you'd like
Notifications of failures would be fantastic, but for now I'll probably settle for a bash script

Additional context
Possible solution: https://stackoverflow.com/questions/51838862/python-bot-keep-running-forever

SSL Errors

Describe the bug
Bot restarts frequently due to some SSL error I haven't quite figured out yet.

To Reproduce
Any information necessary to create the bug.

Expected behavior
A clear and concise description of what you expected to happen.

Comment Link
Please include a link to the comment you made that caused the issue.

Additional context
Trace:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 57, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 841, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 301, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x75cfa550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.reddit.com', port=443): Max retries exceeded with url: /api/v1/access_token (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x75cfa550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/requestor.py", line 53, in request
    return self._http.request(*args, timeout=timeout, **kwargs)
  File "/home/pi/.local/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/pi/.local/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/home/pi/.local/lib/python3.7/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.reddit.com', port=443): Max retries exceeded with url: /api/v1/access_token (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x75cfa550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bot.py", line 259, in <module>
    for comment in r.subreddit(SUBREDDIT_NAME).stream.comments():
  File "/home/pi/.local/lib/python3.7/site-packages/praw/models/util.py", line 186, in stream_generator
    for item in reversed(list(function(limit=limit, **function_kwargs))):
  File "/home/pi/.local/lib/python3.7/site-packages/praw/models/listing/generator.py", line 61, in __next__
    self._next_batch()
  File "/home/pi/.local/lib/python3.7/site-packages/praw/models/listing/generator.py", line 71, in _next_batch
    self._listing = self._reddit.get(self.url, params=self.params)
  File "/home/pi/.local/lib/python3.7/site-packages/praw/reddit.py", line 490, in get
    return self._objectify_request(method="GET", params=params, path=path)
  File "/home/pi/.local/lib/python3.7/site-packages/praw/reddit.py", line 574, in _objectify_request
    data=data, files=files, method=method, params=params, path=path
  File "/home/pi/.local/lib/python3.7/site-packages/praw/reddit.py", line 732, in request
    timeout=self.config.timeout,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 336, in request
    url=url,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 262, in _request_with_retries
    url,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 170, in _do_retry
    retry_strategy_state=retry_strategy_state.consume_available_retry(),  # noqa: E501
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 262, in _request_with_retries
    url,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 170, in _do_retry
    retry_strategy_state=retry_strategy_state.consume_available_retry(),  # noqa: E501
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 235, in _request_with_retries
    url,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 195, in _make_request
    timeout=timeout,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/rate_limit.py", line 35, in call
    kwargs["headers"] = set_header_callback()
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/sessions.py", line 282, in _set_header_callback
    self._authorizer.refresh()
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/auth.py", line 352, in refresh
    password=self._password,
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/auth.py", line 149, in _request_token
    response = self._authenticator._post(url, **data)
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/auth.py", line 29, in _post
    "post", url, auth=self._auth(), data=sorted(data.items())
  File "/home/pi/.local/lib/python3.7/site-packages/prawcore/requestor.py", line 55, in request
    raise RequestException(exc, args, kwargs)
prawcore.exceptions.RequestException: error with request HTTPSConnectionPool(host='www.reddit.com', port=443): Max retries exceeded with url: /api/v1/access_token (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x75cfa550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

Make bot more fault-tolerant of bad command parameters

Examples:

Link Command:

Parameters out of order: !MFAImageBot link <index> <album-link>
Missing index: !MFAImageBot link <album-link>
No parameters: !MFAImageBot link

Some of these could eventually be new feature paths. For now it's easiest to respond with a generic help/usage message.

Increase text size for direct image link as QOL Improvement

Is your feature request related to a problem? Please describe.
Longer text for the direct image link. Current situation is something like #6 which can be difficult to click/tap on mobile.

Describe the solution you'd like
Use more text for the image link. Something like Direct link to image #6 would work.

Describe alternatives you've considered
NA

Additional context
Link to comment suggestion

Parse edited comments as well

Is your feature request related to a problem? Please describe.
Sometimes folks will edit their comment to be of the proper format or to add images. Bot should also be looking at and parsing edited comments. I believe there's a feed for this but the bot might have to become a moderator.

More robust calling of the bot

Bot only responds when the token is at the very beginning of a comment. This can be more robust to at least be "at the start of a line" without too much refactoring/coding.

Bot should respond to !mfaimagebot at least at the start of a line.

This is an improvement for the bot. The alternative is to keep current behavior and people will stay frustrated.

Track comments that have been responded to

Problems:

Currently, if the bot dies and is restarted, it will respond to all comments in the feed that it sees (barring limits from reddit's API, etc.) regardless of whether it has responded to that comment already.

Solution:

Write comment hashes to a separate file and check whether a comment hash is in that file already before responding.

Set up monitoring of imgur API

This SHOULD be fine but I don't want to have issues.

https://apidocs.imgur.com/?version=latest#rate-limits

The Imgur API uses a credit allocation system to ensure fair distribution of capacity. Each application can allow approximately 1,250 uploads per day or approximately 12,500 requests per day. If the daily limit is hit five times in a month, then the app will be blocked for the rest of the month. The remaining credit limit will be shown with each requests response in the X-RateLimit-ClientRemaining HTTP header.

Detect and use first album link in a text post

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

It can be hard for people to use the bot for Inspiration posts with Imgur albums embedded in the text. They need to copy the specific link before they can call the bot.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Be able to find the first Imgur album link in a text post and use as default when calling MFAImageBot.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Bot should work with Imgur Galleries as well

Is your feature request related to a problem? Please describe.
Bot isn't able to get direct image links from Imgur galleries

Describe the solution you'd like
Bot should be able to parse and deal with Imgur galleries the same way it does for normal albums.

Track environment/deployment

Terraform and Docker might be overkill. But with all of the modules installed and the nightmare of python versioning anyway it is probably best that I define some of this stuff to make it easy to deploy to a server or something.

Check out Heroku, AWS tiny servers, anything free, maybe just use my RPi to do it.