2FA/OTP issues about debbit HOT 12 CLOSED

Latest update fixed my issue! Thx!

from debbit.

I guess I'll continue this thread. I successfully got 2 purchases in. I'll see what happens tomorrow. Not sure I can stay logged in that long.

"pipenv run ./debbit.py" ends up opening debbit.py in wordpad for me (new instructions on how to run from source), whereas pipenv shell and python debbit.py works.

from debbit.

Only email was supported for 2FA. I turned on text message 2FA on my Amazon account and added the code to support text message 2FA in https://github.com/jakehilborn/debbit/pull/16/files and merged it.

Since you're running from source, can you try it out and let me know if it works for you?

from debbit.

That seems to have worked, but I don't know how to get cookies to work in firefox with geckodriver/selenium, so there's really no advantage to having 2FA support at the moment.

And a new secondary issue that seems to have popped up (on both release and master unfortunately):

ERROR: 2020-05-10 00:07:19,029 cardETFCU_amazon_gift_card_reload error: Traceback (most recent call last):
File "debbit.py", line 281, in web_automation_wrapper
result = merchant.web_automation(driver, merchant, amount)
File "/home/vm/scripts/src/merchants/amazon_gift_card_reload.py", line 111, in web_automation
driver.find_element_by_xpath("//span[contains(text(),'ending in " + merchant.card[-4:] + "')]").click()
File "/home/vm/.local/share/virtualenvs/src-fUWS5gmn/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "/home/vm/.local/share/virtualenvs/src-fUWS5gmn/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "/home/vm/.local/share/virtualenvs/src-fUWS5gmn/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/home/vm/.local/share/virtualenvs/src-fUWS5gmn/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: Element could not be scrolled into view

from debbit.

Yes I've seen that ending in " + merchant.card[-4:] issue on my side as well. Typically, 1 of the 5 retries fixes it. I'm investigating what I can do to make that part more resilient.

The latest code on the master branch should fix your cookie issue. Set your config file to have close_browser: no which will re-use the same Firefox instance from purchase to purchase. When you first start debbit you'll need to do the 2FA thing. However, once it's running you should be able to just let it run without a hitch.

from debbit.

I'm not getting any success. The first gets stuck on the reload page. The next 4 attempts get stuck on the password page, looking for "ap_email" when it doesn't exist.

v1.0.1-dev Traceback (most recent call last):
File "debbit.py", line 281, in web_automation_wrapper
result = merchant.web_automation(driver, merchant, amount)
File "C:\Run\debbit\debbit-master\src\merchants\amazon_gift_card_reload.py", line 32, in web_automation
driver.find_element_by_id('ap_email').send_keys(merchant.usr)
File "C:\Users\John.virtualenvs\src-lRJ7kzJg\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "C:\Users\John.virtualenvs\src-lRJ7kzJg\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\Users\John.virtualenvs\src-lRJ7kzJg\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\John.virtualenvs\src-lRJ7kzJg\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="ap_email"]

After these failures, the browser closes as well, so OTP needs to be redone as well next time.

from debbit.

Hmmm, I wonder what I screwed up. I tested all these scenarios with 2FA enabled on my account before pushing the changes to GitHub.

Can you zip up your failures directory and email it to [email protected] or upload it to this GitHub Issue? I'll take a look at the screenshots and error messages to see what happened. Thanks for the feedback by the way. It's always good to get the behavior of these webpages with different people's accounts that have slightly different preferences/behavior.

from debbit.

Sent. It was actually 2 attempts on the credit card page and 3 stuck on the login page. I'm assuming this was because amazon automatically logged me out after the 2nd try and waiting ~5000 seconds.

The screenshots and html files aren't exactly purged of every bit of identifying information, and I do have an awful lot of cards added to my account.

Is the firefox window supposed to stay open forever or does it close after the burst is done?

from debbit.

Thank you so much for the error reports, I was able to see what's wrong. I pushed the fix to master, please pull the latest code and try it out! I'll keep it running on my side too to see if any errors show up. I've also implemented email notifications for failures that you may find useful. Just put your email in config.txt like shown in sample_config.txt. Emails will be sent from [email protected], hopefully they don't go straight to spam.

Yes, the Firefox window is supposed to stay open forever. The reason is that geckodriver doesn't allow setting cookies for any website other than the site it is currently on. This makes setting cookies difficult for websites that have many domains like website.com, login.website.com, payment.website.com, etc. Geckodriver requires visiting each site individually to set any saved cookies. And often pages like payment.website.com redirect to login.website.com when not logged in so setting cookies is super tricky and hard to generalize.

The workaround I'm using is that the Firefox window stays open so cookies are preserved. I know it's not great, and I'm considering writing a Firefox plugin that can override the cookie domain limitation. If you're using close_browser: no then I highly recommend also using hide_web_browser: yes so that there aren't Firefox windows visible on your screen all the time.

from debbit.

Sweet! If it fails again let me know and I'll be happy to debug. Hoping it works this time though, you've sure gone through a lot of hassle and I appreciate all the feedback.

I spun up a Windows VM. You're right, pipenv run ./debbit.py does not work. I updated the instructions to use pipenv run python debbit.py which should work on all platforms.

from debbit.

Day 2 worked fine. Now I just need to review your change you made on the amazon script to figure out where the fix happened, because I'm getting a similar error on my cable company's website.

from debbit.

I noticed that Amazon will always ask for the password if not logged in. However, if its your first time logging in you need to specify your email. If it's a 2nd time login, then your email is pre-filled on the page. The change I made was to check if the email input field existed, and only fill in if it is found.

if driver.find_elements_by_id('ap_email'):  # if first run, fill in email. If subsequent run, nothing to fill in
    driver.find_element_by_id('ap_email').send_keys(merchant.usr)

This code is a little tricky. find_elements_by_id (notice the s in elements) will return an empty list if nothing is found instead of throwing an error.

Note, a good way to debug is to review the files in the failures directory. Often I will replace the line driver.get('https://www.amazon.com/asv/reload/order') with driver.get('file:///path/to/failures/2020-05-10_10-00-12-050078_merchant.html'). Then use Selenium to poke around the HTML file to see what elements were there when the automation failed.

I encourage you to create a pull request or comment/email me your local cable company's website automation script. The more merchants the merrier 🙂.

from debbit.