charlie0227 / crawler_shopee Goto Github PK

View Code? Open in Web Editor NEW

51.0 4.0 15.0 84 KB

Shopee coin getter is a script to collect daily shopee coins.

Python 93.90% Dockerfile 6.10%

shopee coin crawler selenium python

crawler_shopee's Introduction

Shopee coin getter

Shopee coin getter is a script to collect daily shopee coins.

dependencies

python==3.6
selenium==3.8.0

Installation

Docker

Getting Started

git clone https://github.com/charlie0227/crawler_shopee.git
cd crawler_shopee
cp env.py.sample env.py

Filled in your username and password in env.py

text_username = "<your shopee username>" 
text_password = "<your shopee password>"
cookie_name = "cookie.pkl"

Usage

Simply run command

docker run -it --rm -v <your-path>/crawler_shopee:/code charlie27/python36-selenium-chromedriver sh -c "python main.py"

Or build by yourself

docker build -t shopee:latest .
docker run -it --rm -v <your-path>/crawler_shopee:/code shopee sh -c "python main.py"

You'll need to enter SMS authenticate first time if used password to login

Please Enter SMS code in 60 seconds:

crontab

0 0 * * * /usr/bin/docker run --rm -v <your-path>/crawler_shopee:/code shopee sh -c "python main.py" >/dev/null

Running the tests

Explain how to run the unit test

docker run -it --rm -v .:/code charlie27/python36-selenium-chromedriver sh -c "python unit_test.py"

Method

checkPopModal()

Auto close the advertisement modal shopee show first time

checkLogin()

check is login or not

loginByCookie()

First, check your cookie is able to login, if success goto clickCoin

loginByPass()

Second, use your account and password to login

checkSMS()

Third, if you login by password first time, you'll need to pass SMS authenticate.

clickCoin()

Last, goto https://shopee.tw/shopee-coins-internal/?scenario=1 to own your shopee daily coin after login.

License

MIT

crawler_shopee's People

Stargazers

Watchers

Forkers

gotraveltoworld epireve changdt rodney-peng yylin48 loserkidz joe159119 7elev3n b4456609 mvn-bachhuynh-dn mk386 alvinliew88 iq-scm yicheng2024 azwanzuharimi

crawler_shopee's Issues

Login Failed

Init driver done.
[INFO] pop modal not found
[INFO] cookie.pkl not found
[INFO] Login Failed
[ERROR] Login Modal not showingWebDriverException('unknown error: Element

...

is not clickable at point (1182, 17). Other element would receive the click:

\n (Session info: headless chrome=71.0.3578.98)\n (Driver info: chromedriver=2.45.615279 (12b89733300bd268cff3b78fc76cb8f3a7cc44e5),platform=Linux 4.9.184-linuxkit x86_64)', None, None)
Traceback (most recent call last):
File "main.py", line 85, in loginByPass
login_button = self.driver.find_elements_by_css_selector(".navbar__link--account")[1].click()
File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 501, in _execute
return self._parent.execute(command, params)
File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 311, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 237, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Element

...

is not clickable at point (1182, 17). Other element would receive the click:

(Session info: headless chrome=71.0.3578.98)
(Driver info: chromedriver=2.45.615279 (12b89733300bd268cff3b78fc76cb8f3a7cc44e5),platform=Linux 4.9.184-linuxkit x86_64)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 165, in
a.loginByPass()
File "main.py", line 89, in loginByPass
self.exit()
AttributeError: 'Crawler' object has no attribute 'exit'

[main.py:138 -        checkPopModal() ] 2020-09-14 10:12:55,390 root         INFO     pop modal not found
[main.py:153 -        loginByCookie() ] 2020-09-14 10:12:57,155 root         INFO     Use cookie.pkl to login
[main.py:146 -           checkLogin() ] 2020-09-14 10:13:02,575 root         INFO     Login Failed
[main.py:164 -          loginByPass() ] 2020-09-14 10:13:03,648 root         ERROR    Login Modal not showingElementClickInterceptedException('element click intercepted: Element <a class="navbar__link navbar__link--account navbar__link--tappable navbar__link--hoverable navbar__link-text navbar__link-text--medium" href="/buyer/login?next=https%253A%252F%252Fshopee.com.my%252F">...</a> is not clickable at point (1174, 17). Other element would receive the click: <div class="shopee-popup__overlay"></div>\n  (Session info: headless chrome=85.0.4183.102)', None, None)
[main.py:256 -                close() ] 2020-09-14 10:13:03,763 root         INFO     Program exit

My suspect is that when you open https://shopee.com.my for the first time, it prompts a new modal that asks users to choose a language and the modal is blocking it from clicking the login button.