simoncos / zhihu-analysis-python Goto Github PK

InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised

知乎的个性域名ID，也就是个人主页网址下的https://www.zhihu.com/ID/mnichangxin/activities，ID即为个性域名，现在需要这个ID，希望能解决

你好，在运行你的爬虫的时候，auth.py是可以跑通的，但是在运行zhihu.py的时候，虽然没有报错，但是运行的非常快，很快就结束了。接着我运行了user.py,运行完出现以下内容：

//anaconda/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:843: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
---2016-11-21 22:00:04.381761 start ---
2016-11-21 22:00:04.422632 Process-2 user: lan-xiao-miao-78

//anaconda/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:843: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
Process-2: other error when processing user lan-xiao-miao-78(<type 'exceptions.AttributeError'>, AttributeError("'NoneType' object has no attribute 'find'",), <traceback object at 0x1040b8f80>)

//anaconda/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:843: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
Process-2: other error when processing user lan-xiao-miao-78(<type 'exceptions.AttributeError'>, AttributeError("'NoneType' object has no attribute 'find'",), <traceback object at 0x1062d3ea8>)

//anaconda/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:843: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
Process-2: other error when processing user lan-xiao-miao-78(<type 'exceptions.AttributeError'>, AttributeError("'NoneType' object has no attribute 'find'",), <traceback object at 0x10631cc20>)

//anaconda/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:843: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
Process-2: other error when processing user lan-xiao-miao-78(<type 'exceptions.AttributeError'>, AttributeError("'NoneType' object has no attribute 'find'",), <traceback object at 0x106362998>)

//anaconda/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:843: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
Process-2: other error when processing user lan-xiao-miao-78(<type 'exceptions.AttributeError'>, AttributeError("'NoneType' object has no attribute 'find'",), <traceback object at 0x100732c68>)

Process-2: stopped lan-xiao-miao-78, already maximum attempts

2016-11-21 22:00:09.811255 Process-2 while loop breaks

2016-11-21 22:00:09.811456 Process-2 finished.

2016-11-21 22:00:09.816415 Process-5 while loop breaks

2016-11-21 22:00:09.816634 Process-5 finished.

2016-11-21 22:00:09.839833 Process-3 while loop breaks

2016-11-21 22:00:09.840157 Process-3 finished.

2016-11-21 22:00:09.857533 Process-4 while loop breaks

2016-11-21 22:00:09.858124 Process-4 finished.

---2016-11-21 22:00:09.861912 elapsed 5.47989702225 seconds ---

Repl Closed

然后生成的data文件里面内容都是空的。

接着我运行了topic.py，和前一个一样，运行完topic文件夹里文件都是空的，运行结果如下：

//anaconda/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:843: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
---2016-11-22 11:49:08.749264 start loading data ---
number of distinct elements: 0
---2016-11-22 11:49:08.750311 elapsed 0.00101089477539 seconds ---
---2016-11-22 11:49:08.750334 start crawling ---
2016-11-22 11:49:20.979695 Process-2 while loop breaks

2016-11-22 11:49:20.979980 Process-2 finished.

2016-11-22 11:49:20.980298 Process-3 while loop breaks

2016-11-22 11:49:20.980539 Process-3 finished.

2016-11-22 11:49:20.999034 Process-4 while loop breaks

2016-11-22 11:49:20.999378 Process-4 finished.

2016-11-22 11:49:21.008017 Process-5 while loop breaks

2016-11-22 11:49:21.008303 Process-5 finished.

2016-11-22 11:49:21.021097 Process-6 while loop breaks

2016-11-22 11:49:21.021413 Process-6 finished.

2016-11-22 11:49:21.030949 Process-7 while loop breaks

2016-11-22 11:49:21.030947 Process-8 while loop breaks

2016-11-22 11:49:21.031316 Process-8 finished.

2016-11-22 11:49:21.031297 Process-7 finished.

2016-11-22 11:49:21.050290 Process-9 while loop breaks

2016-11-22 11:49:21.050616 Process-9 finished.

2016-11-22 11:49:21.058523 Process-10 while loop breaks

2016-11-22 11:49:21.058812 Process-10 finished.

2016-11-22 11:49:21.071325 Process-11 while loop breaks

2016-11-22 11:49:21.071646 Process-11 finished.

2016-11-22 11:49:21.082244 Process-13 while loop breaks
2016-11-22 11:49:21.082280 Process-12 while loop breaks

2016-11-22 11:49:21.082624 Process-12 finished.

2016-11-22 11:49:21.082630 Process-13 finished.

---2016-11-22 11:49:21.084341 elapsed 12.3340880871 seconds ---

Repl Closed

请问你运行完也是这样子吗，求帮忙

Process-2: other error when processing user: zhao-che(<type 'exceptions.ValueError'>, ValueError('No JSON object could be decoded',), <traceback object at 0x7f9ee2f4df80>)

Subject : Can we have a README file in English

Hey hie, The project you uploaded seems interesting and I would love to contribute to it. So can we have a README in English language so that It will be easy for everyone to have a look at it.

Thanks in advance :)