emptysuns / 18comic-download Goto Github PK
View Code? Open in Web Editor NEWpython爬虫自动多线程下载禁漫天堂漫画
18comic-download's People
Contributors
Stargazers
Watchers
18comic-download's Issues
单章页数过多会出错
大佬，又是我。哈哈哈。
- 单章超过一定页数（暂时知道是490~500）会出BUG，详细见下面的记录。推测是页数过多，导致线程爆炸，cpu爆了。建议超过一定数量，线程做分批处理。
下面是建议（仅仅是个人的一些想法，勿怪，伸手党本不应该这么多要求）：
2. 下载路径建议放在exe同层目录下download文件夹。如果改成由用户自己输入则每次都需要输入，这十分麻烦。所以同层目录下比较好。
3. 某些漫画可能已经完成了（进入文件夹查看），但是cmd中一直是显示“正在开始多线程下载，请稍后.....”，等了很久也是没有反应的，并不会提示“由于不可抗网络因素下载失败，重新下载”。当然这个只要重新整个下载就可以了。是否需要增加一个超时重新连接？
4. 章节重复问题，如果一个漫画有20多话，下载到10话的时候卡住了，比如上诉的3的情况。这时候只能A，全部重新下载。B手动一个个加地址下载后10个。能否增加话数中间选择？
5. 18comic似乎有流量检测，短时间大流量下载会导致IP封禁一段时间（限速，限连）。此时重新连接就会一直出错。是否能自动走其他分流线路，比如18comic.org不行，就走18comic.vip，18comic1.one，18comic2.one？
第1次下载,请输入您想要下载comic的下载链接:
https://18comic.org/photo/148490
解析成功,开始下载 https://18comic.org/photo/148490
成功创建目录 ./download/深夜便利店 03
正在开始多线程下载（线程数量:495)请稍后......
Exception in thread DownloadPageIs71:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs386:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs351:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs338:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs310:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs67:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs286:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs22:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Traceback (most recent call last):
Exception in thread DownloadPageIs285:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs281:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs58:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 264, in load
b = b + s
MemoryError
Exception in thread DownloadPageIs82:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2164, in save
save_handler(self, fp, filename)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\JpegImagePlugin.py", line 761, in _save
ImageFile._save(im, fp, [("jpeg", (0, 0) + im.size, 0, rawmode)], bufsize)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 542, in _save
s = e.encode_to_file(fh, bufsize)
MemoryError
Exception in thread DownloadPageIs257:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 264, in load
b = b + s
MemoryError
Exception in thread DownloadPageIs109:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 264, in load
b = b + s
MemoryError
Exception in thread DownloadPageIs289:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 247, in load
s = read(self.decodermaxblock)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\JpegImagePlugin.py", line 399, in load_read
s = self.fp.read(read_bytes)
MemoryError
Exception in thread DownloadPageIs173:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 264, in load
b = b + s
MemoryError
Exception in thread DownloadPageIs19:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs194:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 247, in load
s = read(self.decodermaxblock)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\JpegImagePlugin.py", line 399, in load_read
s = self.fp.read(read_bytes)
MemoryError
Exception in thread DownloadPageIs393:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 247, in load
s = read(self.decodermaxblock)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\JpegImagePlugin.py", line 399, in load_read
s = self.fp.read(read_bytes)
MemoryError
Exception in thread DownloadPageIs165:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs177:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 264, in load
b = b + s
MemoryError
Exception in thread DownloadPageIs50:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2164, in save
save_handler(self, fp, filename)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\JpegImagePlugin.py", line 761, in _save
ImageFile._save(im, fp, [("jpeg", (0, 0) + im.size, 0, rawmode)], bufsize)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 542, in _save
s = e.encode_to_file(fh, bufsize)
MemoryError
Exception in thread DownloadPageIs334:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 247, in load
s = read(self.decodermaxblock)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\JpegImagePlugin.py", line 399, in load_read
s = self.fp.read(read_bytes)
MemoryError
Exception in thread DownloadPageIs188:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs344:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs263:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs214:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs398:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs322:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
File "H:\新建文件夹\Catch18comic-1.py", line 217, in
main(id)
Exception in thread DownloadPageIs406:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs133:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs361:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs335:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs220:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs227:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs40:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 104, in download_image
image_bytes = BytesIO(comic_page.content)
UnboundLocalError: local variable 'comic_page' referenced before assignment
Exception in thread DownloadPageIs211:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2164, in save
save_handler(self, fp, filename)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\JpegImagePlugin.py", line 761, in _save
ImageFile._save(im, fp, [("jpeg", (0, 0) + im.size, 0, rawmode)], bufsize)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 542, in _save
s = e.encode_to_file(fh, bufsize)
MemoryError
Exception in thread DownloadPageIs305:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 264, in load
b = b + s
MemoryError
Exception in thread DownloadPageIs306:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in
_bootstrap_inner
self.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in
run
self._target(*self._args, **self._kwargs)
File "H:\新建文件夹\Catch18comic-1.py", line 107, in download_image
image_source.save(comic_local_position)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 2130, in save
self._ensure_mutable()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 619, in _ensure_mutable
self._copy()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL\Image.py",
line 612, in _copy
self.load()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\site-packages\PIL
\ImageFile.py", line 264, in load
b = b + s
MemoryError
File "H:\新建文件夹\Catch18comic-1.py", line 176, in main
downloadByThread(comic_num, url_path_list)
File "H:\新建文件夹\Catch18comic-1.py", line 155, in downloadByThread
thread.start() # 开始线程
RuntimeError: can't start new thread
发现图片序号不连续的情况
图集编号：105924
共计121张图片。前120的编号都是连续的，最后一张的文件名是122，没有121。
于是按照序号推算图片名字，下载到121时会一直404错误。
感觉直接读取文件名太慢了，遇到那种10+页的要依次读取每一个网页。
我现在想到有三个方案：
- 不推算序号，直接读取每个图片的文件名。
- 失败次数超过阈值时，读取每个图片文件名
- 失败次数超过阈值是，查看那个失败文件的名字是否存在，否的话总数序号+1
我想请教一下作者是如何将错位的图片拼接正确的
在2.2版本中，使用的convertImg方法已经不能再适配新的规则了，新的规则貌似不同的图片错位程度不同，那么请问作者您是如何从网站的源码中分析出它的拼接规则，我试图重它的源码中找到拼接成canvas的方法，但水品有限，又试着直接将canvas转成image，但是它被污染且无法跨域。
关于下载线程数是否需要设定最大值的讨论
现在的动态线程，是有多少图片（pages），就启用多少线程。之前的一个图集，7000多个图片。虽然非高峰期下载并没有什么问题，不过我在想，可能设置一个上限比较好。毕竟网页浏览时，一个页面最多500个图片，而且也不是全部一次性载入，可能也就载入浏览位置的20个图片左右。
那么，如果设置上限的话，线程怎么管理，这方面我完全没有经验。那些下载软件那种总数控制、独立超时统计、还可以统计即时下载速度的功能不知道怎么实现。
- 没有上限的优点：
- 网速跑满
- 没有上限的缺点：
- 线程太多可能被服务器拉黑
- 现在的线程处理是有join()的，高峰期线程越多，阻塞主线程造成假死的概率越大（我用了requests.get中的timeout参数好像没有效果）
- 有上限的优点：
- 防止被服务器拉黑
- 响应更快，不容易卡死
- 可以做个进度条更直观
- 有上限的缺点：
1.下载速度可能变慢，特别是高峰期
连接尝试失败
本人小白，请见谅。配置好python后，打开Catch18comic.py，输入网址解析成功开始就显示”由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。”
- 是否是18comic进行了反扒？地址如下 https://18comic2.one/photo/239508/
- 在win32 下打包成exe ，提示“文件未找到cacert.pem resource not found in pip._vendor” 哪里错了吗？环境为win7-32 sp1，python-3.8.7，pip 21.0.1，setuptool 49.2.1，xml已安装
- 依赖库方面，beautifulsoup已经升级为beautifulsoup4，PIL已升级为pillow。
图片错乱
有问题请教这个图片错位问题！！！我看了大部分的图片，，不能10分切割，，有的是2有的6有的10有的20！！无法统一，这个应该怎么解决
这个脚本很好用
不过有个问题就是假如是多集的本子就不能一次过下到完
我试了下韩漫只下载第一章
不过整体效果很好跑超快
能下载 新的问题在于 下后再达到第五集时会出现闪退亦或者卡住error的情况
试了几下还是一样
缺点在于韩漫只能下载一个而不是一套的，通常下载后都是下载最前面的第一集
缺点在于韩漫只能下载一个而不是一套的，通常下载后都是下载最前面的第一集
是不是要处理反爬啊，比如这个漫画 227153
id: 227153
我觉得应该另外写个用于整理已下载图集的模块
（我一个朋友）在下载文件过多时，特别是那种有很多集的连载漫画，因为文件夹的名字比较混乱，找起来比较困难。
现在我已经写了一个方法，在连载漫画的全集下载时，会自动在每个文件夹下生成一个index.html，并通过目录链接相互关联。
但是在此之外，如何从众多文件夹中定位到自己找的那一部、那一集，是还没解决的问题。
可以在写一个模块，实现三个功能：
- 搜索所有下载文件夹，在目录下生成index.html （通过搜索结果单独实现，而不依赖于下载中的记录）
- download下生成一个具有书架功能的index.html，加入所有漫画，并自动链接到第一集的index.html
- 记录阅读进度，支持按进度直接跳转到指定位置。
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.