该项目由是对网站Bangumi 番组计划 网站的爬虫
- Python3.6
- Scrapy
- MongoDB
爬取动画，音乐等信息时会下载其封面
bangumi_anime_id爬取需要的Animate(动画)的IDbangumi_music_id爬取需要的Music（音乐）的IDbangumi_book_id爬取需要的Book（书籍）的IDbangumi_person_id爬取需要的Person（三次元人物）的IDbangumi_character_id爬取需要的Character（二次元人物）的IDbangumi_game_id爬取需要的Game（游戏）的IDbangumi_animate爬取需要的Animate(动画)bangumi_music爬取需要的Music（音乐）bangumi_book爬取需要的Book（书籍）bangumi_person爬取需要的Person（三次元人物）bangumi_character爬取需要的Character（二次元人物）bangumi_game爬取需要的Game（游戏）
添加了docker支持，具体请查看配置文件。在请手动容器（spider）中执行相应的命令即可。
/data目录储存MongoDB数据，/download储存下载的图片
- 请在init中配置MongoDB相关的连接数据
- 请在setting中配置图片下载路径
- 请先爬取ID才能爬取具体内容，例如先爬取动画ID才能爬取具体内容
已实现简单的爬虫数据API，详见Bangumi-Spider-API
- 网页端控制爬虫。
MIT License
Copyright (c) 2019 AllenTom aka TakayamaAren
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent