Giter Club home page Giter Club logo

xjhr_scrapy's Introduction

xjhr_scrapy

前言

​ 之前在上人才网的时候,下载自己的简历,忽然间想,我是不是可以下载别人的,于是尝试了一下,没想到真的可以。于是就写了这个极简版的爬虫,因为是刚学python所有很多写的都很生涩和初级。

项目介绍

​ 项目是基于 python3.6需要下载包BeautifulSoup。其他的都是内置包了。

参数讲解

网站参数
  • Degree(学历):0(不限),10(初中),20(高中),30(中技),40(中专),50(大专),60(本科),70(硕士),80(博士)
  • Age(年龄):0(不限),1(16-20),2(21-25),3(26-30),4(31-35),5(36-40),6(41-50),7(51-60),8(25岁以上),9(30岁以上),10(35岁以上)
  • WorkYears(工作年限):0(不限),102(在校学生),101(应届生),1(1年以上),2(2年以上),3(3年以上),5(5年以上),8(8年以上),10(10年以上)
  • Sex(性别):0(不限),1(男),2(女)
  • JobProperty(职业性质):0(不限),1(全职),2(兼职),3(临时),4(实习)
  • PublishDate(更新时间):0(不限),1(1天内),2(2天内),3(3天内),7(7天内),15(15天内),30(30天内),60(60天内),90(90天内),180(180天内),365(365天内)
  • Orderid(排序类型):0(更新时间),1(发布时间),2(热度)
  • WorkPlace(当前所在地):太多,请至网站获取,可空
  • Key(关键字):自由填写,可空
  • JobType(岗位类别):太多,请至网站获取,默认0不限
自定义参数
  • page_num(获取页数):默认为0即获取到的最大页数

  • isMustPhoto(是否必须照片):默认false,如果true,则只下载有照片的

  • savePath(保存路径):参数形式为r'E:\rc',其中r不可缺少

使用方法

​ 修改源文件参数,有默认参数,可以不传参,也可以自己定制

​ 传参示例a.customization(sex=2,age=2,isMustPhoto=True,publishDate=60)

​ 运行即可

不足之处

​ 因为是初学,所以爬虫的健壮性还没有考虑,也没有做太多限制,包括传参什么的还很简单,之后会继续改进

警告

此代码只是用来研究学习,不要下载文档做非法用途,如若不听,后果自负!

xjhr_scrapy's People

Contributors

s554097550 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.