Giter Club home page Giter Club logo

rent591_crawler's Introduction

rent591 scrapy web scrawler

install package

pip -r install requirements.txt

set up mongoDB (need docker/docker-compose before install)

> cd db
> docker-compose up -d

start scrapy web crawler

> cd crawlers/rent_591
> scrapy crawl rent591

Restful API format

get /rent?<parameter1=value>&<parameter2=value>...&<parameterN=value>

API parameter

'sex_limit':    number (0: male | 1: female | 2: no limit)
'phone_number': string (09XX-XXX-XXX)
'city':         string (台北市 | 新北市)
'renter_sex':   number (0: male | 1: female)
'home_owner':   number (0: not home owner | 1: home owner)
'first_name':   string (張,黃....else)

API response

{
    "data": [
        {
            "city": "台北市",
            "house_id": 7227694,
            "house_recent": "分租套房",
            "house_type": "公寓",
            "phone_number": "0987-097-311",
            "renter": "詹小姐",
            "renter_sex": 1,
            "renter_type": "屋主",
            "sex_limit": 2
        },...
    ]
}

reference

  1. https://ithelp.ithome.com.tw/articles/10191506

rent591_crawler's People

Contributors

mark1002 avatar

Stargazers

Yu avatar  avatar  avatar M.C. Shih avatar Cuda Chen avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.