Giter Club home page Giter Club logo

bookingscraper's Introduction

Build Status Coverage Status Codacy Badge BCH compliance Known Vulnerabilities Total alerts Language grade: Python Run on Repl.it

Booking site web scraper

Web scraper that downloads all of the accommodations for the chosen country and saves them in a file.

Installation

Create virtual environment and run:

pip install -r requirements.txt

After that just run booking script:

python booking.py

Autocompletion

If you want to use the feature of the autocompletion run:

activate-global-python-argcomplete

After that open new terminal in order to update and load new settings.

TBD

  • Add option for choosing a time frame.
  • Add option for choosing the number of rooms.

Disclaimer

Data fetched from booking is only for personal use, you are not allowed to copy and paste content from Booking.com on to your own or third party pages (including social media pages such as Facebook, Twitter, Instagram etc.).

This applies to all types of content that can be found on Booking.com pages, including but not limited to hotel descriptions, reviews, hotel and room photos, hotel facility information, and prices. Moreover, this restriction also applies to content from Booking.com partner hotel websites and Booking Holdings Group company brands: such as Agoda, Priceline, Kayak, OpenTable, Rentalcars.

Clause 4.1.5 from Booking.com Affiliate Agreement: The Affiliate shall not programmatically evaluate and extract information (including guest reviews) from any part of the Booking.com Website (e.g. screen scrape).

bookingscraper's People

Contributors

aisyram avatar belvenr avatar dariodsa avatar knowa42 avatar mrvnmchm avatar renovate-bot avatar trk9001 avatar tunedmidja avatar zoranpandovski avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bookingscraper's Issues

Track more data points

Right now the scraper just gets names of the accommodations.

Would it be possible to start tracking more data points? E.g. price per night, amenities, location, ratings, reviews etc.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

pip_requirements
requirements.txt
  • openpyxl ==2.6.2
  • argcomplete ==1.12.2

  • Check this box to trigger a request for Renovate to run again on this repository

Help wanted

Congratulation for the nice work.
I was wondering how can it be added the city to the searching arguments?
Thank you in advance.

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Bugged

/Documents/GitHub/BookingScraper/booking.py' && echo Exit status: $? && exit 1
Traceback (most recent call last):
File "/Documents/GitHub/BookingScraper/booking.py", line 214, in
get_data(args.rooms, args.country, args.out_format, args.startdate, args.enddate)
File "/Documents/GitHub/BookingScraper/booking.py", line 152, in get_data
hotels_list: list = prep_data(rooms, country, out_format, start_date, end_date)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/pascal/Documents/GitHub/BookingScraper/booking.py", line 122, in prep_data
parsed_html.find_all("li", {"class": "sr_pagination_item"})[-1]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^
IndexError: list index out of range

TypeError: 'NoneType' object is not subscriptable

Hi there, Excellent work, thanks a lot!
I face an issue when running it though:

The error is:

Exception in thread Thread-18:
Traceback (most recent call last):
  File "C:\Anaconda3\lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\Anaconda3\Projects\Scraper\Booking.com\bthread.py", line 15, in run
    self.process_hotels(self.session, self.offset, self.rooms, self.country)
  File "booking.py", line 56, in process_hotels
    **name = ho.find('a', {'class': 'jq_tooltip'})['title']**
  File "C:\Anaconda3\lib\site-packages\bs4\element.py", line 1071, in __getitem__
    return self.attrs[key]
KeyError: 'title'

This error is thrown out 11 times, for different threads.
The line throwing the error seems to be: name = ho.find('a', {'class': 'jq_tooltip'})['title']

I ran the script booking.py
It produces however a file with a list of hotels in the country.

Add tests

We don't have any tests. Contributions are welcome

Following the setup instructions leads to error

Traceback (most recent call last):
  File "booking.py", line 214, in <module>
    get_data(args.rooms, args.country, args.out_format, args.startdate, args.enddate)
  File "booking.py", line 153, in get_data
    save_data(hotels_list, out_format=out_format, country=country)
  File "booking.py", line 164, in save_data
    file = writer.output_file()
  File "C:\Projects\bookingscraper\file_writer.py", line 22, in output_file
    json.dump(list(self.data), outfile, indent=2, ensure_ascii=False)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\json\__init__.py", line 180, in dump
    fp.write(chunk)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u01c0' in position 23: character maps to <undefined>

Got this after python booking.py. Does it require any arguments or config changes?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.