Giter Club home page Giter Club logo

busrouter-sg's Introduction

BusRouter SG - Singapore Bus Routes Explorer

🚌 πŸ—Ί πŸ‡ΈπŸ‡¬

Explore bus stops and routes on the map for all bus services in Singapore, with realtime bus arrival times and per-bus-stop passing routes overview.

Screenshot of BusRouter SG

Screenshot of BusRouter SG

✨ Complete Feature Set

  • Show full route with all bus stops for every bus service.
  • Show all bus stops when zoomed in.
  • Show all routes that pass through one bus stop.
  • Show bus arrival times for a bus stop.

Previously known as Singapore Bus Routes Explorer, abbreviated as 'SBRE' and previously looks like this:

Screenshot of Singapore Bus Routes Explorer

πŸ”– Story

This web app is first build in 2012, to fulfill my curiosity about buses. Ever since then, I've been maintaing the app and data until today.

πŸ‘‰ Read more about it: Building BusRouter SG .

πŸ›  Technicalities

Data

All data such as bus stops, services and routes are mostly scraped from http://mytransport.sg/, which means they are copyrighted by the Land Transport Authority.

Node.js is required to run the tasks to scrape data, massage the data, and run this web app.

Before running anything, run npm i to install all required dependencies.

Notes before diving in:

  • There are three important terms: stop, service and route.
  • One service have one or many routes, usually maximum two routes. It may be an A-to-B route or A-to-A route (roundtrip).
  • Services with two routes may contain A-to-B and B-to-A routes with different list of stops, usually on the opposite sides of the road. Some services may contain A-to-B and C-to-D routes where the first stop of a route may not be the last stop of the 2nd route, etc.

Before running the scripts, some environment variables are required. This project uses dotenv so one of the easier ways to add environment variables is:

  1. Duplicate .env.example file (copy & paste)
  2. Rename to .env
  3. Open the file and configure the variables

One of environment variables is ltaAccountKey which you'll need to request. The rest should be self-explanatory.

The scripts are:

  • node tasks/fetchServices
  • node tasks/fetchServiceStops
    • Loop through all bus services and scrape https://www.mytransport.sg/content/mytransport/ajax_lib/map_ajaxlib.getBusRouteByServiceId.{SERVICE_NUMBER}.html
    • Grabs a list of stops for every service
    • Generates serviceStops.json, a mapping of every service to stops, separated by routes.
    • Generates stops.json, a list of stops with names and coordinates
  • For route polylines (route lines of a service between all stops, containing just a list of coordinates), everything is scraped from their respective sources, ignoring dirty data, in best-effort ways:
    • MyTransport.SG: node tasks/fetchRoutesMyTransportSG - most reliable so far, but some lines are kind of weird and overlapping in wrong ways
    • OneMap.SG: node tasks/fetchRoutesOneMapSG
    • TowerTransit.SG: node tasks/fetchRoutesTowerTransitSG - pretty reliable source
    • Mapbox: node tasks/fetchRoutesMapboxAPI - using Mapbox Directions API to fill in the blanks, for services that don't have routes.
    • node tasks/exposeFaultyRoutes - for exposing routes that may be faulty, which is good for comparing the quality for all sources above.
  • For stops, everything is scraped from their respective sources too, to get stop names and (hopefully) accurate coordinates:
    • LTA DataMall: node tasks/fetchStopsLTA
    • OneMap.SG: node tasks/fetchStopsOneMap
    • OpenStreetMap: node tasks/fetchStopsOverPass - data queried via OverPass API, generated with Overpass Turbo
    • node tasks/repositionStops - Generates stops2.json with additional stop names and (hopefully) more accurate coordinates
    • node tasks/repositionStopLabels - Generates stops3.json with adjusted label positionings.
  • node tasks/fetchRoutesLTA
    • Fetches all routes with stop information for approximate arrival times for first and last buses.
    • The approximate arrival times are for every single stop. They are not the first bus that departs from first stop or last bus towards last stop. Every. Single. Stop.
    • Generates routes.lta.json, a mapping of all stops to services with approximate arrival times for first and last buses.
  • For generating GeoJSON files
    • node tasks/geojsonRoutes
    • node tasks/geojsonStops
  • For generating final JSON files to be consumed by the web app
    • node tasks/fetchAbbrs
    • node tasks/generateServices
      • Service names are dynamically generated from the stops
      • Generates services.final.json for usage in web app
    • node tasks/generateStops
      • Stop coordinates are rounded to 5-number precision
      • Generates stops.final.json for usage in web app
    • node tasks/generateRoutesPolyline
    • node tasks/generateFirstLast
      • Approximate arrival times for first and last buses for every service in every single stop
      • Generates firstlast.final.json for usage in web app
  • For fun
    • node tasks/trivia - Trivia questions with answers

Current version of the data is 3, all located under ./data/3 folder. All JSON files are "immutable", in the sense that all scripts above do not modify existing JSON files that are generated by other scripts. Every script is written to only create or modify its own JSON files and should not overwrite other JSON files.

Everything else are older versions of the data and being left there for legacy reasons.

When there's a data update, run the scripts in these order:

  1. node tasks/fetchServices
  2. node tasks/fetchServiceStops
  3. Stops data
    1. node tasks/fetchStopsOneMapSG
    2. node tasks/fetchStopsOverpass
    3. node tasks/repositionStops
    4. node tasks/repositionStopLabels
  4. Routes data
    1. node tasks/fetchRoutesMyTransportSG --override
      • Mutates/updates the routes data because of override option
      • Doesn't delete services/routes that are gone
    2. node tasks/fetchRoutesOneMapSG --override
      • Mutates/updates the routes data because of override option
      • Doesn't delete services/routes that are gone
    3. node tasks/fetchRoutesMapboxAPI
      • Usually not needed, unless there are routes that are missing
      • Doesn't delete services/routes that are gone
  5. First/last bus data
    1. node tasks/fetchRoutesLTA
  6. GeoJSON data (not used in web app)
    1. node tasks/geojsonStops
    2. node tasks/geojsonRoutes
  7. Finalized data (used in web app)
    1. node tasks/fetchAbbrs
      • To be used for next two scripts.
    2. node tasks/generateStops
    3. node tasks/generateServices
    4. node tasks/generateRoutesPolyline
    5. node tasks/generateFirstLast

Visualization

There's a separate visualization mini-site on /visualization/ URL.

The scripts to generate the data, in order:

  1. node visualization/build-routes
    • Reads stops2.json and transform routes data with "levels" for 3D extrusion
    • Generates visualization/data/routes.json for usage on mini-site and visualization/data/levels.json to be read by the build-stops script
  2. node visualization/build-stops
    • Reads stops.geojson and buffered into circle polygons which will be 3D-extruded
    • Generates visualization/data/stops.3d.json for usage on mini-site

Web App

The scripts for the web app:

  • npm start - start server for development
  • npm run build - build for production and deployment, in ./dist folder.

πŸ“œ License

Data Β© LTA Β© OneMap Β© OSM contributors. Everything else: MIT

🎀 Feedback

If you have any feedback, tweet me at @cheeaun.

πŸ™‡β€ Credits

busrouter-sg's People

Contributors

cheeaun avatar cpwc avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.