lta-rs / lta-rs Goto Github PK

View Code? Open in Web Editor NEW

19.0 3.0 5.0 2 MB

🚍 Singapore LTA Datamall Rust Client

Home Page: https://lta-rs.github.io/lta-rs/lta/

License: MIT License

Rust 100.00%

lta rust datamall singapore public-transportation async-first

lta-rs's Introduction

lta-rs

🚍 Singapore LTA Datamall async first Rust client. lta-rs is used to interact with lta-datamall

lta-rs in action

Cargo.toml setup

[dependencies]
lta = { version = "0.7.0" }

API key setup

You can get your API key from here

use lta::{LTAResult, LTAClient, Client, Traffic, TrafficRequests};

#[tokio::main]
async fn main() -> LTAResult<()> {
    let api_key = std::env::var("API_KEY").expect("API_KEY not found!");
    let client = LTAClient::with_api_key(api_key)?;
    let erp_rates = Traffic::get_erp_rates(&client, None).await?;
    println!("{:?}", erp_rates);
    Ok(())
}

Feature flags

Feature	Description
`default` (i.e no features added )	Uses `reqwest` under the hood
`reqwest-blocking`	Uses `reqwest::blocking` under the hood
`ureq-blocking`	Uses `ureq` under the hood
`fastfloat`	Enables the `fastfloat` impl for parsing floats (uses SIMD)
`non-blocking-traits`	Exports traits that can be use to impl non-blocking clients
`blocking-traits`	Exports traits that can be use to impl blocking clients

Feature flags examples

Using ureq only

[dependencies]
lta = { version = "0.7.0", default-features = false, features = ["ureq-blocking"]}

Implementing another blocking backend

[dependencies]
lta = { version = "0.7.0", default-features = false, features = ["blocking-traits"]}

Implementing another async backend

[dependencies]
lta = { version = "0.7.0", default-features = false, features = ["non-blocking-traits"]}

Backend Support

Backend	Status	Github CI Run
reqwest	Official ✔	Yes ✔ ️
reqwest blocking	Official ✔ ️	Yes ✔️
ureq	Official ✔ ️	Yes ✔️
surf	TBA ⭕ ️	No ⭕

Examples

Example	Description
bus_timing.rs	How to get bus timing (async used)
reqwest_blocking.rs	How to use reqwest blocking feature
ureq_blocking.rs	How to use ureq backend
custom_client.rs	How to create custom backend clients

General advice

Reuse LTAClient<T> as it holds a connection pool internally
Reduce the number of times you call the API, take a look at Update Freq in the documentation and prevent yourself from getting blacklisted. Use a caching mechanism.

Getting help

You can get help via GitHub issues. I will try my best to respond to your queries 😄

Changelog

Changelog can be found here

Requirements

Rust compiler 1.56

Frequently Asked Questions

Q: Is this library being actively developed?

A: Project is currently in maintenance mode. Won't really have any new features. Just bug fixes, minor upgrades etc.

Q: What are the APIs available?

A: All of the APIs are implemented. Take a look at the official LTA docs.

Q: Where do I get the official docs from lta?

A: You can get them here

License

lta-rs is licensed under MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

Donations

For Singapore based users, you can donate using paylah!

lta-rs's People

Contributors

Stargazers

Watchers

Forkers

euwbah shuimeihe tayweichong 00mjk lovung

lta-rs's Issues

Ergonomic upgrades for `skip` parameters

Instead of taking Option<u32> in this case

async fn get_taxi_avail(client: &LTAClient, skip: Option<u32>) -> LTAResult<Vec<Coordinates>> {
    build_req_with_skip::<TaxiAvailResp, _, _>(client, api_url!("/Taxi-Availability"), skip).await
}

We can change the skip to impl Into<Option<u32>>.

async fn get_taxi_avail(client: &LTAClient, skip: impl Into<Option<u32>>) -> LTAResult<Vec<Coordinates>> {
    build_req_with_skip::<TaxiAvailResp, _, _>(client, api_url!("/Taxi-Availability"), skip).await
}

This allows you to call it like this.

let _ = get_taxi_avail(&client, 500);
let _ = get_taxi_avail(&client, None);

Currently the impl are seperated into 2 diff mods. 1 ser and 1 de. After looking into serde even more, it seems that the recommended way of doing serde is to have it in 1 mod which contains both serde fns.

Things to take note.

Serde of chrono Example
NextBus Example

Possibly move to Azure Pipelines

For compilation for Windows and Mac

Optional Params

Some of the APIs have optional parameters and this should be reflected in the lta-rs as well.

Affected APIs:

Bus Arrival ( Optional service_no )
Passenger Volume By Origin Destination ( Optional date )
Bicycle Parking ( Optional dist )

Implmentation

Just use Option<T>, types that require Date should use chrono types, which means chrono will be re-exported so that the user don't have to add chrono to their dep and prevent incompat.

New traffic flow API and revised traffic speed bands

See https://datamall.lta.gov.sg/content/dam/datamall/datasets/LTA_DataMall_API_User_Guide.pdf

BusServices failed to parse due to interger overflow

Currently for bus 36B the following is returned

        {
            "ServiceNo": "36B",
            "Operator": "GAS",
            "Direction": 1,
            "Category": "TRUNK",
            "OriginCode": "93049",
            "DestinationCode": "93199",
            "AM_Peak_Freq": "08-08",
            "AM_Offpeak_Freq": "08-288",
            "PM_Peak_Freq": "08-135",
            "PM_Offpeak_Freq": "-",
            "LoopDesc": ""
        },

288 is out of range of u8. However, we should not change our implementation to >u8 because this is a typo and can be checked here

The original json is the same except 288. Instead of 288, its 28

Move image to another host

Image in documentation is in base64 right not and image in crate is adding to the size of the library

Global Config

After thinking about it for some time, I think it's time to remove the global config struct and let the user create their own

I am getting Error EOF while parsing a value

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: Json(Error("EOF while parsing a value", line: 1, column: 0)), url: None }', src\libcore\result.rs:1009:5
stack backtrace:
   0: std::sys::windows::backtrace::set_frames
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
   1: std::sys::windows::backtrace::set_frames
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
   2: std::sys::windows::backtrace::set_frames
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
   3: std::sys_common::backtrace::print
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys_common\backtrace.rs:59
   4: std::sys_common::backtrace::print
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys_common\backtrace.rs:59
   5: std::panicking::default_hook
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:227
   6: std::panicking::rust_panic_with_hook
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:491
   7: std::panicking::continue_panic_fmt
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:398
   8: std::panicking::rust_begin_panic
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:325
   9: core::panicking::panic_fmt
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libcore\panicking.rs:95
  10: core::result::unwrap_failed<reqwest::error::Error>
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libcore\macros.rs:26
  11: core::result::Result<lta::bus::bus_stops::BusStopsResp, reqwest::error::Error>::unwrap<lta::bus::bus_stops::BusStopsResp,reqwest::error::Error>
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libcore\result.rs:808
  12: lta::bus::get_bus_stops
             at C:\Users\zeon000\.cargo\git\checkouts\lta-rs-f6ada22c85a9129d\add9969\src\bus.rs:331
  13: cratetest::main
             at .\src\main.rs:5
  14: std::rt::lang_start::{{closure}}<()>
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\rt.rs:74
  15: std::rt::lang_start_internal::{{closure}}
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\rt.rs:59
  16: std::rt::lang_start_internal::{{closure}}
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\rt.rs:59
  17: panic_unwind::__rust_maybe_catch_panic
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libpanic_unwind\lib.rs:102
  18: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:289
  19: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:289
  20: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:289
  21: std::rt::lang_start<()>
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\rt.rs:74
  22: main
  23: invoke_main
             at d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
  24: invoke_main
             at d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
  25: BaseThreadInitThunk
  26: RtlUserThreadStart
error: process didn't exit successfully: `target\debug\cratetest.exe` (exit code: 101)

Code

fn main() {
    match lta::bus::get_bus_stops() {
        Ok(t) => println!("{:?}", t),
        Err(e) => println!("{:?}", e)
    }

    match lta::bus::get_bus_services() {
        Ok(t) => println!("{:?}", t),
        Err(e) => println!("{:?}", e)
    }
}

Difficulty testing if new feature is added

Since the main lib depends on the the sub libs ie lta_models, lta_utils_commons, lta_async and lta_blocking, it gets unnecessarily difficult to test on the main lib. To solve this problem we should move the testing to the individual crates

Broken get_bus_arrival

Sometimes the bus_arrival response returns a json with some of the fields empty and from_str breaks. Currently still investigating the cause as it happens rarely.

Wrong implementation of async get_bike_parking

Implementation of async get_bike_parking does not take in any params at all! Params should follow sync version

Typo for `FacilityReqeusts` trait

No such known host

Stacktrace

Error { kind: Hyper(Error { kind: Connect, cause: Os { code: 11001, kind: Other, message: 
"No such host is known." } }), 
url: Some("http://datamall2.mytransport.sg/ltaodataservice/BusArrivalv2?BusStopCode=83139&ServiceNo=15") }

Legitimate key
No other errors

Wrong error returned when failed to parse body

Looking at this run, it seems that the body failed to parse but LTAErrror::BackendError is returned.

I am getting API key not init!

Stacktrace

thread 'tests::get_arrivals' panicked at 'API key not init!', src\client_config.rs:34:21
stack backtrace:
   0: std::sys::windows::backtrace::set_frames
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
   1: std::sys::windows::backtrace::set_frames
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
   2: std::sys::windows::backtrace::set_frames
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
   3: std::sys_common::backtrace::print
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys_common\backtrace.rs:59
   4: std::sys_common::backtrace::print
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys_common\backtrace.rs:59
   5: std::panicking::default_hook
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:221
   6: std::panicking::rust_panic_with_hook
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:491
   7: std::panicking::begin_panic<str*>
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:425
   8: lta::client_config::ClientConfig::get_req_builder
             at .\src\client_config.rs:34
   9: lta::bus::get_arrival
             at .\src\bus.rs:121
  10: lta::tests::get_arrivals
             at .\src\lib.rs:28
  11: lta::tests::get_arrivals::{{closure}}
             at .\src\lib.rs:26
  12: core::ops::function::FnOnce::call_once<closure,()>
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libcore\ops\function.rs:238
  13: test::run_test::{{closure}}
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libtest\lib.rs:1454
  14: test::run_test::{{closure}}
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libtest\lib.rs:1454
  15: test::run_test::{{closure}}
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libtest\lib.rs:1454
  16: panic_unwind::__rust_maybe_catch_panic
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libpanic_unwind\lib.rs:102
  17: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
  18: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
  19: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
  20: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
  21: std::thread::{{impl}}::spawn_unchecked::{{closure}}::{{closure}}
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\thread\mod.rs:477
  22: std::thread::{{impl}}::spawn_unchecked::{{closure}}::{{closure}}
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\thread\mod.rs:477
  23: std::thread::{{impl}}::spawn_unchecked::{{closure}}::{{closure}}
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\thread\mod.rs:477
  24: panic_unwind::__rust_maybe_catch_panic
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libpanic_unwind\lib.rs:102
  25: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
  26: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
  27: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
  28: std::panicking::try
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
  29: alloc::boxed::{{impl}}::call_once
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\liballoc\boxed.rs:683
  30: alloc::boxed::{{impl}}::call_once
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\liballoc\boxed.rs:683
  31: alloc::boxed::{{impl}}::call_once
             at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\liballoc\boxed.rs:683
  32: BaseThreadInitThunk
  33: RtlUserThreadStart

Code

fn main() {
    match lta::bus::get_arrival(83139, "15") {
        Ok(t) => println!("{:?}", t),
        Err(e) => println!("{:?}", e)
    }
}

New Platform Crowd Density API

From datamall docs

2 New Platform Crowd Density APIs are launched! These two APIs
return real-time and forecasted platform crowdedness information for
the MRT/LRT stations of a train network line.

BusService struct causes panic due to broken data

Parsing updated bus_services.json will fail as there is empty string data that is parsed to a u32

Only 500 records per request

From LTA documention

With the exception ofthe following Bus Arrival APIlisted below(see Table 1), API responses returned are limited to 500records of the dataset per call.This numbermay be adjusted from time to time. To retrieve subsequentrecordsof the dataset, you need to append the $skip operatorto the API call (URL).For example, to retrieve the next 500records (501stto the 1000th), the API callshould be:http://datamall2.mytransport.sg/ltaodataservice/BusRoutes?$skip=500
To retrieve the following set of 500 records, append ‘?$skip=1000’, and so on. Just remember, each URL call returns only a max of 500 records!

This will be a huge breaking change since there is an additional param to be added to most function. Really sorry about this, totally overlooked this restriction

Regarding this issue, I will try to find ways to prevent breakage of public APIs

Bus

BusRoutes: 26000 skips
BusStops: 5000 skips
BusServices: 500 skips

Traffic

ErpRate: 2000 skips
VMS: None
Carpark Avail: 2000 skips
TrafficIncidents: None
TrafficSpeedBand: 59000 skips
EstTravelTime: None
FaultyTrafficLight: None
RoadDetails (Road Works): 6500 skips
RoadDetails (Road Opening): 500 skips
TrafficImages: None
BikeParking: None, more than 500 records returned

Train

TrainServiceAlert: None

Crowd

All: None

Taxi

TaxiAvail: 2500 skips
TaxiStands: None, however adding skips does not change the data returned

Imrpove BusRoute ser/de performance

de_route                time:   [909.66 us 916.22 us 923.17 us]                     
                        change: [-13.582% -12.257% -10.988%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  10 (10.00%) high mild
  4 (4.00%) high severe

Benchmarking ser_route: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.6s or reduce sample count to 50

ser_route               time:   [1.4571 ms 1.4629 ms 1.4700 ms]                       
                        change: [-6.6361% -6.1160% -5.6175%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  7 (7.00%) high mild
  2 (2.00%) high severe

Current impl

    pub mod str_time_option {
        use chrono::NaiveTime;
        use serde::{Deserialize, Deserializer, Serializer};

        const FORMAT: &str = "%H:%M:%S";

        pub fn serialize<S>(opt_time: &Option<NaiveTime>, serializer: S) -> Result<S::Ok, S::Error>
        where
            S: Serializer,
        {
            match opt_time {
                Some(time) => {
                    let s = format!("{}", time.format(FORMAT));
                    serializer.serialize_str(&s)
                }
                None => serializer.serialize_none(),
            }
        }

        pub fn deserialize<'de, D>(deserializer: D) -> Result<Option<NaiveTime>, D::Error>
        where
            D: Deserializer<'de>,
        {
            let s: String = String::deserialize(deserializer)?;
            if s.eq("-") {
                return Ok(None);
            }

            let time_res = NaiveTime::parse_from_str(&s, FORMAT);

            match time_res {
                Ok(r) => Ok(Some(r)),
                Err(_) => {
                    let chars = s.chars();
                    let hr: String = chars.clone().take(2).collect();
                    let min: String = chars.skip(3).take(4).collect();
                    let mut hr_u32: u32 = hr.parse().unwrap();
                    let min_u32: u32 = min.parse().unwrap();
                    if hr_u32 == 24 {
                        hr_u32 = 0
                    }
                    let time = NaiveTime::from_hms(hr_u32, min_u32, 0);
                    Ok(Some(time))
                }
            }
        }
    }

Enum deserialization may panic

Current implementations of enums does not have a default type if a new variant is introduced by LTA and will panic

r#async::tests::get_cp_avail fails on latest commit

It looks like the failure is caused by

failures:
340

341
---- r#async::tests::get_cp_avail stdout ----
342
Error: BackendError(reqwest::Error { kind: Decode, source: Error("invalid value: integer `-46`, expected u32", line: 1, column: 3544) })
343
thread 'r#async::tests::get_cp_avail' panicked at 'assertion failed: `(left == right)`
344
  left: `1`,
345
 right: `0`: the test returned a termination value with a non-zero status code (1) which indicates a failure', /rustc/7abab1efb21617ba6845fa86328dffa16cfcf1dc/library/test/src/lib.rs:195:5
346
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Have tests that does not require actual API to be alive

Allow for custom URLs

Blocking traits
Async traits

Unergonomic usage of `Bus::get_arrival`

In #30 we added the type parameter S: Into<Option<u32>> to improve the ergonomics of skip. However, Bus::get_arrival also got updated to include S: Into<Option<A>>, A: AsRef<str>. On first glance this seems like a good idea since we should be able to pass in any data type that is A: AsRef<str>, however based on justbus-rs usage, this is unergonomic.

let arrivals = Bus::get_arrival::<_, &str>(&client, bus_stop, None).await?;

Ideally, we should NOT have to provide &str typeparameter, so instead we should just S: Into<Option<&str>>

Feature: Allow for multiple clients

Currently, there is only 1 client available for users to use. There are some use cases where a user might want to have multiple clients with various different API keys

Current implementation

lazy_static! {
    pub static ref CLIENT_CONFIG: Mutex<ClientConfig> = Mutex::new(ClientConfig::new());
}

Planned implementation

Not tested and might not compile

lazy_static! {
    pub static ref CLIENT_CONFIG: Vec<Mutex<ClientConfig>> = vec![Mutex::new(ClientConfig::new())];
}

Wrong URL

pub async fn get_bus_routes(client: &LTAClient) -> Result<Vec<bus_routes::BusRoute>, LTAError> {
    build_req_async::<bus_routes::BusRouteResp, _>(client, bus_services::URL).await
}

URL should be bus_routes::URL

Fix badge

Compiling for `aarch64-unknown-linux-gnu` fails if target does not have OpenSSL installed

Looks like v0.5.0 fails to compile to aarch64-unknown-linux-gnu when target does not have OpenSSL installed. Since the requirement of OpenSSL has been removed in favour of rustls, this should not happen. We can simply change Cargo.toml to disable default features for reqwest.

reqwest = { version = "0.11", features = ["json", "rustls"], default-features = false}

LTA 1 April 2020 Update

Traffic images API update
Geospatial Whole Island API update
ZoneID update

Wrong function name for get_taxi_avail for async

Function declared as

/// Returns location coordinates of all Taxis that are currently available for
/// hire. Does not include "Hired" or "Busy" Taxis.
///
/// **Update freq**: 1min
pub async fn get_passenger_vol_by(client: &LTAClient) -> LTAResult<Vec<Coordinates>> {
    build_req_async::<TaxiAvailResp, _>(client, taxi_avail::URL).await
}

This is wrong and function name should be called get_taxi_avail.

Best regards,
Wei Chong