lta-rs / lta-rs Goto Github PK
View Code? Open in Web Editor NEW๐ Singapore LTA Datamall Rust Client
Home Page: https://lta-rs.github.io/lta-rs/lta/
License: MIT License
๐ Singapore LTA Datamall Rust Client
Home Page: https://lta-rs.github.io/lta-rs/lta/
License: MIT License
From LTA documention
With the exception ofthe following Bus Arrival APIlisted below(see Table 1), API responses returned are limited to 500records of the dataset per call.This numbermay be adjusted from time to time. To retrieve subsequentrecordsof the dataset, you need to append the $skip operatorto the API call (URL).For example, to retrieve the next 500records (501stto the 1000th), the API callshould be:http://datamall2.mytransport.sg/ltaodataservice/BusRoutes?$skip=500
To retrieve the following set of 500 records, append โ?$skip=1000โ, and so on. Just remember, each URL call returns only a max of 500 records!
This will be a huge breaking change since there is an additional param to be added to most function. Really sorry about this, totally overlooked this restriction
Regarding this issue, I will try to find ways to prevent breakage of public APIs
BusRoutes: 26000 skips
BusStops: 5000 skips
BusServices: 500 skips
ErpRate: 2000 skips
VMS: None
Carpark Avail: 2000 skips
TrafficIncidents: None
TrafficSpeedBand: 59000 skips
EstTravelTime: None
FaultyTrafficLight: None
RoadDetails (Road Works): 6500 skips
RoadDetails (Road Opening): 500 skips
TrafficImages: None
BikeParking: None, more than 500 records returned
TrainServiceAlert: None
All: None
TaxiAvail: 2500 skips
TaxiStands: None, however adding skips does not change the data returned
Current implementations of enums does not have a default type if a new variant is introduced by LTA and will panic
After thinking about it for some time, I think it's time to remove the global config struct and let the user create their own
Currently for bus 36B the following is returned
{
"ServiceNo": "36B",
"Operator": "GAS",
"Direction": 1,
"Category": "TRUNK",
"OriginCode": "93049",
"DestinationCode": "93199",
"AM_Peak_Freq": "08-08",
"AM_Offpeak_Freq": "08-288",
"PM_Peak_Freq": "08-135",
"PM_Offpeak_Freq": "-",
"LoopDesc": ""
},
288 is out of range of u8. However, we should not change our implementation to >u8 because this is a typo and can be checked here
The original json is the same except 288. Instead of 288, its 28
Stacktrace
thread 'tests::get_arrivals' panicked at 'API key not init!', src\client_config.rs:34:21
stack backtrace:
0: std::sys::windows::backtrace::set_frames
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
1: std::sys::windows::backtrace::set_frames
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
2: std::sys::windows::backtrace::set_frames
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
3: std::sys_common::backtrace::print
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys_common\backtrace.rs:59
4: std::sys_common::backtrace::print
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys_common\backtrace.rs:59
5: std::panicking::default_hook
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:221
6: std::panicking::rust_panic_with_hook
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:491
7: std::panicking::begin_panic<str*>
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:425
8: lta::client_config::ClientConfig::get_req_builder
at .\src\client_config.rs:34
9: lta::bus::get_arrival
at .\src\bus.rs:121
10: lta::tests::get_arrivals
at .\src\lib.rs:28
11: lta::tests::get_arrivals::{{closure}}
at .\src\lib.rs:26
12: core::ops::function::FnOnce::call_once<closure,()>
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libcore\ops\function.rs:238
13: test::run_test::{{closure}}
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libtest\lib.rs:1454
14: test::run_test::{{closure}}
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libtest\lib.rs:1454
15: test::run_test::{{closure}}
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libtest\lib.rs:1454
16: panic_unwind::__rust_maybe_catch_panic
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libpanic_unwind\lib.rs:102
17: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
18: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
19: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
20: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
21: std::thread::{{impl}}::spawn_unchecked::{{closure}}::{{closure}}
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\thread\mod.rs:477
22: std::thread::{{impl}}::spawn_unchecked::{{closure}}::{{closure}}
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\thread\mod.rs:477
23: std::thread::{{impl}}::spawn_unchecked::{{closure}}::{{closure}}
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\thread\mod.rs:477
24: panic_unwind::__rust_maybe_catch_panic
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libpanic_unwind\lib.rs:102
25: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
26: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
27: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
28: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\panicking.rs:289
29: alloc::boxed::{{impl}}::call_once
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\liballoc\boxed.rs:683
30: alloc::boxed::{{impl}}::call_once
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\liballoc\boxed.rs:683
31: alloc::boxed::{{impl}}::call_once
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\liballoc\boxed.rs:683
32: BaseThreadInitThunk
33: RtlUserThreadStart
Code
fn main() {
match lta::bus::get_arrival(83139, "15") {
Ok(t) => println!("{:?}", t),
Err(e) => println!("{:?}", e)
}
}
From datamall docs
2 New Platform Crowd Density APIs are launched! These two APIs
return real-time and forecasted platform crowdedness information for
the MRT/LRT stations of a train network line.
de_route time: [909.66 us 916.22 us 923.17 us]
change: [-13.582% -12.257% -10.988%] (p = 0.00 < 0.05)
Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
10 (10.00%) high mild
4 (4.00%) high severe
Benchmarking ser_route: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.6s or reduce sample count to 50
ser_route time: [1.4571 ms 1.4629 ms 1.4700 ms]
change: [-6.6361% -6.1160% -5.6175%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
7 (7.00%) high mild
2 (2.00%) high severe
Current impl
pub mod str_time_option {
use chrono::NaiveTime;
use serde::{Deserialize, Deserializer, Serializer};
const FORMAT: &str = "%H:%M:%S";
pub fn serialize<S>(opt_time: &Option<NaiveTime>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
match opt_time {
Some(time) => {
let s = format!("{}", time.format(FORMAT));
serializer.serialize_str(&s)
}
None => serializer.serialize_none(),
}
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<Option<NaiveTime>, D::Error>
where
D: Deserializer<'de>,
{
let s: String = String::deserialize(deserializer)?;
if s.eq("-") {
return Ok(None);
}
let time_res = NaiveTime::parse_from_str(&s, FORMAT);
match time_res {
Ok(r) => Ok(Some(r)),
Err(_) => {
let chars = s.chars();
let hr: String = chars.clone().take(2).collect();
let min: String = chars.skip(3).take(4).collect();
let mut hr_u32: u32 = hr.parse().unwrap();
let min_u32: u32 = min.parse().unwrap();
if hr_u32 == 24 {
hr_u32 = 0
}
let time = NaiveTime::from_hms(hr_u32, min_u32, 0);
Ok(Some(time))
}
}
}
}
We should separate the HTTP client by backends, and make the current one the default. For example, if no feature flag is used,
reqwest
will be used, and ureq
is used, it wont use reqwest
's blocking feature
reqwest-async
implreqwest-blocking
implureq-blocking
implSometimes the bus_arrival response returns a json with some of the fields empty and from_str breaks. Currently still investigating the cause as it happens rarely.
Currently, by the looks of it, lta_utils_commons
and lta_models
can be merged into one crate rather than 2 crate. Will take a look at it and see if there are any thing that will break if I merge those 2
For compilation for Windows and Mac
Test should panic rather than printing the error.
Some of the APIs have optional parameters and this should be reflected in the lta-rs as well.
Just use Option<T>
, types that require Date
should use chrono types, which means chrono will be re-exported so that the user don't have to add chrono to their dep and prevent incompat.
pub async fn get_bus_routes(client: <AClient) -> Result<Vec<bus_routes::BusRoute>, LTAError> {
build_req_async::<bus_routes::BusRouteResp, _>(client, bus_services::URL).await
}
URL should be bus_routes::URL
In #30 we added the type parameter S: Into<Option<u32>>
to improve the ergonomics of skip
. However, Bus::get_arrival
also got updated to include S: Into<Option<A>>, A: AsRef<str>
. On first glance this seems like a good idea since we should be able to pass in any data type that is A: AsRef<str>
, however based on justbus-rs usage, this is unergonomic.
let arrivals = Bus::get_arrival::<_, &str>(&client, bus_stop, None).await?;
Ideally, we should NOT have to provide &str
typeparameter, so instead we should just S: Into<Option<&str>>
Image in documentation is in base64 right not and image in crate is adding to the size of the library
Currently, there is only 1 client available for users to use. There are some use cases where a user might want to have multiple clients with various different API keys
lazy_static! {
pub static ref CLIENT_CONFIG: Mutex<ClientConfig> = Mutex::new(ClientConfig::new());
}
Not tested and might not compile
lazy_static! {
pub static ref CLIENT_CONFIG: Vec<Mutex<ClientConfig>> = vec![Mutex::new(ClientConfig::new())];
}
Dear BudiNverse,
Thanks for sharing the ERP data. I am trying to download the ERP rates from Dec 2016 to Mar 2018. However, I am only able to obtain the data of 2020-07-27. May I ask how could I post a date to obtain past datasets please? Thank you.
Best regards,
Wei Chong
Looking at this run, it seems that the body failed to parse but LTAErrror::BackendError
is returned.
Parsing updated bus_services.json
will fail as there is empty string data that is parsed to a u32
It looks like the failure is caused by
failures:
340
341
---- r#async::tests::get_cp_avail stdout ----
342
Error: BackendError(reqwest::Error { kind: Decode, source: Error("invalid value: integer `-46`, expected u32", line: 1, column: 3544) })
343
thread 'r#async::tests::get_cp_avail' panicked at 'assertion failed: `(left == right)`
344
left: `1`,
345
right: `0`: the test returned a termination value with a non-zero status code (1) which indicates a failure', /rustc/7abab1efb21617ba6845fa86328dffa16cfcf1dc/library/test/src/lib.rs:195:5
346
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Instead of taking Option<u32>
in this case
async fn get_taxi_avail(client: <AClient, skip: Option<u32>) -> LTAResult<Vec<Coordinates>> {
build_req_with_skip::<TaxiAvailResp, _, _>(client, api_url!("/Taxi-Availability"), skip).await
}
We can change the skip to impl Into<Option<u32>>
.
async fn get_taxi_avail(client: <AClient, skip: impl Into<Option<u32>>) -> LTAResult<Vec<Coordinates>> {
build_req_with_skip::<TaxiAvailResp, _, _>(client, api_url!("/Taxi-Availability"), skip).await
}
This allows you to call it like this.
let _ = get_taxi_avail(&client, 500);
let _ = get_taxi_avail(&client, None);
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: Json(Error("EOF while parsing a value", line: 1, column: 0)), url: None }', src\libcore\result.rs:1009:5
stack backtrace:
0: std::sys::windows::backtrace::set_frames
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
1: std::sys::windows::backtrace::set_frames
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
2: std::sys::windows::backtrace::set_frames
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys\windows\backtrace\mod.rs:104
3: std::sys_common::backtrace::print
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys_common\backtrace.rs:59
4: std::sys_common::backtrace::print
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\sys_common\backtrace.rs:59
5: std::panicking::default_hook
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:227
6: std::panicking::rust_panic_with_hook
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:491
7: std::panicking::continue_panic_fmt
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:398
8: std::panicking::rust_begin_panic
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:325
9: core::panicking::panic_fmt
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libcore\panicking.rs:95
10: core::result::unwrap_failed<reqwest::error::Error>
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libcore\macros.rs:26
11: core::result::Result<lta::bus::bus_stops::BusStopsResp, reqwest::error::Error>::unwrap<lta::bus::bus_stops::BusStopsResp,reqwest::error::Error>
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libcore\result.rs:808
12: lta::bus::get_bus_stops
at C:\Users\zeon000\.cargo\git\checkouts\lta-rs-f6ada22c85a9129d\add9969\src\bus.rs:331
13: cratetest::main
at .\src\main.rs:5
14: std::rt::lang_start::{{closure}}<()>
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\rt.rs:74
15: std::rt::lang_start_internal::{{closure}}
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\rt.rs:59
16: std::rt::lang_start_internal::{{closure}}
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\rt.rs:59
17: panic_unwind::__rust_maybe_catch_panic
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libpanic_unwind\lib.rs:102
18: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:289
19: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:289
20: std::panicking::try
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\/src\libstd\panicking.rs:289
21: std::rt::lang_start<()>
at /rustc/b3af09205b9af6026453946feadfd78b61b7a9fc\src\libstd\rt.rs:74
22: main
23: invoke_main
at d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
24: invoke_main
at d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
25: BaseThreadInitThunk
26: RtlUserThreadStart
error: process didn't exit successfully: `target\debug\cratetest.exe` (exit code: 101)
Code
fn main() {
match lta::bus::get_bus_stops() {
Ok(t) => println!("{:?}", t),
Err(e) => println!("{:?}", e)
}
match lta::bus::get_bus_services() {
Ok(t) => println!("{:?}", t),
Err(e) => println!("{:?}", e)
}
}
Function declared as
/// Returns location coordinates of all Taxis that are currently available for
/// hire. Does not include "Hired" or "Busy" Taxis.
///
/// **Update freq**: 1min
pub async fn get_passenger_vol_by(client: <AClient) -> LTAResult<Vec<Coordinates>> {
build_req_async::<TaxiAvailResp, _>(client, taxi_avail::URL).await
}
This is wrong and function name should be called get_taxi_avail
.
Implementation of async get_bike_parking does not take in any params at all! Params should follow sync version
Stacktrace
Error { kind: Hyper(Error { kind: Connect, cause: Os { code: 11001, kind: Other, message:
"No such host is known." } }),
url: Some("http://datamall2.mytransport.sg/ltaodataservice/BusArrivalv2?BusStopCode=83139&ServiceNo=15") }
Traffic images API update
Geospatial Whole Island API update
ZoneID update
Looks like v0.5.0 fails to compile to aarch64-unknown-linux-gnu
when target does not have OpenSSL installed. Since the requirement of OpenSSL has been removed in favour of rustls, this should not happen. We can simply change Cargo.toml
to disable default features for reqwest.
reqwest = { version = "0.11", features = ["json", "rustls"], default-features = false}
Since the main lib depends on the the sub libs ie lta_models
, lta_utils_commons
, lta_async
and lta_blocking
, it gets unnecessarily difficult to test on the main lib. To solve this problem we should move the testing to the individual crates
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.