Problem of `StopTime`'s attributes when updating `Collections::vehicle_journey` about transit_model HOT 3 CLOSED

I did some experimentation by bringing back the 3 fields inside the object StopTime. The fields are of type Option<String> which is a bit expensive in memory so I used instead a Box<Option<String>> which allow me to allocate the size of a String only when necessary (Box is about 8 bytes, a String is 24 bytes). Below are some data when trying to convert from GTFS to NTFS on a big dataset (more than 1Gb of stop_times.txt).

The first graphic is comparing the time performance and they are pretty similar.

The second graphic compares the memory usage.

In this case, we can see a increase of about 20-25% of memory when bringing back the 3 fields inside StopTime. This is a significant increase.

After some discussions about these results, the preferred option is to keep the memory structure as it is but to not exposed it as public but through API. These API would assure the coherence of the data so the developer can't forget about it. This API must be in Collections which is the structure that contains vehicle_journeys, stop_time_ids, stop_time_headsigns, and stop_time_comments. These 4 collections needs to become private and a well-designed API need to expose the needed functionalities to manipulate VehicleJourney and StopTime.

from transit_model.

Some more thinking about this idea of wrapping the complexity of VehicleJourney and StopTime manipulation in API at Collections level.

First of all, the idea would be to propose the following API at Collections level:

• fn get_vehicle_journey(&self, id: &str) -> &VehicleJourney;
• fn add_vehicle_journey(&mut self, vehicle_journey: VehicleJourney) -> Result<()>;
• fn remove_vehicle_journey(&mut self, id: &str) -> VehicleJourney;

These API should be enough to manipulate VehicleJourney (maybe we can think of more API to assist some specific processings but they'll be more or less wrapper around these 3 APIs). These 3 API will take care of maintaining the coherence of stop_time_ids, stop_time_headsigns, and stop_time_comments.

However, these is one problem. The coherence of stop_time_ids, stop_time_headsigns, and stop_time_comments is mainly based on the keys of these maps. The key of these maps is (Idx<VehicleJourney>, u32):

• Idx<VehicleJourney> is the index of the VehicleJourney in Collections::vehicle_journeys
• u32 is the stop_sequence of the stop_time

If we add or delete a VehicleJourney, we need to check in these 3 maps for all keys that refers to this modified VehicleJourney. But we should do the same thing for anything that affects StopTime inside a VehicleJourney. But VehicleJourney owns the StopTime, not Collections. There is no possibility to make VehicleJourney::stop_times private without exposing an API for Collections. But if it is exposed for one, it is exposed for anyone.

A possible solution would be to change the way the model is organized. We remove the field VehicleJourney::stop_times: Vec<StopTime> and we add the field Collections::stop_times: BTreeMap<Idx<VehicleJourney>, Vec<StopTime>>. This way, we move ownership of StopTime to Collections and we can add some new API to manipulate StopTime in Collections.

Any opinion about that?

from transit_model.

Probably won't happen ever, I'm closing this issue.

from transit_model.