cityscope / vehicleclustering Goto Github PK

Review when a bike is available for a platoon or ride

Right now we are running the experiment only on Volpe site that has a lot of specificity.

To develop a more generic framework, it should be possible to run a simulation on any site assuming we have:

• The right GIS layer (roads, buildings, bike station? , etc etc)
• The right demographic data (Bluebikes, replica, etc etc)

@NaroaCS and/or @jmugicagonz can you start to work a 2nd site with the right data (we were speaking about greater Cambridge first)? And feel free to report or ask if there are any questiosn related to the GIS itself or with the GAMA Model.

Pheromone levels are not increasing, keeping a 0 value in every tag during the simulation. Investigation is needed

Hi @agrignard !

A quick question: I want to run the experiments in the Andorra server, and I was wondering if Gama is already installed yet.
If it is --> Do I need to update it? What’s the command line to update it?
If it is not --> How do I need to install it? I guess I’ll have to clone it from GitHub, right? I can’t find how to install it through command line

Thank you!!

Switch the map to an area larger than Kendall to have more realistic bike trips.
Since we will have an OD matrix based on Bluebikes data, we can have an area that at least includes. As a first iteration for the demand, I chose the following area:
lat_min = 42.355013
lat_max = 42.369192
lon_min= -71.117067
lon_max = -71.075183

For the moment we dont consider that the users consume battery. This should be added by modiying some previous version of this implementation that is now commented in the code.

Hi @agrignard !

Could you, please, help me with a question on GAMA headless mode?

When I run the simulations in headless mode on the server, I always see the following note: Number of cpus used:1 (available: 4)
I have specified the number of cores as indicated here (https://gama-platform.org/wiki/Headless):
eg) bash gama-headless.sh -hcp 4 samples/pheromone-random.xml outputHeadless
I have tried both with 4 and 16 but it keeps saying Number of cpus used:1 (available: 4)

The specs of the server are the following:

CPU(s):              4
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           4

How can I use all the 4 available cpus?

Thank you!

Hey @agrignard !

I've been checking how to get a proper output from the simulation so that Edu can run it on the Tangible swarms and I might need a bit of help from you. He explained to us that what he needs is either a Gama simulation that he can replicate running it in real time or a log file, that would contain data with a high frequency (sim step around 10Hz) that gives him info on:
a) the speed and angle of each vehicle b) the rest of the variables that we want to represent (i,e, status, battery level). This would be the data that he would send to the robots.

I have two questions:

• What do you think would be the best way of saving this information? Should we use a -csv file or would it be better to use something like a Gama monitor or an inspector?

• Right now the agents are generated at a random place at a random time, but we want to be able to save and replicate this simulation so that we identify some scenarios that we want to test with the Tangible Swarms. Is this what the saveSimulaition is supposed to do? And then we could run that .gsim?

Hi @agrignard,
I can't add collaborators to this repo, could you please add @larryshi02? Thank you!

In the UrbanSwarm simulation, vehicles can only take one decision per time step. This means that if it takes a vehicle 1 minute to get to the next intersection and the time step is 5 minutes, the vehicle will wait for 4 min at the intersection until it can take the next decision. @agrignard How did you handle this in the Urban Swarms simulation?

We have tried a couple of things to solve this, but for now thought the best was to at least have an estimate of the error that we are introducing. For that, we got a good average of what is the mean segment distance (from intersection to intersection) by running a long simulation and tracking the agents, and with that we can estimate the error as a function of the time step. Error in the average segment = step size – time that it takes to do the average segment

For example: If the average distance is 100m and the vehicle speed is 10 km/h, it will take 36 s to do this distance. In this case, with a step size of 1 min, on an average step, the vehicle would lose 24s per vehicle per intersection, with 2 minutes of step size we would lose 84s, etc…. Therefore, the ideal step size is 36s or less so that a vehicle takes the 20s on a certain road it will wait for 16s and so on, and the least favorable case is that if it takes 37s it will wait for 35s…

Add user to bike trip and bike to user trip

Hi @tmightyquinn,
Could you take a look at the .gitignore file? I think we should remove the fact that it ignores the .csv so that we can upload the demand files. Also, in my last commit, it uploaded some extra stuff.

Hi @jmugicagonz and @larryshi02
I leave here some questions and notes about the pheromones. Let's see if we can figure out these issues together:

-pheromoneMark: In the original code when carrying trash: myself.pheromoneMark <- (singlePheromoneMark * int(self.trash/carriableTrashAmount));
https://github.com/mitmedialab/UrbanSwarms/blob/80e4cb809f6e48231a117a18d99fc30ee1141f7d/UrbanSwarm/models/swarmBot.gaml#L267
And when the trash is deposited: myself.pheromoneMark <- 0.0;
https://github.com/mitmedialab/UrbanSwarms/blob/80e4cb809f6e48231a117a18d99fc30ee1141f7d/UrbanSwarm/models/swarmBot.gaml#L353
We have just a placeholder for this

Also, @jmugicagonz this is initialized with a value that I don't understand, I think you did this. Shouldn't it be 0?

-pheromoneToDiffuse: We have a different implementation of this, we take into account the pheromone in the tag and they take into account the pheromone of the bike.
We do: pheromoneToDiffuse <- max(tag.pheromoneMap)*diffusion;

• Evaporation: In the original code it's proportional to the cycles, but Quinn noted that it should take into account the step size so that it's propotrional to time and not cycles. I think this is correct
  Theirs:
  https://github.com/mitmedialab/UrbanSwarms/blob/80e4cb809f6e48231a117a18d99fc30ee1141f7d/UrbanSwarm/models/swarmBot.gaml#L238
  Ours:
  

  VehicleClustering/CS_CityScope_GAMA/models/Agents.gaml

  Line 530 in d5fd379

  tag.pheromoneMap[k] <- tag.pheromoneMap[k] - (singlePheromoneMark * evaporation * step*(cycle - tag.lastUpdate));

• Quinn left a note saying that one of the saves alters local copy, I think it is correct that it does so. Or is it an issue?

  VehicleClustering/CS_CityScope_GAMA/models/Agents.gaml

  Line 511 in d5fd379

  pmap[previousTag] <- 0.0; //alters local copy only :)

Hi @agrignard, @LAAP. (I don't have Edu's GitHub, it would be great if you could add him to this conversation)

Together with @tmightyquinn and @jmugicagonz, we had gathered these questions about the previous UrbanSwarn simulation, could you help us with this?

1. What does the pheromone_mark represent?
2. Why are exploratoryRate and diffusion related as they are? (diffusion = (1-exploratoryRate)/2)
3. What are pheromone_roads? What is the purpose of these pheromone values? Who uses them, and for what? How are they marked (sth like: max distance to point, etc.)
4. Should pheromone evaporation be proportional to time or the number of cycles?
5. How many times should an RFID tag be updated by a given bike?
6. Why do bikes move into the low_battery state when batteryLife < lastDistanceToChargingStation/speed? Why divide by speed?

Updates in choosing candidate bikes:

• Removed the TripSafetyFactor because we don't know in advance how long the trip is going to be and for now the bike doesn't consume battery when the user rides. (Left as a comment in case we want to use it in future versions)
• The variable is maxWaitTime and with that and the speed we define the rideDistance that corresponds to that wait time, and then rideDistance is the value used when choosing candidate bikes

@jmugicagonz

I pushed some code that affects pheromone visualizations and the CSVs in the "data" folder were also unintentionally uploaded. Do you know why the folder was updated? Maybe it has something to do with .gitignore?

I've been changing and fixing some stuff related to charging. I have:

• updated the units in log files from sec to percentages
• bikes are initialized at a random battery level between the min and the max -> [25-100%]. WIth the previous simulation we saw that the distribution tended to be random
• the minSafeBattery is now a 25% of the maxBatteryLife
• the clusterTreshold is also a funcion of the max battery now (5%)
• the decision to go for a charge depends only on the battery level, if it's below the minSafeBattery, but there's no numberOfStepsReserved or distanceSafetyFactor
• we want to define the charging by the time that it takes
• fixed a bug that was taking the min V2IChargingRate instead of the right one

One of the this that I've seen is that with lower V2V charge rates the battery levels were more heterogeneous, but if V2V is high bikes share battery in the beginning until they homogenize, and then they all start to run out of battery around the same time, creating a peak in bikes that are charging and leaving no bikes available. Depending on the battery capacity and the V2I charging rate this peak happens at different times and has a different magnitude, so it can have from no impact to a very severe impact on served trips. Here are some examples:

a) V2IChargingRate: 1.4 / chargingStationCapacity: 10/ V2V Charging Rate [m/s]: 2.469

b) V2IChargingRate: 1.4 / chargingStationCapacity: 10/ V2V Charging Rate [m/s]: 13.88

c) V2IChargingRate: 13.888 / chargingStationCapacity: 25/ V2V Charging Rate [m/s]: 13.88

in bikeChargeEvents.csv

Dear all,

I just saw some commit like the last one where many meta files from GAMA are actually commited on this repo.

@jmugicagonz Is there a reason why those data are commited?

We still have some units that are not intuitive both in the .csv and the .txt output files. We should review and polish these.

Hi @jmugicagonz,

I remember that you mentioned something about multiplying some floats by a 100 and casting to int, I see that you did this for instance here:

What was the issue? Is there any other way to fix it? Otherwise I guess we're getting values 100 times higher than they should be.

in Loggers.gaml > Logger > log.

This parameter seems unused

Some wait times are above the maximum wait time, we need to review how we filter the bikes when deciding which one to assign

Hi @jmugicagonz,

I removed the main folder CS_CityScope_GAMA so that we have one level less of folders; so now the readme and the folders models, includes etc. are at the same level. I also removed some data from the includes folder, which was related to Energy, the GameIT model etc. I think we can still clean it up a bit more but I'm afraid of deleting something that you consider important. Could you review this and maybe simplify a bit more the structure of the files? I think we can just have three folders under includes:

• demand: .csv files containing the origin, destination and time
• graph: road networks
• buildings: shapefile of the buildings

Also, we can probably just have the current code under models (and not have a subfolder CityScope) and the results can be in a separate folder, so that in the main folder we would have:

• includes
• images
• models
• results
• headless? -> Do you know what this is? I think it's from the previous code, I think we can remove it

Then we should double-check that the paths are correctly set in the code.

Note: The new demand is under includes>Demand but there is also some graph and building maps from the previous simulation in this folder too in case you want to check them out.

Hi @tmightyquinn @jmugicagonz !
I have a quick question about the log files, what does the 'level' variable determine?
We define logging levels here:

@jmugicagonz

Juan,

Do you know if there is an easy way to deactivate battery sharing in the Gama code? I am currently working on setting up experiments that don't incorporate battery sharing.

Thanks.

GitIgnore files are not working properly, and more files than they should are still being uploaded

Just noting some general questions for all of us:
Parameters.gaml
Why do we need a followDistance
Why do we check so many parameters in order to decide to get a charge ? this one and this other one
What is rideDistance

Agents.gaml
Does this line check if there are bikes available?

Hi @agrignard,

Sorry to bother you! Trying to un Gama in headless mode I get this error:
.java.lang.ClassCastException: msi.gama.kernel.experiment.BatchAgent cannot be cast to msi.gama.kernel.simulation.SimulationAgent

The experiment that I'm trying to run is a batch experiment, but does this error mean that I can't run a batch experiment in headless mode? I've checked the Gama documentation and I see that it says: "For more systematic parameter values samples, you should turn towards a more adapted tool such as GAMAR, to generate a ExperimentFile.xml with a huge number of simulations." Is this what I would need to do, instead of running a batch experiment?

Just in case it helps, this is my .xml file:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Experiment_plan>
        <Simulation id="2" sourcePath="../../../VehicleClustering/CS_CityScope_GAMA/models/clustering.gaml" until="cycle >= numberOfDays * numberOfHours * 3600 / step" experiment="batch_experiments_headless">
          <Parameters>
          </Parameters>
          <Outputs>
          </Outputs>
        </Simulation>
</Experiment_plan>

And this is the batch experiment that I want to run (defined in my clustering.gaml file):

Hi @agrignard !
I am trying to run the experiments on a server and want to take advantage of the multithreading.
I see that the execution parameters can be adjusted in the Preferences tab on the graphical interface and that the default is 8. I have seen that this can also be set when defining the experiment in the .gaml script by adding parallel: nun_threads. This is what I have done.

The server has 40 threads:

So I have defined my experiment as:
experiment batch_experiments_pheromone type: batch parallel: 40 repeat: 15 until: (cycle >= numberOfDays * numberOfHours * 3600 / step) {
parameter var: evaporation among: [0.05, 0.1, 0.15, 0.2,0.25,0.3];
parameter var: exploitationRate among: [0.6,0.65,0.7, 0.75, 0.8];
parameter var: numBikes among: [150, 250, 350];
parameter var: WanderingSpeed among: [1/3.6#m/#s,3/3.6#m/#s,5/3.6#m/#s];
}

But when I launch the experiment, it still only launches 4 threads in parallel:

Any ideas on how to make it run on more than 4 threads?

Thank you!

Maybe we should have a demand that is more realistic than a random on a window; do have demand data based on BlueBikes usage and for this simulation realism in the demand is important.

A new log file is needed for the set up. This means, every time the simulation is started, a CSV file is generated containing the values of the parameters used for that simulation. To decide which parameters to include in the CSV file.

Double check if users are generated as they should according to the demand input .csv, preliminary results show that there might be some bug.

For some reason, every charge in a charging station has the same duration. It needs investigation

Even if trips are set to start randomnly inside a specific range, they are concentrating in specific periods inside those ranges. Need investigation.

Hi @agrignard !
I am a bit confused about a memory error that I am getting in launching the simulations on the server. Currently, I am running 15 threads in parallel in a server that has 1 T of RAM and 40 threads.

I have set the specs in Gama.ini as follows:

-Xms4096m
-Xmx100g
-Xss1g
-Xmn50g

This is the error that I am getting

Message: Your system is running out of memory. GAMA will exit now. Please try to quit other applications and relaunch it
Exception in thread "Timer-0" java.lang.OutOfMemoryError: Java heap space
        at java.prefs/java.util.prefs.FileSystemPreferences.sync(FileSystemPreferences.java:768)
        at java.prefs/java.util.prefs.FileSystemPreferences.flush(FileSystemPreferences.java:844)
        at java.prefs/java.util.prefs.FileSystemPreferences.syncWorld(FileSystemPreferences.java:484)
        at java.prefs/java.util.prefs.FileSystemPreferences$3.run(FileSystemPreferences.java:451)
        at java.base/java.util.TimerThread.mainLoop(Timer.java:566)
        at java.base/java.util.TimerThread.run(Timer.java:516)
naroa@matlaberp4:~/GAMA_1.9.0_march13/headless$ 

And this is the behavior of the server:

It seems that the memory use is very low, but the cache and buffer are high (green plot). Does the cache and buffer memory trigger the OutOfMemory error? I also see some peaks in the load, which I am not sure should have happened, but not sure if this can also have an impact on memory usage.
Do you have any ideas on why this might be happening? Is Gama accumulating data that I should be clearing? I am saving everything that I need on .csv-s, so I wouldn't need any other info on the simulations.

Thanks for your help!

I'm wondering what is the syntax I can use to call a method when I'm displaying an experiment.

For example, if I want to add a facet to the bike species that takes the value returned by the function availableForPlatoon, is there a way I can call that function? Or, if there's a way to simply call a parameter within the bike species, that would be helpful too.

It seems that the pheromone levels at every intersection always stay at 0 in all directions. Maybe there's a bug in the updatePheromones logic?

@agrignard - You mentioned a method to determine the number of runs for each parameter setup.
Did you refer to the method described in 2.4. and 2.6 in this article: https://www.jasss.org/18/4/4.html ?

Simulation should start always at the same time. It is currently starting at the local computer time.

TODO: check if chargingStationCapacity is working
It is used here: