RuntimeError: table 'taz_control_data' never checkpointed. Closing remaining open files:output\pipeline.h5...done about populationsim HOT 13 CLOSED

That sounds like it is failing pretty early. It seems like the taz data control is not established correctly. How may minutes or seconds would you say that it ran before it got to this message?

from populationsim.

from populationsim.

activitysim.log
Please note that it takes me less than 30 seconds to have this message. I could not understand the problem there.

Any help would do.

thanks.

Issa

from populationsim.

I'll leave it to Jeff to provide a more helpful response, but it looks like it read the control_taz.csv file fine, so I'm guessing there must be an inconsistency in the settings.ymal or controls.csv. Either that or it could be a data type in the control_taz file. In other issues we have found that PopulationSim doesn't do well with decimals in the control input files. You might ensure that everything is a round integer in your taz file and all your control files.

from populationsim.

By the way, I am very desperate and disappointed by this error message, which does not have a clear meaning. I tried everything, still to no avail. Any help would be really appreciated.

I attached also the yaml file (in text here).

Issa
‏‏settings.txt

from populationsim.

I would echo that PopulationSim would benefit from better error reporting in general, and one thing that would help that would be some standard checks up front for known issues so that those issues can be flagged and a more helpful error message can be provided. I see that taz is your lowest level of geography. That means that num_hh specifically needs to be called out and pointed to the correct field in your controls.csv. Again, I would check for and round off any decimals. Ensure all control files are integer only...

from populationsim.

Well, I do not have any kind of decimals in the control file. This is not the issue here.
Al the data types there are numerical. No letters or anything. I will post also controls and the geo_cross_walk files (moved them to text).

Thanks.

‏‏geo_cross_walk.txt
‏‏control_taz.txt

from populationsim.

The other file that would need to be shared is the controls.csv in the config folder

from populationsim.

‏‏controls.txt
Attached.

Sorry for the mess, but I need to run this ASAP.

from populationsim.

I don't know how the seed was processed. It looks like in your control file you are referencing a "hhnum" filed in your seed. I know we have had issues when we try and mess with the expression for "num_hh". You have this - households.hhnum > 0

We have this:
(households.WGTP > 0) & (households.WGTP < np.inf)

I'll let you know if I think of anything else, but that's currently my best thought. Maybe Jeff or others can review and provide better instruction.

I would add that after you get this working, you are going to have conflicting controls. All your size controls are going to fight directly with your duplicate population control -
population,taz,persons,1000000,pop,persons.hhnum >= 1

I don't think that population control will work as intended. I suggest removing that one, and letting the population by age controls do the work of that total population control.

Related to this, you might consider doing tests where you start with a smaller set of controls and then add controls and see how those impact the results (once you get it working).

from populationsim.

Here is the original runtime error

Just reiterating what I sent in the email for better documentation.

PopulationSim expects you to be consistent with geography names, it is even case sensitive. In your settings file, where you list the geographies, “taz” is in lower case but while defining the taz control table, you use uppercase.

• tablename: TAZ_control_data
  filename : control_taz.csv

As per the error message it cannot find "taz_control_data" since "taz" was listed as a geography. However, you defined "TAZ_control_data" as the control table for that geography. Defining that table by the name "taz_control_table" should probably resolve this error.
Please try using consistent geography names everywhere. Please update this thread if that fixes it.

from populationsim.

@issa-jtmt : Since this has been resolved can we please close this issue? I will open another issue with an enhancement/documentation tag

from populationsim.

from populationsim.