Error in modeleditor from IDF class when getting schedules in convert_idf_to_trnbuild(), when the idf was loaded from cache (see error below):

commdct = None, key = 'SCHEDULE:DAY:HOURLY', block = None, defaultvalues = True

    def newrawobject(data, commdct, key, block=None, defaultvalues=True):
        """Make a new object for the given key.
    
        Parameters
        ----------
        data : Eplusdata object
            Data dictionary and list of objects for the entire model.
        commdct : list of dicts
            Comments from the IDD file describing each item type in `data`.
        key : str
            Object type of the object to add (in ALL_CAPS).
    
        Returns
        -------
        list
            A list of field values for the new object.
    
        """
        dtls = data.dtls
        key = key.upper()
    
        key_i = dtls.index(key)
>       key_comm = commdct[key_i]
E       TypeError: 'NoneType' object is not subscriptable

For some buildings, we encounter an "adjacency" error ("adjacent wall not found" in TRNBuild) only for some adjacent wall. Not all adjacent wall are not found.
For example when translating "NECB - Warehouse.idf", we find in the BUI file (attached to this issue) at lines 604 and 627 that "ADJACENT" and "ADJ_SURF" are empty:

line 604: FLOOR =c_000019             : SURF=   59 : AREA=     236.881 : ADJACENT= : ADJ_SURF= :  : GEOSURF=0.102000

line 627: CEILING=c_000017             : SURF=   81 : AREA=     236.881 : ADJACENT= : ADJ_SURF= : 

Whereas in line 604 it should be:

line 604: FLOOR =c_000019             : SURF=   59 : AREA=     236.881 : ADJACENT=z_000001 : ADJ_SURF=81 :  : GEOSURF=0.102000

And line 627:

line 627: CEILING=c_000017             : SURF=   81 : AREA=     236.881 : ADJACENT=z_000002 : ADJ_SURF=59 : 

Moreover, we do not understand why this kind of error happens while for other adjacent walls there is no errors, for example line 602 and 625 of the file:

line 602: WALL  =c_000021             : SURF=   57 : AREA=     110.544 : ADJACENT=z_000001 : ADJ_SURF=79 : FRONT : GEOSURF=0.034000

line 625: WALL  =c_000021             : SURF=   79 : AREA=     110.544 : ADJACENT=z_000002 : ADJ_SURF=57 : BACK : GEOSURF=0.147827

T3D_NECB 2011 - Warehouse_NOT WORKING.zip

I ran archetypal on branch 123-conditioning-settings-are-not-the-same and while the umi template succesfully created, there is no heating schedule in the resulting umi template and as a result no heating energy use in UMI.

200109_UMI_Templates.zip

Run as E+ v=9.2; the error is: "Cooling setpoint (90) exceeds maximum value (50)" which appears to be a transition isssue.
RefBldgStand-aloneRetailPost1980_v1.4_7.2_3A_USA_GA_ATLANTA.zip

Example in sample0.idf

InternalMass,
      CoreInternalMass,                 !- Name of internal mass surface
      B_Off_Thm_0,                                          !- Construction Name
      Core,                               !- Name of Associated Thermal Zone
      7.71749999999999;                                        !- Total area exposed to Zone [m2]

In the trnsys.py file, we have several function to translate an IDF file to an input file for TRNBuild. From line 417 to line 436 (region Get schedules from IDF) we get the used schedules in the IDF file.
However, this action takes around 90% of the total time of all the procedures.
After investigation, it seems to be at line 429 where we use s.to_year_week_day() that the major part of the time is consumed.

Embodied energy is currently ignored because it is not included in the material description within an idf file.

One solution could be to feed archetypal an external file with embodied energy parameters for each of the materials included in the EnergyPlus model (see image bellow). The structure of this file is to be discussed here.

Add an option to overwrite the idf file used to generate the trnbuild file (.bui)

When reducing an IDF file, we ignore the attic characterictics.
Problem: for the roof, the attic construction should be used! Now, it is the ceiling construction that is used for the roof (usually less insulated than a roof construction).

Hi Samuel and Louis, this issue relates to the JOSS review of your paper. Here are the detailed comments mentioned in the review page.

Paper summary:
I would suggest writing just a few words to give more high-level context to the non-expert, for example:

• a few words on Energy+: what it does (BES) and why it's important (major software in the domain)
• a few words on UMI: what it does (UBEM), and why its building models are different from Energy+
• a few words on TRNSYS: why do you consider this software and not, for example, IDA-ICE?

All of that could fit in 3-4 sentences, no need to give over-detailed explanations.

Misc comments on the paper:

• I'm a little confused by your definition of an archetype. It's not simply an advanced physical model or BEM, which might represent a real building, but rather the model of a theoretical building representative of the building stock.
• The need is stated at the end of each section. It would be easier to understand if you stated the need at the beginning of each section (problem -> solution), or you could state it in a separate section that would serve as a justification to developing archetypal.
• Please support the following statement in the TRNBuild conversion section: "archetypal answers a need from researchers [...]". Either rephrase or support with a reference.
• The E+ simulation environment functionality is not described clearly in the paper or the documentation.
• You mention the shoeboxer method. It is not clear if your method expands or modifies the shoeboxer method, or if it implements it. If it modifies or expands it, it would be good to see mathematical details in a companion paper or in the documentation.
• You would like to provide a scripting language for the modification of UMI template files. Could that not be considered an additional feature, rather than what sounds like a sub-feature of the conversion from E+ to UMI?

Documentation:
Generally speaking it seems incomplete: there are examples for using the E+ to UMI conversion, E+ to TRNBuild, but it is very vague on using archetypal as a simulation environment, the modification of UMI Templates is not covered by examples, the "getting started" section is left blank and the tutorials are incomplete or blank.
The user is left to finding out on their own how to use archetypal for any task other than converting models.

You seem to have done a great deal of work, nevertheless some more work on the documentation is in my opinion necessary if You wish to release the software.

Let me know what are your thoughts on these comments.
Kind regards
Thibault

Run as E+ v=9.2; something is funny in the transition and this error occurs for several idf files:
"Heating setpoint cannot be higher than the cooling setpoint. "

bad_idf_transition_files.zip

issueFiles.zip
When running through Archetypal these IDFs are E+ v=9.5 and it gives an error that it wants v=9.2. Using as-version also does not work.

Complexity reduction algorithm should merge a core template into a perimeter template if it's core zone represents less than a certain floor area percentage

When an idf file (that does not contain an sql object statement) is run with the output_report="sql" parameter, the eprun will not produce an sql file.

Another JOSS review issue! Once again, just a minor one.

You should check your use of acronyms. M in BEM is given as Models but at one point is used in a way that means modelling. UBEM is given in full form twice in the document, once with the M referring to Models and once referring to Modelling. I would also not say UBEM models as it sounds weird when you think it means Urban Building Energy Modelling models.

In fact, my preference would be to avoid using both BEM and UBEM in your paper, just use the phrases in full form. It doesn't add much to the paper length, but does aid in clarity.

Note: this issue comes to you as part of my wider JOSS review

The docs specify E+ v.8.9.0, when v.9.2.0 is already available (I can safely say that archetypal doesn't work with v.9.2.0!). Version issues crop up relatively late on in the use of archetypal, and are not particularly explicit in providing advice on how to solve them in the error messages. Could you check for E+ version on python interactive import of archetypal, then raise errors then about any user E+ version issues, instead of waiting for one of the archetypal functions to fail later on? It might also help to make the version dependency even more explicit in e.g. the package README.

Note: this issue comes to you as part of my wider JOSS review

ar.settings paths do not refer to the paths given in config. For Example :

ar.settings.data_folder returns 'data' instead of 'tests/temp/data'

The same problem happens for ar.settings.imgs_folder ; ar.settings.cache_folder ; ar.settings.logs_folder.

Installing on MacOS fails to complete due to a pip wheel not being found for 'regex' (one of the dependencies of a dependency). This isn't an issue with archetypal, per se, but was solved by a conda installation workflow (i.e. creating a conda environment, installing all dependencies from conda / conda forge, then installing archetypal with pip). Would it be possible to:

a) have a working conda environment requirements file in this repository, to minimise direct pip dependency, and

b) update the docs to include information on installing archetypal using a conda workflow, instead of directly with pip?

Note: this issue comes to you as part of my wider JOSS review

Edit: I see there is already a requirements file, which is used in the Docker instance, but it would need updating to work with MacOS too...

When the output_directory attribute is specified in run_eplus and that multiple idfs are run sequentially, the first result is return for all runs.

eg.:

idfs = [
    Path("archetypes/NECB 2011/NECB 2011 - Retail Standalone.idf"),
    Path("archetypes/NECB 2011/NECB 2011 - Large Office.idf"),
    Path("archetypes/NECB 2011/NECB 2011 - MidRise Apartment.idf"),
]
res = {
  name.basename(): ar.run_eplus(
     eplus_file=name,
     weather_file=weather_file,
     output_directory=output,
     prep_outputs=True,
     design_day=True,
     output_report="htm",
  )
  for name in idfs
}

will produce de same values for each element of res

@zberzolla, The comparison of B_Off_0 template with it's eqivalent after being passed through the archetypal pipeline (ran the B_Off_0 template, opened the SAMPLE0.idf in archetypal (UmiTemplateLibrary.from_idf()) gives the following differences in the Conditioning settings.

Before

After

invoking the archetypal convert command in the terminal on a fresh docker container with miniconda preinstalled (and no EnergyPlus installation) triggers an endless loop in eppy_load() > while idf_object is None > except Exception as e

This is the command used :

archetypal -v convert --trnsidf-exe docker/trnsidf/trnsidf.exe "tests/input_data/trnsys/ASHRAE90.1_Warehouse_STD2004_Rochester.idf"

Correct behavior

The command should warn if no EnergyPlus distribution is found (before entering the loop)

This example code from Convert IDF to UMI doesn't work:

from archetypal import BuildingTemplate
template_obj = BuildingTemplate.from_idf(
    x.idf, sql=x.idf.sql, DataSource=x.idf.name
)

^ Variable x is not yet defined elsewhere in the page

from archetypal import UmiTemplate
template_json = UmiTemplate(
     name="my_umi_template",
     BuildingTemplates=template_obj
).to_json()

^ template_obj is expected to be iterable. In the docs example, only one file is loaded, so template_obj (assuming x. is removed) is only one, uniterable object.

Note: this issue comes to you as part of my wider JOSS review

Create YearSchedule for Umi template from DaySchedule and WeekSchedule classes

UMI version 2.6.16, a building template now has en entry for the default WWR.

I've attempted to read and run an EnergyPlus file following the instructions given in the documentation. Very few steps worked, although many issues seem to be a fault of dependencies. Below I describe what hasn't worked. archetypal version: v1.3.0-dev (56f95d0), E+ version: 9.2.0, installation method: installing using conda, as per installation instructions.

Reading

Loading only an IDF file works fine, but loading an IDF and weather file at once fails with the following error trace:

ValueError                                Traceback (most recent call last)
<ipython-input-8-493961940195> in <module>
----> 1 idf2 = load_idf(eplus_file, weather)

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/archetypal/idfclass.py in load_idf(eplus_file, idd_filename, output_folder, include, weather_file, ep_version)
    774             include=include,
    775             epw=weather_file,
--> 776             ep_version=ep_version if ep_version is not None else settings.ep_version,
    777         )
    778         log("Eppy load completed in {:,.2f} seconds\n".format(time.time() - start_time))

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/archetypal/idfclass.py in _eppy_load(file, idd_filename, output_folder, include, epw, ep_version)
    825             IDF.setiddname(idd_filename, testing=True)
    826             # load the idf object
--> 827             idf_object = IDF(file, epw=epw)
    828             # Check version of IDF file against version of IDD file
    829             idf_version = idf_object.idfobjects["VERSION"][0].Version_Identifier

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/archetypal/idfclass.py in __init__(self, *args, **kwargs)
     57             **kwargs:
     58         """
---> 59         super(IDF, self).__init__(*args, **kwargs)
     60         self._sql_file = None
     61         self.schedules_dict = self.get_all_schedules()

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/eppy/modeleditor.py in __init__(self, idfname, epw)
    548         if idfname != None:
    549             self.idfname = idfname
--> 550             self.read()
    551         if epw != None:
    552             self.epw = epw

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/geomeppy/patches.py in read(self)
    254             raise IDDNotSetError(errortxt)
    255         self.idfobjects, block, self.model, idd_info, idd_index, versiontuple = idfreader1(
--> 256             self.idfname, self.iddname, self, commdct=self.idd_info, block=self.block
    257         )
    258         self.__class__.setidd(idd_info, idd_index, block, versiontuple)

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/geomeppy/patches.py in idfreader1(fname, iddfile, theidf, conv, commdct, block)
     88
     89     """
---> 90     versiontuple = iddversiontuple(iddfile)
     91     block, data, commdct, idd_index = readdatacommdct1(
     92         fname, iddfile=iddfile, commdct=commdct, block=block

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/eppy/idfreader.py in iddversiontuple(afile)
     49         return (0,)
     50     vers = line.split()[-1]
---> 51     return versiontuple(vers)
     52
     53

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/eppy/idfreader.py in versiontuple(vers)
     34     def versiontuple(vers):
     35         """version tuple"""
---> 36         return tuple([int(num) for num in vers.split(".")])
     37
     38     try:

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/eppy/idfreader.py in <listcomp>(.0)
     34     def versiontuple(vers):
     35         """version tuple"""
---> 36         return tuple([int(num) for num in vers.split(".")])
     37
     38     try:

ValueError: invalid literal for int() with base 10: 'Ap,IL,USA,TMY3,725300,41'

Running

Once an IDF is loaded (using the successful method idf = load_idf(eplus_file)), the run command (idf.run_eplus(weather_file)) causes an exception: TypeError: run_eplus() takes 1 positional argument but 2 were given.

If I run the file without first loading it (archetypal.idfclass.run_eplus(eplus_file, weather)) I get an EnergyPlus error:

EnergyPlusProcessError                    Traceback (most recent call last)
<ipython-input-27-9a3f51f38d18> in <module>
----> 1 run_eplus(eplus_file, weather)

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/archetypal/idfclass.py in run_eplus(eplus_file, weather_file, output_directory, ep_version, output_report, prep_outputs, simulname, keep_data, annual, design_day, epmacro, expandobjects, readvars, output_prefix, output_suffix, version, verbose, keep_data_err, include, process_files, custom_processes, return_idf, return_files)
   1751             }
   1752
-> 1753             _run_exec(**runargs)
   1754
   1755             log(

~/miniconda3/envs/archetypal/lib/python3.7/site-packages/archetypal/idfclass.py in _run_exec(tmp, eplus_file, weather, output_directory, annual, design_day, idd, epmacro, expandobjects, readvars, output_prefix, output_suffix, version, verbose, ep_version, keep_data_err, output_report, include)
   1977                     tmp.copytree(failed_dir / output_prefix)
   1978                 raise EnergyPlusProcessError(
-> 1979                     cmd=cmd, idf=eplus_file.basename(), stderr=stderr_r
   1980                 )
   1981

EnergyPlusProcessError: AdultEducationCenter.idf:
Program Version,EnergyPlus, Version 9.2.0-921312fa1d, YMD=2020.03.02 14:23,
   ** Severe  ** Line: 1056 You must run the ExpandObjects program for "HVACTemplate:Thermostat"
   ** Severe  ** Line: 1065 You must run the ExpandObjects program for "HVACTemplate:Zone:IdealLoadsAirSystem"
   **  Fatal  ** Errors occurred on processing input file. Preceding condition(s) cause termination.
   ...Summary of Errors that led to program termination:
   ..... Reference severe error count=2
   ..... Last severe error=Line: 1065 You must run the ExpandObjects program for "HVACTemplate:Zone:IdealLoadsAirSystem"
   ************* Warning:  Node connection errors not checked - most system input has not been read (see previous warning).
   ************* Fatal error -- final processing.  Program exited before simulations began.  See previous error messages.
   ************* EnergyPlus Warmup Error Summary. During Warmup: 0 Warning; 0 Severe Errors.
   ************* EnergyPlus Sizing Error Summary. During Sizing: 0 Warning; 0 Severe Errors.
   ************* EnergyPlus Terminated--Fatal Error Detected. 0 Warning; 2 Severe Errors; Elapsed Time=00hr 00min  0.21sec

Note: this issue comes to you as part of my wider JOSS review

GasMaterial constructor uses the name input rather than the type input to set the gas type. This causes errors when loading a library if the name is not a valid GasMaterial type.

repro:

from archetypal import UmiTemplateLibrary
lib = UmiTemplateLibrary.open("<path>/BostonTemplateLibrary_2.json")
gas = lib.GasMaterials[0]
gas.Name = "My Gas"
lib.save("<path>/Library_with_bad_gas.json")
UmiTemplateLibrary.open("<path>/Library_with_bad_gas.json")

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/src/local/archetypal/archetypal/umi_template.py", line 332, in open
    t = cls.loads(f.read(), name)
  File "/usr/src/local/archetypal/archetypal/umi_template.py", line 342, in loads
    t.GasMaterials = [
  File "/usr/src/local/archetypal/archetypal/umi_template.py", line 343, in <listcomp>
    GasMaterial.from_dict(store, allow_duplicates=True)
  File "/usr/src/local/archetypal/archetypal/template/materials/gas_material.py", line 107, in from_dict
    return cls(id=_id, **data, **kwargs)
  File "/usr/src/local/archetypal/archetypal/template/materials/gas_material.py", line 37, in __init__
    self.Type = Name.upper()
  File "/usr/src/local/archetypal/archetypal/template/materials/gas_material.py", line 51, in Type
    assert value.lower() in self._GASTYPES, (
AssertionError: Invalid value 'MY GAS' for material gas type. Gas type must be one of the following:
('air', 'argon', 'krypton', 'xenon', 'sf6')

Non breaking fix would be to try/except using the name first, and then fall back to type? Or maybe to use the type if it exists in kwargs, and if it doesn't, fall back to name?

For example, downloading a idf element using the data portal might need EnergyPlus 7.1

Analogue to OpenStudio Measures, an umi measure api could be used to build scripts that modify UMI Templates

Just two points on your JOSS paper (otherwise, I think it is sufficiently clear):

1. Line 62: striped -> stripped

2. Lines 79-81: Could you add links to the references to DOE benchmark models, to allow interested users to easily access those models?

Note: this issue comes to you as part of my wider JOSS review

It stores the complete dict of windows, rather than a ref:

This means that when loading building templates, it will create a new window object every time, since L336 will always error out:

This is normally handled fine, but in the realtime-graphs setup that I am working on right now, this leads to errors.

I'm pretty sure it's just an artifact of an old call to save() a template library, since current template libraries appear to save windows at the building template layer as refs, so it's just a matter of fixing the json.

CI tests take more than 1 hour to run. Long tests must be refactored.

There is a way of defining windows in E+ that uses to Window object instead of the FenestrationSurface:Detailed. The IDF.wwr() does not seem to capture these cases.

Use this file as an example: SFH_pre_1980.idf

Use PredictedLoadtoSetpoint (+/-) to specify:

On running archetypal convert "SimpleOfficeBuilding.idf" "output" I get the following error:

Traceback (most recent call last):
  File "my_conda_env/bin/archetypal", line 8, in <module>
    sys.exit(cli())
  File "my_conda_env/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "my_conda_env/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "my_conda_env/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "my_conda_env/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "my_conda_env/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "my_conda_env/lib/python3.7/site-packages/archetypal/cli.py", line 303, in convert
    if paths:
UnboundLocalError: local variable 'paths' referenced before assignment

It looks like in cli.py there is a with statement which produces a variable paths which is then expected to exist later on (if paths:). paths might need to be set as e.g. None before the with statement?

Note: this issue comes to you as part of my wider JOSS review

When reducing a large list of IDFs files, I got an error "AssertionError: Heating setpoint must be lower than the cooling setpoint" but have no way of telling which IDF file is causing this error. The last entry above the error was: "Reducing perimeter zones in <_io 0% | | 0/68

It would be nice if the flag reported which file caused the error.

3A_Reference_Buildings.zip

I am trying to convert an E+ idf file to UMI .json template. I am a new Python user and followed the installation instruction (installing using pip) in Python 3.6. I have installed archetypal successfully, but when I am trying to convert the IDF file following "Using the Python Console" I am getting this error: 'UmiTemplateLibrary' is not defined". Please see the attached screenshot from Python IDE.

Please help to figure this problem out!

Thanks,

8 tests fail and 22 error when running tests locally on MacOS. The 22 errors seem to all be trnsys related, but could be handled more gracefully by skipping a test if the operating system is not Windows (see here). Full test log is given here - enjoy!

Note: this issue comes to you as part of my wider JOSS review

There seems to be some issues with the way COPs are calculated.

This recursion call sometimes raises a RecursionError: maximum recursion depth exceeded while calling a Python object

I have not yet been able to reliably repro - error is happening in umiverse flask backend.

Sometimes the following steps execute successfully, sometimes, the final step fails with an recursion error.

1. Fetch records ["MediumOffice_Post1980_2A", "MediumOffice_New2004_2A", "MediumOffice_Pre1980_2A"] from MongoDB (though I have not yet found a single template or collection of templates which always causes the error - my guess is it has to do with the order that the templates arrive as results from the db, if that is not the same every time?)
2. construct the templates with bld.to_template(...)
3. construct the template_lib with UmiTemplateLibrary(BuildingTemplates=[<templates>])
4. run template_lib.unique_components()
5. convert to dict with template_lib.to_dict()

When it fails, this stack trace occurs:

Traceback (most   recent call last):
--
File "/usr/src/app/project/routes.py", line 1104, in   construct_umi_file_task
template_lib_as_dict = template_lib.to_dict()  # as dict
File   "/usr/local/lib/python3.8/site-packages/archetypal/umi_template.py",   line 579, in to_dict
data.update({"Name":   UniqueName(data.get("Name"))})
File   "/usr/local/lib/python3.8/site-packages/archetypal/template/umi_base.py",   line 511, in __new__
return str.__new__(cls, cls.create_unique(content))
File   "/usr/local/lib/python3.8/site-packages/archetypal/template/umi_base.py",   line 536, in create_unique
return cls.create_unique(name)
File   "/usr/local/lib/python3.8/site-packages/archetypal/template/umi_base.py",   line 536, in create_unique
return cls.create_unique(name)
File   "/usr/local/lib/python3.8/site-packages/archetypal/template/umi_base.py",   line 536, in create_unique
return cls.create_unique(name)
[Previous line repeated 934 more times]
File   "/usr/local/lib/python3.8/site-packages/archetypal/template/umi_base.py",   line 528, in create_unique
match = re.match(r"^(.*?)(\D*)(\d+)$", name)
File "/usr/local/lib/python3.8/re.py", line 191, in match
return _compile(pattern, flags).match(string)
File "/usr/local/lib/python3.8/re.py", line 291, in _compile
if isinstance(flags, RegexFlag):
RecursionError: maximum recursion depth   exceeded while calling a Python object

Seems like there is some sort of issue with the regex matching when it shouldn't, causing the recurse call to keep happening.

It's possible to increase the python recursion limit (default is 1k I believe) if it is actually possible for the count to go past 1k, but that seems like it shouldn't be necessary, and anyways doesn't seem like the most performant solution if it is. I wonder if it makes sense to use a method like keeping track of the count for each base name, and then just incrementing that, rather than trying to determine the current count with the regex and then recursing.

Both heating and cooling COPs are set to 0 when reducing from ClimateStudio IDFs. ClimateStudio uses ZoneHVAC:IdealLoadsAirSystem.
archetypal_issue.zip