Open basic version of EMPIRE in Pyomo.

The EMPIRE model and all additional files in the git repository are licensed under the MIT license. In short, that means you can use and change the code of EMPIRE. Furthermore, you can change the license in your redistribution but must mention the original author. We appreciate if you inform us about changes and send a merge request via git. For further information please read the LICENSE file, which contains the license text, or go to https://opensource.org/licenses/MIT

The EMPIRE model is available in the Python-based, open-source optimization modelling language Pyomo. Running the model thus requires some coding skills in Python. To run the model, make sure Python, Pyomo and a third-party solver (gurobi, FICO xpress, or CPLEX) is installed and loaded to the respective computer or cluster. More information on how to install Python and Pyomo can be found here: http://www.pyomo.org/installation. Other python package dependencies can be found in the file environment.yml.

To download, you need to install Git and clone the repository. Note that the repository makes use of Git Large File Storage (LFS) which also needs to be installed for input data-files to be downloaded when cloning the repository. Once both Git and Git LFS has been successfully installed, the model is downloaded to the desired directory.

EMPIRE consists of a programming script used to run the model:

scripts/run.py: The main script used to run EMPIRE. This is the only script a user of EMPIRE needs to use and potentially modify.

Note: The main run-script (scipts/run.py), can run a small test instance of EMPIRE that usually finishes in 1-2 min. A normal test instance requires around 140 GB RAM and thus needs to be run on a high performance cluster (HPC).

The run script uses the empire package that consists of these core modules:

(1) empire.py: Contains the abstract formulation of EMPIRE in Pyomo. This script also contains code related to printing the results.

(2) scenario_random.py: Generates random operational scenarios as .tab-files through sampling.

(3) reader.py: Generates .tab-files input based on data provided in Excel workbooks.

(4) config.py: Defines two configuration objects used by Empire.

(5) model_runner.py: Methods used for setting up an Empire run.

In addition there are modules containing input and output clients, that can be used to read and alter input data, and read ouput/results data.

In the repository, the ‘Data handler’-folder contains the Excel workbooks that are used to store and modify input data. The workbooks are contained within folders representing instance-versions of EMPIRE, e.g. ‘europe_v50’. The ‘test’-folder contains input data for a small test-instance of EMPIRE. Within an instance-version in the ‘Data handler’-folder, there is a folder called ‘ScenarioData’ containing large data sets used to generate stochastic scenarios in EMPIRE. If EMPIRE is run with random scenario generation, representative time series are sampled once per scenario and season for each random input parameter.

The EMPIRE Model reads .tab-files, which provide all needed sets and input data. For editing and storing the data, excel-files are used. There are seven excel-files in total of which six contain indexed input data and one is to provide the indices/sets. The excel-files are sorted by the following categories: General data, generation data, country/node data, set/index data, transmission data, and storage data. These files contain multiple tables regarding for example investment costs and initial capacity.

For more details, please refer to the software documentation in the repository.

System Requirements: Running the base case of EMPIRE in Pyomo typically takes about 40 minutes and requires approximately 140 GB of RAM.

Initial Test: To ensure your setup is correctly configured, start with a test run using a smaller dataset. This test checks if your computer or cluster can successfully connect to the preferred solver.

Run the test using the following command:

C:\Users\name\path_to_folder> python scripts/run.py -d test

No-Optimization Test: To execute a test run without performing optimization, use:

C:\Users\name\path_to_folder> python scripts/run.py --test-run -d test

This can be useful for verifying basic setup and data handling without engaging the full optimization process.

Setup Requirements: Ensure that Pyomo and the preferred solver are installed.

Execution: Run the model using the run.py script. Specify the dataset located in the Data Handler folder. For example, to run the model with the europe_v51 dataset, use:

C:\Users\name\path_to_folder> python scripts/run.py -d europe_v51

Example Script: For running multiple cases on an HPC, refer to the script scripts/copy_and_run_empire_on_hpc.sh. This script uses configurations from config/cluster.json and is designed for NTNU's HPC clusters: Solstorm and Idun.

Usage Linux or macOS: From the project directory, execute the following to run on the Solstorm cluster:

sh scripts/copy_and_run_empire_on_hpc.sh Solstorm

Usage Windows: Install "jq" on Windows:

curl -L -o /usr/bin/jq.exe https://github.com/stedolan/jq/releases/latest/download/jq-win64.exe

From the project directory, execute the following command in Git Bash to run on the Solstorm cluster:

sh scripts/copy_and_run_empire_on_hpc.sh Solstorm

This command copies the EMPIRE code to the Solstorm cluster and performs several runs managed by the SGE task manager. Ensure the "empire_env" conda environment is set up on the cluster with dependencies as listed in environment.yml.

The scripts/run_analysis.py script demonstrates how to modify input data at execution time using data managers.

We welcome any contribution the OpenEMPIRE, whether it is fixing a bug, adding a new feature, or improving documentation, your help is appreciated. For more information, see CONTRIBUTING.