Running the model¶
There are two basic modes for the model: planning mode and operational mode. The mode is set in the run configuration.
In planning mode, constraints are given as upper and lower boundaries and the model decides on an optimal system configuration.
In operational mode, all capacity constraints are fixed and the system is operated with a receding horizon control algorithm (see Model-wide settings for the settings that control the receding horizon).
In either case, there are three ways to run the model:
- With the
calliope run
command-line tool. - By generating and then executing parallel runs with the
calliope generate
command-line tool. - By programmatically creating and running a model from within other Python code, or in an interactive Python session.
Single runs with the command-line tool¶
The included command-line tool calliope run
will execute a given run configuration:
$ calliope run my_model/run.yaml
It will generate and solve the model, then save the results to the the output directory given by output.path
in the run configuration.
Two output formats are available: a collection CSV files or a single NetCDF file. They can be chosen by settings output.format
in the run configuration (set to netcdf
or csv
). The read
module provides methods to read results stored in either of these formats, so that they can then be analyzed with the analysis
module.
Parallel runs¶
Warning
This functionality is currently not Windows-compatible.
Scripts to simplify the creation and execution of a large number of Calliope model runs are generated with the calliope generate
command-line tool as follows:
- Create a
run.yaml
file with aparallel:
section as needed (see Settings for parallel runs). - On the command line, run
calliope generate path/to/run.yaml
. - By default, this will create a new subdirectory inside a
runs
directory in the current working directory. You can optionally specify a different target directory by passing a second path tocalliope generate
, e.g.calliope generate path/to/run.yaml path/to/my_run_files
. - Calliope generates several files and directories in the target path. The most important are the
Runs
subdirectory which hosts the self-contained configuration for the runs andrun.sh
script, which is responsible for executing each run. In order to execute these runs in parallel on a compute cluster, a submit.sh script is also generated containing job control data, and which can be submitted via a cluster controller (e.g.,qsub submit.sh
).
The run.sh
script can simply be called with an integer argument from the sequence (1, number of parallel runs) to execute a given run, e.g. run.sh 1
, run.sh 2
, etc. This way the runs can easily be executed irrespective of the parallel computing environment available.
Note
Models generated via calliope generate
automatically save results as a single NetCDF file per run inside the parallel runs’ Output
subdirectory, regardless of whether the output.path
or output.format
options have been set.
See Settings for parallel runs for details on configuring parallel runs.
Running programmatically from other Python code¶
The most basic way to run a model programmatically from within a Python interpreter is to create a Model
instance with a given run.yaml
configuration file, and then call its run()
method:
import calliope
model = calliope.Model(config_run='/path/to/run_configuration.yaml')
model.run()
If config_run
is not specified (i.e. model = Model()
), an error is raised. See Built-in example models for information on instantiating a simple example model without specifying a run configuration.
config_run
can also take an AttrDict
object containing the configuration. Furthermore, Model()
has an override
parameter, which takes an AttrDict
with settings that will override the given run settings.
After instantiating the Model
object, and before calling the run()
method, it is possible to manually inspect and adjust the configuration of the model.
After the model has been solved, an xarray Dataset containing solution variables and aggregated statistics is accessible under the solution
property on the model instance.
The API documentation gives an overview of the available methods for programmatic access.
Extracting results from a completed model run¶
If running single runs via the command-line tool or using the parallel run functionality, results will be saved as either a single NetCDF file per model run or a set of CSV files per model run. These can then be read back into an interactive Python session for analysis – see Analyzing results – or further processed with any other tool available to the modeller.
When working with the in-memory solution
object, which is an n-dimensional xarray.Dataset, the xarray documentation should be consulted (this will be the case either in interactive runs, or after having read it back into memory from disk),
It is easy to extract 2-dimensional slices from the solution by using xarray’s ability to extract pandas DataFrames. See the Tutorials for examples of how this is done.
The easiest path to extracting data from a model without dealing with xarray, pandas, or other Python data analysis tools, is to set the output.format
in the run configuration to csv
, which results in CSV files that can be read for example with common spreadsheet software.
Debugging failing runs¶
What will typically go wrong, in order of decreasing likelihood:
- The model is improperly defined or missing data. Calliope will attempt to diagnose some common errors and raise an appropriate error message.
- The model is consistent and properly defined but infeasible. Calliope will be able to construct the model and pass it on to the solver, but the solver (after a potentially long time) will abort with a message stating that the model is infeasible.
- There is a bug in Calliope causing the model to crash either before being passed to the solver, or after the solver has completed and when results are passed back to Calliope.
Calliope provides some run configuration options to make it easier to determine the cause of the first two of these possibilities. See the debugging options described in the configuration reference.
Python debugging¶
If using Calliope interactively in a Python session and/or developing custom constraints and analysis functionality, we recommend reading up on the Python debugger and (if using IPython or Jupyter Notebooks) making heavy use of the %debug magic.
Previous: Run configuration | Next: Analyzing results