The Calliope model and backend objects¶

In this tutorial, we use the urban scale example model to go into a bit more detail on the public and non-public properties of the calliope.Model and calliope.Model.backend objects.

In [1]:

Copied!

from pathlib import Path

import calliope

calliope.set_log_verbosity("INFO", include_solver_output=False)
from pathlib import Path

import calliope

calliope.set_log_verbosity("INFO", include_solver_output=False)

Model input¶

In [2]:

Copied!

# Initialise the model with the Urban Scale example model
m = calliope.examples.urban_scale()
# Initialise the model with the Urban Scale example model
m = calliope.examples.urban_scale()

[2024-09-10 15:51:43] INFO     Model: initialising

[2024-09-10 15:51:43] INFO     Model: preprocessing stage 1 (model_run)

[2024-09-10 15:51:49] INFO     Model: preprocessing stage 2 (model_data)

[2024-09-10 15:51:49] INFO     Model: preprocessing complete

In [3]:

Copied!

# Get information on the model
print(m.info())
# Get information on the model
print(m.info())

[2024-09-10 15:51:49] WARNING  /home/docs/checkouts/readthedocs.org/user_builds/calliope/conda/v0.7.0.dev4/lib/python3.12/site-packages/calliope/model.py:529: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  msize = dict(self._model_data.dims)

Model name:   Urban-scale example model
Model size:   {'costs': 1, 'techs': 12, 'carriers': 3, 'nodes': 4, 'timesteps': 48} (30 valid node:tech:carrier combinations)

Model definition dictionary¶

m._model_def_dict is a python dictionary that holds all the data from the model definition YAML files, restructured into one dictionary.

The underscore before the method indicates that it defaults to being hidden (i.e. you wouldn't see it by trying a tab auto-complete and it isn't documented)

In [4]:

Copied!

m._model_def_dict.keys()
m._model_def_dict.keys()

Out[4]:

dict_keys(['nodes', 'templates', 'techs', 'parameters'])

techs hold only the information about a technology that is specific to that node

In [5]:

Copied!

m._model_def_dict["techs"]["pv"]
m._model_def_dict["techs"]["pv"]

Out[5]:

{'name': 'Solar photovoltaic power',
 'color': '#F9D956',
 'base_tech': 'supply',
 'carrier_out': 'electricity',
 'template': 'interest_rate_setter',
 'carrier_export': 'electricity',
 'source_unit': 'per_area',
 'area_use_per_flow_cap': 7,
 'flow_out_parasitic_eff': 0.85,
 'flow_cap_max': 250,
 'area_use_max': 1500,
 'lifetime': 25,
 'cost_flow_cap': {'data': 1350, 'index': 'monetary', 'dims': 'costs'}}

nodes hold only the information about a technology that is specific to that node

In [6]:

Copied!

m._model_def_dict["nodes"]["X2"]["techs"]["pv"]
m._model_def_dict["nodes"]["X2"]["techs"]["pv"]

Out[6]:

{'cost_flow_out': {'data': -0.0203, 'index': 'monetary', 'dims': 'costs'},
 'cost_export': {'data': -0.0491, 'index': 'monetary', 'dims': 'costs'}}

Model data¶

m._model_data is an xarray Dataset. Like _model_def_dict it is a hidden prperty of the Model as you are expected to access the data via the public property inputs

Until we solve the model, inputs is the same as _model_data

We can find the same PV flow_cap_max data as seen in m._model_run

In [9]:

Copied!

m._model_data.flow_cap_max.sel(techs="pv").to_series().dropna()
m._model_data.flow_cap_max.sel(techs="pv").to_series().dropna()

Out[9]:

carriers     nodes
electricity  X1       250.0
             X2       250.0
             X3        50.0
Name: flow_cap_max, dtype: float64

Building and checking the optimisation problem¶

Calling m.build allows us to build the optimisation problem, which creates arrays of Python objects from the YAML math formulation.

In [10]:

Copied!

m.build()
m.build()

[2024-09-10 15:51:49] INFO     Model: backend build starting

[2024-09-10 15:51:50] INFO     Optimisation Model | parameters | Generated.

[2024-09-10 15:51:51] INFO     Optimisation Model | variables | Generated.

[2024-09-10 15:51:53] INFO     Optimisation Model | global_expressions | Generated.

[2024-09-10 15:51:56] INFO     Optimisation Model | constraints | Generated.

[2024-09-10 15:51:56] INFO     Optimisation Model | piecewise_constraints | Generated.

[2024-09-10 15:51:56] INFO     Optimisation Model | objectives | Generated.

[2024-09-10 15:51:56] INFO     Model: backend build complete

As with the calliope Model, the backend has its own dataset containing all the arrays of backend objects

There is then a public API to access filtered views on this dataset, e.g. input parameters...

... or constraints

You can also access backend arrays in text format, to debug the problem:

In [14]:

Copied!

m.backend.get_constraint(
    "area_use_capacity_per_loc", as_backend_objs=False
).to_pandas().dropna(how="all", axis=0)
m.backend.get_constraint(
    "area_use_capacity_per_loc", as_backend_objs=False
).to_pandas().dropna(how="all", axis=0)

Out[14]:

	lb	body	ub
nodes
X1	NaN	variables[area_use][0]	500.0
X2	NaN	variables[area_use][1]	1300.0
X3	NaN	variables[area_use][2]	900.0

We can increase the verbosity of the constraint/global expression "body" by calling the backend method verbose_strings. We do not do this automatically as it entails a memory/time overhead on building the model and is only necessary for debugging your optimisation problem.

In [15]:

Copied!





m.backend.verbose_strings()
m.backend.get_constraint(
    "area_use_capacity_per_loc", as_backend_objs=False
).to_pandas().dropna(how="all", axis=0)
m.backend.verbose_strings()
m.backend.get_constraint(
    "area_use_capacity_per_loc", as_backend_objs=False
).to_pandas().dropna(how="all", axis=0)

Out[15]:

	lb	body	ub
nodes
X1	NaN	variables[area_use][X1, pv]	500.0
X2	NaN	variables[area_use][X2, pv]	1300.0
X3	NaN	variables[area_use][X3, pv]	900.0

Updating the optimisation problem in-place¶

If we want to update a parameter value or fix a decision variable, we can do so now that we have built the optimisation problem

In [16]:

Copied!





m.backend.update_parameter("flow_cap_max", m.inputs.flow_cap_max * 2)
m.backend.get_parameter("flow_cap_max", as_backend_objs=False).sel(
    techs="pv"
).to_series().dropna()
m.backend.update_parameter("flow_cap_max", m.inputs.flow_cap_max * 2)
m.backend.get_parameter("flow_cap_max", as_backend_objs=False).sel(
    techs="pv"
).to_series().dropna()

Out[16]:

carriers     nodes
electricity  X1       500.0
             X2       500.0
             X3       100.0
Name: flow_cap_max, dtype: float64

Solve the optimisation problem¶

Once we have all of our optimisation problem components set up as we desire, we can solve the problem.

In [17]:

Copied!

m.solve()
m.solve()

[2024-09-10 15:51:57] INFO     Optimisation model | starting model in plan mode.

[2024-09-10 15:51:58] INFO     Backend: solver finished running. Time since start of solving optimisation problem: 0:00:00.762237

[2024-09-10 15:51:58] INFO     Postprocessing: zero threshold of 1e-10 not required

[2024-09-10 15:51:58] INFO     Postprocessing: ended. Time since start of solving optimisation problem: 0:00:00.982148

[2024-09-10 15:51:58] INFO     Backend: model solve completed. Time since start of solving optimisation problem: 0:00:00.989217

The results are stored in m._model_data and can be accessed by the public property m.results

We can also view the data within the backend directly

In [19]:

Copied!

m.backend.get_variable("flow_cap", as_backend_objs=False).to_series().dropna()
m.backend.get_variable("flow_cap", as_backend_objs=False).to_series().dropna()

Out[19]:

nodes  techs               carriers   
N1     N1_to_X2            heat           189.87211
       N1_to_X3            heat           10.382599
       X1_to_N1            heat           567.70227
X1     X1_to_N1            heat           567.70227
       X1_to_X2            electricity    274.76645
       X1_to_X3            electricity    47.452088
       chp                 electricity    260.44701
                           gas            643.07904
                           heat           208.35761
       demand_electricity  electricity     1.307938
       demand_heat         heat            1.719711
       pv                  electricity          0.0
       supply_gas          gas            643.07904
       supply_grid_power   electricity    34.114372
X2     N1_to_X2            heat           189.87211
       X1_to_X2            electricity    274.76645
       boiler              gas            227.26339
                           heat           193.17388
       demand_electricity  electricity    269.34435
       demand_heat         heat           365.13767
       pv                  electricity    14.358886
       supply_gas          gas            227.26339
X3     N1_to_X3            heat           10.382599
       X1_to_X3            electricity    47.452088
       boiler              gas             0.018353
                           heat              0.0156
       demand_electricity  electricity    64.927357
       demand_heat         heat            9.398229
       pv                  electricity        100.0
       supply_gas          gas             0.018353
Name: flow_cap, dtype: object

Save¶

In [20]:

Copied!





# We can save at any point, which will dump the entire m._model_data to file.
# NetCDF is recommended, as it retains most of the data and can be reloaded into a Calliope model at a later date.


output_path = Path(".") / "outputs" / "4_calliope_model_object"
output_path.mkdir(parents=True, exist_ok=True)

m.to_netcdf(output_path / "example.nc")  # Saves a single file
m.to_csv(
    output_path / "csv_files", allow_overwrite=True
)  # Saves a file for each xarray DataArray
# We can save at any point, which will dump the entire m._model_data to file.
# NetCDF is recommended, as it retains most of the data and can be reloaded into a Calliope model at a later date.


output_path = Path(".") / "outputs" / "4_calliope_model_object"
output_path.mkdir(parents=True, exist_ok=True)

m.to_netcdf(output_path / "example.nc")  # Saves a single file
m.to_csv(
    output_path / "csv_files", allow_overwrite=True
)  # Saves a file for each xarray DataArray