Development guide¶
The code lives on GitHub at calliope-project/calliope.
Development takes place in the master
branch. Stable versions are tagged off of master
with semantic versioning.
Tests are included and can be run with py.test
from the project’s root directory.
See the list of open issues and planned milestones for an overview of where development is heading, and join us on Gitter to ask questions or discuss code.
Installing a development version¶
As when installing a stable version, using conda
is recommended.
First, clone the repository:
$ git clone https://github.com/calliope-project/calliope
Using Anaconda/conda, install all requirements, including the free and open source GLPK solver, into a new environment, e.g. calliope_dev
:
$ conda env create -f ./calliope/requirements.yml -n calliope_dev
$ source activate calliope_dev
Then install Calliope itself with pip:
$ pip install -e ./calliope
Creating modular extensions¶
Constraint generator functions¶
By making use of the ability to load custom constraint generator functions (see Loading optional constraints), a Calliope model can be extended by additional constraints easily without modifying the core code.
Constraint generator functions are called during construction of the model with the Model
object passed as the only parameter.
The Model
object provides, amongst other things:
- The Pyomo model instance, under the property
m
- The model data under the
data
property - An easy way to access model configuration with the
get_option()
method
A constraint generator function can add constraints, parameters, and variables directly to the Pyomo model instance (Model.m
). Refer to the Pyomo documentation for information on how to construct these model components.
The default cost-minimizing objective function provides a good example:
import pyomo.core as po # pylint: disable=import-error
def objective_cost_minimization(model):
"""
Minimizes total system monetary cost.
Used as a default if a model does not specify another objective.
"""
m = model.m
def obj_rule(m):
return sum(model.get_option(y + '.weight') *
sum(m.cost[y, x, 'monetary']
for x in m.x) for y in m.y)
m.obj = po.Objective(sense=po.minimize, rule=obj_rule)
m.obj.domain = po.Reals
See the source code of the ramping_rate()
function for a more elaborate example.
The process of including custom, optional constraints is as follows:
First, create the source code (see e.g. the above example for the ramping_rate
function) in a file, for example my_constraints.py
Then, assuming your custom constraint generator function is called my_first_custom_constraint
and is defined in my_constraints.py
, you can tell Calliope to load it by adding it to the list of optional constraints in your model configuration as follows:
constraints:
- constraints.optional.ramping_rate
- my_constraints.my_first_custom_constraint
This assumes that the file my_constraints.py
is importable when the model is run. It must therefore either be in the directory from which the model is run, installed as a Python module (see this document on how to create importable and installable Python packages), or the Python import path has to be adjusted according to the official Python documentation.
Subsets¶
Calliope internally builds many subsets to better manage constraints, in particular, subsets of different groups of technologies. These subsets can be used in the definition of constraints and are used extensively in the definition of Calliope’s built-in constraints. See the detailed definitions in calliope.sets
, an overview of which is included here.
Main sets & sub-sets¶
Technologies:
m.y_demand
: all demand sourcesm.y_sd_r_area
: if any r_area constraints are defined (shared)m.y_sd_finite_r
: if finite resource limit is defined (shared)
m.y_supply
: all basic supply technologiesm.y_sd_r_area
: if any r_area constraints are defined (shared)m.y_sd_finite_r
: if finite resource limit is defined (shared)
m.y_storage
: specifically storage technologiesm.y_supply_plus
: all supply+ technologiesm.y_sp_r_area
: If any r_area constraints are definedm.y_sp_finite_r
: if finite resource limit is definedm.y_sp_r2
: if secondary resource is allowed
m.y_conversion
: all basic conversion technologiesm.y_conversion_plus
: all conversion+ technologiesm.y_cp_2out
: secondary carrier(s) outm.y_cp_3out
: tertiary carrier(s) outm.y_cp_2in
: secondary carrier(s) inm.y_cp_3in
: tertiary carrier(s) in
m.y_transmission
: all transmission technologiesm.y_unmet
: dummy supply technologies to log
Locations:
m.x_trans
: all transmission locationsm.x_r
: all locations which act as system sources/sinksm.x_store
: all locations in which storage is allowed
Meta-sets¶
Technologies:
m.y
: all technologies, includes:m.y_demand
m.y_supply
m.y_storage
m.y_supply_plus
m.y_conversion
m.y_conversion_plus
m.y_transmission
m.y_sd
: all basic supply & demand technologies, includes:m.y_demand
m.y_supply
m.y_store
: all technologies that have storage capabilities, includes:m.y_storage
m.y_supply_plus
Locations:
m.x
: all locations, includes:m.x_trans
m.x_r
m.x_store
Time functions and masks¶
Like custom constraint generator functions, custom functions that adjust time resolution can be loaded dynamically during model initialization. By default, Calliope first checks whether the name of a function or time mask refers to a function from the calliope.time_masks
or calliope.time_functions
module, and if not, attempts to load the function from an importable module:
time:
masks:
- {function: week, options: {day_func: 'extreme', tech: 'wind', how: 'min'}}
- {function: my_custom_module.my_custom_mask, options: {...}}
function: my_custom_module.my_custom_function
function_options: {...}
Profiling¶
To profile a Calliope run with the built-in national-scale example model, then visualize the results with snakeviz:
make profile # will dump profile output in the current directory
snakeviz calliope.profile # launch snakeviz to visually examine profile
Use mprof plot
to plot memory use.
Other options for visualizing:
Interactive visualization with KCachegrind (on macOS, use QCachegrind, installed e.g. with
brew install qcachegrind
)pyprof2calltree -i calliope.profile -o calliope.calltree kcachegrind calliope.calltree
Generate a call graph from the call tree via graphviz
# brew install gprof2dot gprof2dot -f callgrind calliope.calltree | dot -Tsvg -o callgraph.svg
Checklist for new release¶
Pre-release¶
- Make sure all unit tests pass
- Make sure documentation builds without errors
- Make sure the release notes are up-to-date, especially that new features and backward incompatible changes are clearly marked
Create release¶
- Change
_version.py
version number - Update changelog with final version number and release date
- Commit with message “Release vXXXX”, then add a “vXXXX” tag, push both to GitHub
- Create a release through the GitHub web interface, using the same tag, titling it “Release vXXXX” (required for Zenodo to pull it in)
- Upload new release to PyPI:
make all-dist
- Update the conda-forge package:
- Fork conda-forge/calliope-feedstock, and update
recipe/meta.yaml
with: - Version number:
{% set version = "XXXX" %}
- MD5 of latest version from PyPI:
{% set md5 = "XXXX" %}
- Reset
build: number: 0
if it is not already at zero - If necessary, carry over any changed requirements from
requirements.yml
orsetup.py
- Version number:
- Fork conda-forge/calliope-feedstock, and update
- Submit a pull request from an appropriately named branch in your fork (e.g.
vXXXX
) to the conda-forge/calliope-feedstock repository
Post-release¶
- Update changelog, adding a new vXXXX-dev heading, and update
_version.py
accordingly, in preparation for the next master commit
Note
Adding ‘-dev’ to the version string, such as __version__ = '0.1.0-dev'
, is required for the custom code in doc/conf.py
to work when building in-development versions of the documentation.
Previous: Built-in example models | Next: API Documentation