Skip to content

calliope.backend.backend_model.BackendModel(inputs, instance, **kwargs)

Bases: BackendModelGenerator, Generic[T]

Abstract base class to build backend models that interface with solvers.

Parameters:

Name Type Description Default
inputs Dataset

Calliope model data.

required
instance T

Interface model instance.

required
Source code in src/calliope/backend/backend_model.py
def __init__(self, inputs: xr.Dataset, instance: T, **kwargs) -> None:
    """Abstract base class to build backend models that interface with solvers.

    Args:
        inputs (xr.Dataset): Calliope model data.
        instance (T): Interface model instance.
    """
    super().__init__(inputs, **kwargs)
    self._instance = instance
    self.shadow_prices: ShadowPrices

constraints property

Slice of backend dataset to show only built constraints

global_expressions property

Slice of backend dataset to show only built global expressions

inputs = inputs.copy() instance-attribute

objectives property

Slice of backend dataset to show only built objectives

parameters property

Slice of backend dataset to show only built parameters

shadow_prices: ShadowPrices instance-attribute

valid_component_names property

variables property

Slice of backend dataset to show only built variables

add_constraint(name, constraint_dict) abstractmethod

Add constraint equation to backend model in-place. Resulting backend dataset entries will be constraint objects.

Parameters:

Name Type Description Default
name str

Name of the constraint

required
constraint_dict UnparsedConstraintDict

Constraint configuration dictionary, ready to be parsed and then evaluated.

required
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def add_constraint(
    self, name: str, constraint_dict: parsing.UnparsedConstraintDict
) -> None:
    """
    Add constraint equation to backend model in-place.
    Resulting backend dataset entries will be constraint objects.

    Args:
        name (str):
            Name of the constraint
        constraint_dict (parsing.UnparsedConstraintDict):
            Constraint configuration dictionary, ready to be parsed and then evaluated.
    """

add_global_expression(name, expression_dict) abstractmethod

Add global expression (arithmetic combination of parameters and/or decision variables) to backend model in-place. Resulting backend dataset entries will be linear expression objects.

Parameters:

Name Type Description Default
name str

Name of the global expression

required
expression_dict UnparsedExpressionDict

Global expression configuration dictionary, ready to be parsed and then evaluated.

required
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def add_global_expression(
    self, name: str, expression_dict: parsing.UnparsedExpressionDict
) -> None:
    """
    Add global expression (arithmetic combination of parameters and/or decision variables)
    to backend model in-place.
    Resulting backend dataset entries will be linear expression objects.

    Args:
        name (str):
            Name of the global expression
        expression_dict (parsing.UnparsedExpressionDict):
            Global expression configuration dictionary, ready to be parsed and then evaluated.
    """

add_objective(name, objective_dict) abstractmethod

Add objective arithmetic to backend model in-place. Resulting backend dataset entry will be a single, unindexed objective object.

Parameters:

Name Type Description Default
name str

Name of the objective.

required
objective_dict UnparsedObjectiveDict

Objective configuration dictionary, ready to be parsed and then evaluated.

required
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def add_objective(
    self, name: str, objective_dict: parsing.UnparsedObjectiveDict
) -> None:
    """
    Add objective arithmetic to backend model in-place.
    Resulting backend dataset entry will be a single, unindexed objective object.

    Args:
        name (str):
            Name of the objective.
        objective_dict (parsing.UnparsedObjectiveDict):
            Objective configuration dictionary, ready to be parsed and then evaluated.
    """

add_parameter(parameter_name, parameter_values, default=np.nan, use_inf_as_na=False) abstractmethod

Add input parameter to backend model in-place. If the backend interface allows for mutable parameter objects, they will be generated, otherwise a copy of the model input dataset will be used. In either case, NaN values are filled with the given parameter default value.

Parameters:

Name Type Description Default
parameter_name str

Name of parameter.

required
parameter_values DataArray

Array of parameter values.

required
default Any

Default value to fill NaN entries in parameter values array. Defaults to np.nan.

nan
use_inf_as_na bool

If True, will consider np.inf parameter value entries as np.nan and consequently try to fill those entries with the parameter default value. Defaults to False.

False
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def add_parameter(
    self,
    parameter_name: str,
    parameter_values: xr.DataArray,
    default: Any = np.nan,
    use_inf_as_na: bool = False,
) -> None:
    """
    Add input parameter to backend model in-place.
    If the backend interface allows for mutable parameter objects, they will be
    generated, otherwise a copy of the model input dataset will be used.
    In either case, NaN values are filled with the given parameter default value.

    Args:
        parameter_name (str): Name of parameter.
        parameter_values (xr.DataArray): Array of parameter values.
        default (Any, optional):
            Default value to fill NaN entries in parameter values array.
            Defaults to np.nan.
        use_inf_as_na (bool, optional):
            If True, will consider np.inf parameter value entries as np.nan and
            consequently try to fill those entries with the parameter default value.
            Defaults to False.
    """

add_variable(name, variable_dict) abstractmethod

Add decision variable to backend model in-place. Resulting backend dataset entries will be decision variable objects.

Parameters:

Name Type Description Default
name str

Name of the variable.

required
variable_dict UnparsedVariableDict

Variable configuration dictionary, ready to be parsed and then evaluated.

required
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def add_variable(
    self, name: str, variable_dict: parsing.UnparsedVariableDict
) -> None:
    """
    Add decision variable to backend model in-place.
    Resulting backend dataset entries will be decision variable objects.

    Args:
        name (str):
            Name of the variable.
        variable_dict (parsing.UnparsedVariableDict):
            Variable configuration dictionary, ready to be parsed and then evaluated.
    """

delete_component(key, component_type) abstractmethod

Delete a list object from the backend model object.

Parameters:

Name Type Description Default
key str

Name of object.

required
component_type str

Object type.

required
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def delete_component(self, key: str, component_type: _COMPONENTS_T) -> None:
    """Delete a list object from the backend model object.

    Args:
        key (str): Name of object.
        component_type (str): Object type.
    """

fix_variable(name, where=None) abstractmethod

Fix the variable value to the value quantified on the most recent call to solve. Fixed variables will be treated as parameters in the optimisation.

Parameters:

Name Type Description Default
name str

Variable to update.

required
where DataArray

If provided, only a subset of the coordinates in the variable will be fixed. Must be a boolean array or a float equivalent, where NaN is used instead of False. Defaults to None

None
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def fix_variable(self, name: str, where: Optional[xr.DataArray] = None) -> None:
    """
    Fix the variable value to the value quantified on the most recent call to `solve`.
    Fixed variables will be treated as parameters in the optimisation.

    Args:
        name (str): Variable to update.
        where (xr.DataArray, optional):
            If provided, only a subset of the coordinates in the variable will be fixed.
            Must be a boolean array or a float equivalent, where NaN is used instead of False.
            Defaults to None
    """

get_constraint(name, as_backend_objs=True, eval_body=False) abstractmethod

Get constraint data as either a table of details or as an array of backend interface objects. Can be used to inspect and debug built constraints.

Parameters:

Name Type Description Default
name str

Name of constraint, as given in YAML constraint key.

required
as_backend_objs bool

TODO: hide this and create a method to edit constraints that handles differences in interface APIs If True, will keep the array entries as backend interface objects, which can be updated to change the underlying model. Otherwise, constraint body, and lower and upper bounds are given in a table. Defaults to True.

True
eval_body bool

If True and as_backend_objs is False, will attempt to evaluate the constraint body. If the model has been optimised, this attempt will produce a numeric value to see where the constraint sits between the lower or upper bound. If the model has not yet been optimised, this attempt will fall back on the same as if eval_body was set to False, i.e. a string representation of the linear expression in the constraint body. Defaults to False.

False

Returns:

Type Description
Union[DataArray, Dataset]

Union[xr.DataArray, xr.Dataset]: If as_backend_objs is True, will return an xr.DataArray. Otherwise, a xr.Dataset will be given, indexed over the same dimensions as the xr.DataArray, with variables for the constraint body, and upper (ub) and lower (lb) bounds.

Source code in src/calliope/backend/backend_model.py
@abstractmethod
def get_constraint(
    self, name: str, as_backend_objs: bool = True, eval_body: bool = False
) -> Union[xr.DataArray, xr.Dataset]:
    """
    Get constraint data as either a table of details or as an array of backend interface objects.
    Can be used to inspect and debug built constraints.

    Args:
        name (str): Name of constraint, as given in YAML constraint key.
        as_backend_objs (bool, optional): TODO: hide this and create a method to edit constraints that handles differences in interface APIs
            If True, will keep the array entries as backend interface objects,
            which can be updated to change the underlying model.
            Otherwise, constraint body, and lower and upper bounds are given in a table.
            Defaults to True.
        eval_body (bool, optional):
            If True and as_backend_objs is False, will attempt to evaluate the constraint body.
            If the model has been optimised, this attempt will produce a numeric value to see where the constraint sits between the lower or upper bound.
            If the model has not yet been optimised, this attempt will fall back on the same as
            if `eval_body` was set to False, i.e. a string representation of the linear expression in the constraint body.
            Defaults to False.

    Returns:
        Union[xr.DataArray, xr.Dataset]:
            If as_backend_objs is True, will return an xr.DataArray.
            Otherwise, a xr.Dataset will be given, indexed over the same dimensions as the xr.DataArray, with variables for the constraint body, and upper (`ub`) and lower (`lb`) bounds.
    """

get_global_expression(name, as_backend_objs=True, eval_body=True) abstractmethod

Extract global expression array from backend dataset

Parameters:

Name Type Description Default
name str

Name of global expression

required
as_backend_objs bool

TODO: hide this and create a method to edit expressions that handles differences in interface APIs. If True, will keep the array entries as backend interface objects, which can be updated to update the underlying model. Otherwise, global expression values are given directly. If the model has not been successfully optimised, expression values will all be provided as strings. Defaults to True.

True
eval_body bool

If True and as_backend_objs is False, will attempt to evaluate the expression. If the model has been optimised, this attempt will produce a numeric value. If the model has not yet been optimised, this attempt will fall back on the same as if eval_body was set to False, i.e. a string representation of the linear expression. Defaults to True.

True

Returns:

Type Description
DataArray

xr.DataArray: global expression array.

Source code in src/calliope/backend/backend_model.py
@abstractmethod
def get_global_expression(
    self, name: str, as_backend_objs: bool = True, eval_body: bool = True
) -> xr.DataArray:
    """Extract global expression array from backend dataset

    Args:
        name (str): Name of global expression
        as_backend_objs (bool, optional): TODO: hide this and create a method to edit expressions that handles differences in interface APIs.
            If True, will keep the array entries as backend interface objects,
            which can be updated to update the underlying model.
            Otherwise, global expression values are given directly.
            If the model has not been successfully optimised, expression values will all be provided as strings.
            Defaults to True.
        eval_body (bool, optional):
            If True and as_backend_objs is False, will attempt to evaluate the expression.
            If the model has been optimised, this attempt will produce a numeric value.
            If the model has not yet been optimised, this attempt will fall back on the same as
            if `eval_body` was set to False, i.e. a string representation of the linear expression.
            Defaults to True.

    Returns:
        xr.DataArray: global expression array.
    """

get_parameter(name, as_backend_objs=True) abstractmethod

Extract parameter from backend dataset.

Parameters:

Name Type Description Default
name str

Name of parameter.

required
as_backend_objs bool

TODO: hide this and create a method to edit parameter values (to handle interfaces with non-mutable params) If True, will keep the array entries as backend interface objects, which can be updated to update the underlying model. Otherwise, parameter values are given directly, with default values in place of NaNs. Defaults to True.

True

Returns:

Type Description
DataArray

xr.DataArray: parameter array.

Source code in src/calliope/backend/backend_model.py
@abstractmethod
def get_parameter(self, name: str, as_backend_objs: bool = True) -> xr.DataArray:
    """
    Extract parameter from backend dataset.

    Args:
        name (str): Name of parameter.
        as_backend_objs (bool, optional): TODO: hide this and create a method to edit parameter values (to handle interfaces with non-mutable params)
            If True, will keep the array entries as backend interface objects,
            which can be updated to update the underlying model.
            Otherwise, parameter values are given directly, with default values in place of NaNs.
            Defaults to True.

    Returns:
        xr.DataArray: parameter array.
    """

get_variable(name, as_backend_objs=True) abstractmethod

Extract decision variable array from backend dataset

Parameters:

Name Type Description Default
name str

Name of variable.

required
as_backend_objs bool

TODO: hide this and create a method to edit variables that handles differences in interface APIs. If True, will keep the array entries as backend interface objects, which can be updated to update the underlying model. Otherwise, variable values are given directly. If the model has not been successfully optimised, variable values will all be None. Defaults to True.

True

Returns:

Type Description
DataArray

xr.DataArray: Decision variable array.

Source code in src/calliope/backend/backend_model.py
@abstractmethod
def get_variable(self, name: str, as_backend_objs: bool = True) -> xr.DataArray:
    """Extract decision variable array from backend dataset

    Args:
        name (str): Name of variable.
        as_backend_objs (bool, optional): TODO: hide this and create a method to edit variables that handles differences in interface APIs.
            If True, will keep the array entries as backend interface objects,
            which can be updated to update the underlying model.
            Otherwise, variable values are given directly.
            If the model has not been successfully optimised, variable values will all be None.
            Defaults to True.

    Returns:
        xr.DataArray: Decision variable array.
    """

get_variable_bounds(name) abstractmethod

Extract decision variable upper and lower bound array from backend dataset

Parameters:

Name Type Description Default
name str

Name of variable.

required

Returns:

Type Description
Dataset

xr.Dataset: Contains the arrays for upper ("ub", a.k.a. "max") and lower ("lb", a.k.a. "min") variable bounds.

Source code in src/calliope/backend/backend_model.py
@abstractmethod
def get_variable_bounds(self, name: str) -> xr.Dataset:
    """Extract decision variable upper and lower bound array from backend dataset

    Args:
        name (str): Name of variable.

    Returns:
        xr.Dataset: Contains the arrays for upper ("ub", a.k.a. "max") and lower ("lb", a.k.a. "min") variable bounds.
    """

load_results()

Evaluate backend decision variables, global expressions, and parameters (if not in inputs) after a successful model run.

Returns:

Type Description
Dataset

xr.Dataset: Dataset of optimal solution results (all numeric data).

Source code in src/calliope/backend/backend_model.py
def load_results(self) -> xr.Dataset:
    """
    Evaluate backend decision variables, global expressions, and parameters (if not in inputs)
    after a successful model run.

    Returns:
        xr.Dataset: Dataset of optimal solution results (all numeric data).
    """

    def _drop_attrs(da):
        da.attrs = {
            k: v for k, v in da.attrs.items() if k in self._COMPONENT_ATTR_METADATA
        }
        return da

    all_variables = {
        name_: _drop_attrs(self.get_variable(name_, as_backend_objs=False))
        for name_, var in self.variables.items()
        if var.notnull().any()
    }
    all_global_expressions = {
        name_: _drop_attrs(
            self.get_global_expression(name_, as_backend_objs=False, eval_body=True)
        )
        for name_, expr in self.global_expressions.items()
        if expr.notnull().any()
    }

    results = xr.Dataset({**all_variables, **all_global_expressions}).astype(float)

    return results

log(component_type, component_name, message, level='debug')

Log to module-level logger with some prettification of the message

Parameters:

Name Type Description Default
message str

Message to log.

required
level Literal['info', 'warning', 'debug', 'error', 'critical']

Log level. Defaults to "debug".

'debug'
Source code in src/calliope/backend/backend_model.py
def log(
    self,
    component_type: _COMPONENTS_T,
    component_name: str,
    message: str,
    level: Literal["info", "warning", "debug", "error", "critical"] = "debug",
):
    """Log to module-level logger with some prettification of the message

    Args:
        message (str): Message to log.
        level (Literal["info", "warning", "debug", "error", "critical"], optional): Log level. Defaults to "debug".
    """
    getattr(LOGGER, level)(
        f"Optimisation model | {component_type}:{component_name} | {message}"
    )

to_lp(path) abstractmethod

Write the optimisation problem to file in the linear programming LP format. The LP file can be used for debugging and to submit to solvers directly.

Parameters:

Name Type Description Default
path Union[str, Path]

Path to which the LP file will be written.

required
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def to_lp(self, path: Union[str, Path]) -> None:
    """Write the optimisation problem to file in the linear programming LP format.
    The LP file can be used for debugging and to submit to solvers directly.

    Args:
        path (Union[str, Path]): Path to which the LP file will be written.
    """

unfix_variable(name, where=None) abstractmethod

Unfix the variable so that it is treated as a decision variable in the next call to solve.

Parameters:

Name Type Description Default
name str

Variable to update

required
where DataArray

If provided, only a subset of the coordinates in the variable will be unfixed. Must be a boolean array or a float equivalent, where NaN is used instead of False. Defaults to None

None
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def unfix_variable(self, name: str, where: Optional[xr.DataArray] = None) -> None:
    """
    Unfix the variable so that it is treated as a decision variable in the next call to `solve`.

    Args:
        name (str): Variable to update
        where (xr.DataArray, optional):
            If provided, only a subset of the coordinates in the variable will be unfixed.
            Must be a boolean array or a float equivalent, where NaN is used instead of False.
            Defaults to None
    """

update_parameter(name, new_values) abstractmethod

Update parameter elements using an array of new values. If the parameter has not been previously defined, it will be added to the optimisation problem based on the new values given (with NaNs reverting to default values). If the new values have fewer dimensions than are on the parameter array, the new values will be broadcast across the missing dimensions before applying the update.

Parameters:

Name Type Description Default
name str

Parameter to update

required
new_values Union[DataArray, SupportsFloat]

New values to apply. Any empty (NaN) elements in the array will be skipped.

required
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def update_parameter(
    self, name: str, new_values: Union[xr.DataArray, SupportsFloat]
) -> None:
    """Update parameter elements using an array of new values.
    If the parameter has not been previously defined, it will be added to the optimisation problem based on the new values given (with NaNs reverting to default values).
    If the new values have fewer dimensions than are on the parameter array, the new values will be broadcast across the missing dimensions before applying the update.

    Args:
        name (str): Parameter to update
        new_values (Union[xr.DataArray, SupportsFloat]): New values to apply. Any empty (NaN) elements in the array will be skipped.
    """

update_variable_bounds(name, *, min=None, max=None) abstractmethod

Update the bounds on a decision variable. If the variable bounds are defined by parameters in the math formulation, the parameters themselves will be updated.

Parameters:

Name Type Description Default
name str

Variable to update.

required
min Union[DataArray, SupportsFloat]

If provided, the Non-NaN values in the array will be used to defined new lower bounds in the decision variable. Defaults to None.

None
max Union[DataArray, SupportsFloat]

If provided, the Non-NaN values in the array will be used to defined new upper bounds in the decision variable. Defaults to None.

None
Source code in src/calliope/backend/backend_model.py
@abstractmethod
def update_variable_bounds(
    self,
    name: str,
    *,
    min: Optional[Union[xr.DataArray, SupportsFloat]] = None,
    max: Optional[Union[xr.DataArray, SupportsFloat]] = None,
) -> None:
    """
    Update the bounds on a decision variable.
    If the variable bounds are defined by parameters in the math formulation,
    the parameters themselves will be updated.

    Args:
        name (str): Variable to update.
        min (Union[xr.DataArray, SupportsFloat], optional):
            If provided, the Non-NaN values in the array will be used to defined new lower bounds in the decision variable.
            Defaults to None.
        max (Union[xr.DataArray, SupportsFloat], optional):
            If provided, the Non-NaN values in the array will be used to defined new upper bounds in the decision variable.
            Defaults to None.
    """

verbose_strings() abstractmethod

Update optimisation model object string representations to include the index coordinates of the object.

E.g., variables(flow_out)[0] will become variables(flow_out)[power, region1, ccgt, 2005-01-01 00:00]

This takes approximately 10% of the peak memory required to initially build the optimisation problem, so should only be invoked if inspecting the model in detail (e.g., debugging)

Only string representations of model parameters and variables will be updated since global expressions automatically show the string representation of their contents.

Source code in src/calliope/backend/backend_model.py
@abstractmethod
def verbose_strings(self) -> None:
    """
    Update optimisation model object string representations to include the index coordinates of the object.

    E.g., `variables(flow_out)[0]` will become `variables(flow_out)[power, region1, ccgt, 2005-01-01 00:00]`

    This takes approximately 10% of the peak memory required to initially build the optimisation problem, so should only be invoked if inspecting the model in detail (e.g., debugging)

    Only string representations of model parameters and variables will be updated since global expressions automatically show the string representation of their contents.
    """