# Tutorial 3: Mixed Integer Linear Programming¶

This example is based on the urban scale example model, but with an override. In the model’s scenarios.yaml file overrides are defined which trigger binary and integer decision variables, creating a MILP model, rather than a conventional LP model.

## Units¶

The capacity of a technology is usually a continuous decision variable, which can be within the range of 0 and energy_cap_max (the maximum capacity of a technology). In this model, we introduce a unit limit on the CHP instead:

    chp:
constraints:
units_max: 4
energy_cap_per_unit: 300
energy_cap_min_use: 0.2
costs:
monetary:
energy_cap: 700
purchase: 40000


A unit maximum allows a discrete, integer number of CHP to be purchased, each having a capacity of energy_cap_per_unit. Any of energy_cap_max, energy_cap_min, or energy_cap_equals are now ignored, in favour of units_max, units_min, or units_equals. A useful feature unlocked by introducing this is the ability to set a minimum operating capacity which is only enforced when the technology is operating. In the LP model, energy_cap_min_use would force the technology to operate at least at that proportion of its maximum capacity at each time step. In this model, the newly introduced energy_cap_min_use of 0.2 will ensure that the output of the CHP is 20% of its maximum capacity in any time step in which it has a non-zero output.

## Purchase cost¶

The boiler does not have a unit limit, it still utilises the continuous variable for its capacity. However, we have introduced a purchase cost:

    boiler:
costs:
monetary:
energy_cap: 35
purchase: 2000


By introducing this, the boiler now has a binary decision variable associated with it, which is 1 if the boiler has a non-zero energy_cap (i.e. the optimisation results in investment in a boiler) and 0 if the capacity is 0. The purchase cost is applied to the binary result, providing a fixed cost on purchase of the technology, irrespective of the technology size. In physical terms, this may be associated with the cost of pipework, land purchase, etc. The purchase cost is also imposed on the CHP, which is applied to the number of integer CHP units in which the solver chooses to invest.

MILP functionality can be easily applied, but convergence is slower as a result of integer/binary variables. It is recommended to use a commercial solver (e.g. Gurobi, CPLEX) if you wish to utilise these variables outside this example model.

## Running the model¶

We now take you through running the model in a Jupyter notebook, which is included fully below. To download and run the notebook yourself, you can find it here. You will need to have Calliope installed.

Notebook

# Calliope Urban Scale MILP Example Model¶

For more details on analysing input/output data, see the full urban scale example model

In :
import calliope

# We increase logging verbosity
calliope.set_log_level('INFO')

In :
model = calliope.examples.milp()

# Note, we see the overrides that we have applied printed here, thanks to inreasing logging verbosity

[2018-10-04 16:31:00] INFO: Model: initialising
[2018-10-04 16:31:00] INFO: Applying overrides milp without a specific scenario name.
[2018-10-04 16:31:00] INFO: Override applied to model.name: Urban-scale example model -> Urban-scale example model with MILP
techs.boiler.costs.monetary.energy_cap:35 applied from override as new configuration
techs.boiler.costs.monetary.purchase:2000 applied from override as new configuration
techs.chp.constraints.energy_cap_min_use:0.2 applied from override as new configuration
techs.chp.constraints.energy_cap_per_unit:300 applied from override as new configuration
techs.chp.constraints.units_max:4 applied from override as new configuration
Override applied to techs.chp.costs.monetary.energy_cap: 750 -> 700
techs.chp.costs.monetary.purchase:40000 applied from override as new configuration
[2018-10-04 16:31:00] INFO: Model: preprocessing stage 1 (model_run)
[2018-10-04 16:31:02] INFO: Model: preprocessing stage 2 (model_data)
[2018-10-04 16:31:03] INFO: Model: preprocessing complete. Time since start: 0:00:02.902346

In :
# Model inputs can be viewed at model.inputs.
# Variables are indexed over any combination of techs, locs, carriers, costs and timesteps,
# although techs, locs, and carriers are often concatenated.
# e.g. chp, X1, heat -> X1::chp::heat
model.inputs

Out:
<xarray.Dataset>
Dimensions:                               (carrier_tiers: 3, carriers: 3, coordinates: 2, costs: 1, loc_carriers: 10, loc_tech_carriers_conversion_plus: 3, loc_techs: 26, loc_techs_area: 3, loc_techs_conversion: 2, loc_techs_conversion_plus: 1, loc_techs_export: 4, loc_techs_finite_resource: 9, loc_techs_investment_cost: 20, loc_techs_milp: 1, loc_techs_non_conversion: 23, loc_techs_om_cost: 9, loc_techs_supply_plus: 3, loc_techs_transmission: 10, locs: 4, techs: 9, timesteps: 48)
Coordinates:
* carrier_tiers                         (carrier_tiers) <U5 'out_2' 'out' 'in'
* loc_techs_export                      (loc_techs_export) object 'X3::pv' ...
* loc_techs_finite_resource             (loc_techs_finite_resource) object 'X2::demand_heat' ...
* costs                                 (costs) object 'monetary'
* loc_techs_investment_cost             (loc_techs_investment_cost) object 'X1::supply_grid_power' ...
* loc_techs_milp                        (loc_techs_milp) object 'X1::chp'
* loc_techs_supply_plus                 (loc_techs_supply_plus) object 'X3::pv' ...
* carriers                              (carriers) object 'gas' 'heat' ...
* coordinates                           (coordinates) object 'y' 'x'
* loc_carriers                          (loc_carriers) object 'X2::electricity' ...
* loc_techs_non_conversion              (loc_techs_non_conversion) object 'X2::demand_heat' ...
* locs                                  (locs) object 'N1' 'X2' 'X1' 'X3'
* techs                                 (techs) object 'heat_pipes' ...
* loc_tech_carriers_conversion_plus     (loc_tech_carriers_conversion_plus) object 'X1::chp::gas' ...
* loc_techs_conversion_plus             (loc_techs_conversion_plus) <U7 'X1::chp' ...
* loc_techs_area                        (loc_techs_area) object 'X3::pv' ...
* timesteps                             (timesteps) datetime64[ns] 2005-07-01 ...
* loc_techs_om_cost                     (loc_techs_om_cost) object 'X1::supply_grid_power' ...
* loc_techs_conversion                  (loc_techs_conversion) <U10 'X3::boiler' ...
* loc_techs                             (loc_techs) object 'X2::demand_heat' ...
* loc_techs_transmission                (loc_techs_transmission) object 'X3::heat_pipes:N1' ...
Data variables:
energy_cap_min_use                    (loc_techs) float64 nan nan nan ...
resource_unit                         (loc_techs_finite_resource) <U15 'energy' ...
resource_area_per_energy_cap          (loc_techs_area) int64 7 7 7
energy_cap_max                        (loc_techs) float64 nan 2e+03 ...
energy_eff                            (loc_techs) float64 nan nan nan ...
parasitic_eff                         (loc_techs_supply_plus) float64 0.85 ...
lifetime                              (loc_techs) float64 nan 25.0 25.0 ...
resource_eff                          (loc_techs_finite_resource) float64 nan ...
energy_prod                           (loc_techs) float64 nan 1.0 1.0 ...
units_max                             (loc_techs_milp) int64 4
energy_cap_per_unit                   (loc_techs) float64 nan nan nan ...
energy_con                            (loc_techs) float64 1.0 nan nan ...
resource_area_max                     (loc_techs_area) int64 1500 1500 1500
force_resource                        (loc_techs_finite_resource) bool True ...
export_carrier                        (loc_techs_export) <U11 'electricity' ...
resource                              (loc_techs_finite_resource, timesteps) float64 -64.73 ...
reserve_margin                        (carriers) float64 nan nan nan
cost_export                           (costs, loc_techs_om_cost, timesteps) float64 nan ...
cost_om_con                           (costs, loc_techs_om_cost) float64 0.1 ...
cost_energy_cap                       (costs, loc_techs_investment_cost) float64 100.0 ...
cost_om_prod                          (costs, loc_techs_om_cost) float64 nan ...
cost_om_annual                        (costs, loc_techs_om_cost) float64 nan ...
cost_depreciation_rate                (costs, loc_techs_investment_cost) float64 0.1102 ...
cost_purchase                         (costs, loc_techs_investment_cost) float64 nan ...
distance                              (loc_techs_transmission) float64 4.0 ...
lookup_remotes                        (loc_techs_transmission) <U18 'N1::heat_pipes:X3' ...
available_area                        (locs) float64 nan 1.3e+03 500.0 900.0
loc_coordinates                       (coordinates, locs) int64 7 7 7 3 ...
colors                                (techs) <U7 '#823739' '#6783E3' ...
inheritance                           (techs) <U29 'transmission' ...
names                                 (techs) <U29 'District heat distribution' ...
carrier_ratios                        (carrier_tiers, loc_tech_carriers_conversion_plus) float64 1.0 ...
carrier_ratios_min                    (carrier_tiers, loc_techs_conversion_plus) float64 0.8 ...
lookup_loc_carriers                   (loc_carriers) <U175 'X2::demand_electricity::electricity,X2::pv::electricity,X2::power_lines:X1::electricity' ...
lookup_loc_techs                      (loc_techs_non_conversion) <U35 'X2::demand_heat::heat' ...
lookup_loc_techs_conversion           (carrier_tiers, loc_techs_conversion) object None ...
lookup_primary_loc_tech_carriers_in   (loc_techs_conversion_plus) <U12 'X1::chp::gas' ...
lookup_primary_loc_tech_carriers_out  (loc_techs_conversion_plus) <U20 'X1::chp::electricity' ...
lookup_loc_techs_conversion_plus      (carrier_tiers, loc_techs_conversion_plus) object 'X1::chp::heat' ...
lookup_loc_techs_export               (loc_techs_export) <U20 'X3::pv::electricity' ...
lookup_loc_techs_area                 (locs) <U6 '' 'X2::pv' 'X1::pv' ...
timestep_resolution                   (timesteps) float64 1.0 1.0 1.0 ...
timestep_weights                      (timesteps) float64 1.0 1.0 1.0 ...
max_demand_timesteps                  (carriers) datetime64[ns] 2005-07-01 ...
Attributes:
model.calliope_version:            0.6.3
model.name:                        Urban-scale example model with MILP
model.subset_time:                 ['2005-07-01', '2005-07-02']
model.timeseries_dateformat:       %Y-%m-%d %H:%M:%S
run.backend:                       pyomo
run.bigM:                          1000000.0
run.cyclic_storage:                True
run.ensure_feasibility:            True
run.mode:                          plan
run.objective:                     minmax_cost_optimization
run.objective_options.cost_class:  monetary
run.objective_options.sense:       minimize
run.operation.use_cap_results:     False
run.solver:                        glpk
run.zero_threshold:                1e-10
calliope_version:                  0.6.3
applied_overrides:                 milp
scenario:                          milp
defaults:                          available_area: null\ncarrier_ratios: ...
allow_operate_mode:                1
In :
# Individual data variables can be accessed easily, to_pandas() reformats the data to look nicer
# Here we look at one of the MILP overrides that we have added, the fixed purchase cost
model.inputs.cost_purchase.to_pandas().dropna(axis=1)

Out:
loc_techs_investment_cost X2::boiler X3::boiler X1::chp
costs
monetary 2000.0 2000.0 40000.0
In :
# Solve the model. Results are loaded into model.results.
# By including logging (see package importing), we can see the timing of parts of the run, as well as the solver's log
model.run()

[2018-10-04 16:31:03] INFO: Backend: starting model run
[2018-10-04 16:31:04] INFO: Backend: model generated. Time since start: 0:00:04.579031
[2018-10-04 16:31:04] INFO: Backend: sending model to solver
[2018-10-04 16:31:06] INFO: Backend: solver finished running. Time since start: 0:00:06.046883
[2018-10-04 16:31:06] INFO: Backend: loaded results
[2018-10-04 16:31:06] INFO: Backend: generated solution array. Time since start: 0:00:06.134792
[2018-10-04 16:31:06] INFO: Postprocessing: started
[2018-10-04 16:31:06] INFO: Postprocessing: All values < 1e-10 set to 0 in unmet_demand
[2018-10-04 16:31:06] INFO: Postprocessing: Model was feasible, deleting unmet_demand variable
[2018-10-04 16:31:06] INFO: Postprocessing: ended. Time since start: 0:00:06.585246

In :
# Model results are held in the same structure as model inputs.
# The results consist of the optimal values for all decision variables, including capacities and carrier flow
# There are also results, like system capacity factor and levelised costs, which are calculated in postprocessing
# before being added to the results Dataset

model.results

Out:
<xarray.Dataset>
Dimensions:                     (carriers: 3, costs: 1, loc_tech_carriers_con: 19, loc_tech_carriers_export: 4, loc_tech_carriers_prod: 21, loc_techs: 26, loc_techs_area: 3, loc_techs_cost: 20, loc_techs_investment_cost: 20, loc_techs_milp: 1, loc_techs_om_cost: 9, loc_techs_purchase: 2, loc_techs_supply_plus: 3, techs: 9, timesteps: 48)
Coordinates:
* costs                       (costs) object 'monetary'
* loc_techs_investment_cost   (loc_techs_investment_cost) object 'X1::supply_grid_power' ...
* loc_techs_milp              (loc_techs_milp) object 'X1::chp'
* loc_tech_carriers_con       (loc_tech_carriers_con) object 'X1::power_lines:X3::electricity' ...
* loc_techs_supply_plus       (loc_techs_supply_plus) object 'X3::pv' ...
* loc_tech_carriers_export    (loc_tech_carriers_export) object 'X1::chp::electricity' ...
* carriers                    (carriers) object 'gas' 'heat' 'electricity'
* techs                       (techs) object 'heat_pipes' 'power_lines' ...
* loc_techs_purchase          (loc_techs_purchase) object 'X3::boiler' ...
* loc_techs_area              (loc_techs_area) object 'X3::pv' 'X2::pv' ...
* timesteps                   (timesteps) datetime64[ns] 2005-07-01 ...
* loc_techs_om_cost           (loc_techs_om_cost) object 'X1::supply_grid_power' ...
* loc_tech_carriers_prod      (loc_tech_carriers_prod) object 'X2::pv::electricity' ...
* loc_techs_cost              (loc_techs_cost) object 'X1::supply_grid_power' ...
* loc_techs                   (loc_techs) object 'X2::demand_heat' ...
Data variables:
energy_cap                  (loc_techs) float64 365.1 195.3 740.7 468.4 ...
carrier_prod                (loc_tech_carriers_prod, timesteps) float64 0.0 ...
carrier_con                 (loc_tech_carriers_con, timesteps) float64 -19.15 ...
cost                        (costs, loc_techs_cost) float64 0.0 18.69 ...
resource_area               (loc_techs_area) float64 350.0 0.0 0.0
resource_con                (loc_techs_supply_plus, timesteps) float64 0.0 ...
resource_cap                (loc_techs_supply_plus) float64 38.95 0.0 0.0
carrier_export              (loc_tech_carriers_export, timesteps) float64 0.0 ...
cost_var                    (costs, loc_techs_om_cost, timesteps) float64 0.0 ...
cost_investment             (costs, loc_techs_investment_cost) float64 0.0 ...
purchased                   (loc_techs_purchase) float64 0.0 1.0
units                       (loc_techs_milp) float64 1.0
operating_units             (loc_techs_milp, timesteps) float64 1.0 1.0 ...
capacity_factor             (loc_tech_carriers_prod, timesteps) float64 0.0 ...
systemwide_capacity_factor  (techs, carriers) float64 nan nan nan nan ...
systemwide_levelised_cost   (carriers, techs, costs) float64 nan nan nan ...
total_levelised_cost        (carriers, costs) float64 0.03351 0.02532 ...
Attributes:
model.calliope_version:            0.6.3
model.name:                        Urban-scale example model with MILP
model.subset_time:                 ['2005-07-01', '2005-07-02']
model.timeseries_dateformat:       %Y-%m-%d %H:%M:%S
run.backend:                       pyomo
run.bigM:                          1000000.0
run.cyclic_storage:                True
run.ensure_feasibility:            True
run.mode:                          plan
run.objective:                     minmax_cost_optimization
run.objective_options.cost_class:  monetary
run.objective_options.sense:       minimize
run.operation.use_cap_results:     False
run.solver:                        glpk
run.zero_threshold:                1e-10
calliope_version:                  0.6.3
applied_overrides:                 milp
scenario:                          milp
defaults:                          available_area: null\ncarrier_ratios: ...
allow_operate_mode:                1
termination_condition:             optimal
solution_time:                     2.880736
time_finished:                     2018-10-04 16:31:06
In :
# We can sum operating units of CHP over all locations and turn the result into a pandas DataFrame
df_units = model.get_formatted_array('operating_units').sum('locs').to_pandas().T

#The information about the dataframe tells us about the amount of data it holds in the index and in each column
df_units.info()

<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 48 entries, 2005-07-01 00:00:00 to 2005-07-02 23:00:00
Data columns (total 1 columns):
chp    48 non-null float64
dtypes: float64(1)
memory usage: 768.0 bytes

In :
# Using .head() to see the first few rows of operating units


Out:
techs chp
timesteps
2005-07-01 00:00:00 1.0
2005-07-01 01:00:00 1.0
2005-07-01 02:00:00 1.0
2005-07-01 03:00:00 1.0
2005-07-01 04:00:00 1.0
In :
# We can plot this by using the timeseries plotting functionality.
# The top-left dropdown gives us the chance to scroll through other timeseries data too.

model.plot.timeseries()

In :
# plot.capacities gives a graphical view of the non-timeseries variables, both input and output

# Note, because we fix unit size, CHP now has a maximum capacity of 300kW,
# compared to 260kW in the non-MILP case

model.plot.capacity()