Tutorial 3: Mixed Integer Linear Programming

This example is based on the urban scale example model, but with an override. In the model’s scenarios.yaml file overrides are defined which trigger binary and integer decision variables, creating a MILP model, rather than a conventional LP model.

Units

The capacity of a technology is usually a continuous decision variable, which can be within the range of 0 and energy_cap_max (the maximum capacity of a technology). In this model, we introduce a unit limit on the CHP instead:

    chp:
        constraints:
            units_max: 4
            energy_cap_per_unit: 300
            energy_cap_min_use: 0.2
        costs:
            monetary:
                energy_cap: 700
                purchase: 40000

A unit maximum allows a discrete, integer number of CHP to be purchased, each having a capacity of energy_cap_per_unit. Any of energy_cap_max, energy_cap_min, or energy_cap_equals are now ignored, in favour of units_max, units_min, or units_equals. A useful feature unlocked by introducing this is the ability to set a minimum operating capacity which is only enforced when the technology is operating. In the LP model, energy_cap_min_use would force the technology to operate at least at that proportion of its maximum capacity at each time step. In this model, the newly introduced energy_cap_min_use of 0.2 will ensure that the output of the CHP is 20% of its maximum capacity in any time step in which it has a non-zero output.

Purchase cost

The boiler does not have a unit limit, it still utilises the continuous variable for its capacity. However, we have introduced a purchase cost:

    boiler:
        costs:
            monetary:
                energy_cap: 35
                purchase: 2000

By introducing this, the boiler now has a binary decision variable associated with it, which is 1 if the boiler has a non-zero energy_cap (i.e. the optimisation results in investment in a boiler) and 0 if the capacity is 0. The purchase cost is applied to the binary result, providing a fixed cost on purchase of the technology, irrespective of the technology size. In physical terms, this may be associated with the cost of pipework, land purchase, etc. The purchase cost is also imposed on the CHP, which is applied to the number of integer CHP units in which the solver chooses to invest.

MILP functionality can be easily applied, but convergence is slower as a result of integer/binary variables. It is recommended to use a commercial solver (e.g. Gurobi, CPLEX) if you wish to utilise these variables outside this example model.

Running the model

We now take you through running the model in a Jupyter notebook, which is included fully below. To download and run the notebook yourself, you can find it here. You will need to have Calliope installed.

Notebook

Calliope Urban Scale MILP Example Model

For more details on analysing input/output data, see the full urban scale example model

In [1]:
import calliope

# We increase logging verbosity
calliope.set_log_level('INFO')
In [2]:
model = calliope.examples.milp()

# Note, we see the overrides that we have applied printed here, thanks to inreasing logging verbosity
[2018-10-04 16:31:00] INFO: Model: initialising
[2018-10-04 16:31:00] INFO: Applying overrides `milp` without a specific scenario name.
[2018-10-04 16:31:00] INFO: Override applied to model.name: Urban-scale example model -> Urban-scale example model with MILP
`techs.boiler.costs.monetary.energy_cap`:35 applied from override as new configuration
`techs.boiler.costs.monetary.purchase`:2000 applied from override as new configuration
`techs.chp.constraints.energy_cap_min_use`:0.2 applied from override as new configuration
`techs.chp.constraints.energy_cap_per_unit`:300 applied from override as new configuration
`techs.chp.constraints.units_max`:4 applied from override as new configuration
Override applied to techs.chp.costs.monetary.energy_cap: 750 -> 700
`techs.chp.costs.monetary.purchase`:40000 applied from override as new configuration
[2018-10-04 16:31:00] INFO: Model: preprocessing stage 1 (model_run)
[2018-10-04 16:31:02] INFO: Model: preprocessing stage 2 (model_data)
[2018-10-04 16:31:03] INFO: Model: preprocessing complete. Time since start: 0:00:02.902346
In [3]:
# Model inputs can be viewed at `model.inputs`. 
# Variables are indexed over any combination of `techs`, `locs`, `carriers`, `costs` and `timesteps`, 
# although `techs`, `locs`, and `carriers` are often concatenated. 
# e.g. `chp`, `X1`, `heat` -> `X1::chp::heat` 
model.inputs
Out[3]:
<xarray.Dataset>
Dimensions:                               (carrier_tiers: 3, carriers: 3, coordinates: 2, costs: 1, loc_carriers: 10, loc_tech_carriers_conversion_plus: 3, loc_techs: 26, loc_techs_area: 3, loc_techs_conversion: 2, loc_techs_conversion_plus: 1, loc_techs_export: 4, loc_techs_finite_resource: 9, loc_techs_investment_cost: 20, loc_techs_milp: 1, loc_techs_non_conversion: 23, loc_techs_om_cost: 9, loc_techs_supply_plus: 3, loc_techs_transmission: 10, locs: 4, techs: 9, timesteps: 48)
Coordinates:
  * carrier_tiers                         (carrier_tiers) <U5 'out_2' 'out' 'in'
  * loc_techs_export                      (loc_techs_export) object 'X3::pv' ...
  * loc_techs_finite_resource             (loc_techs_finite_resource) object 'X2::demand_heat' ...
  * costs                                 (costs) object 'monetary'
  * loc_techs_investment_cost             (loc_techs_investment_cost) object 'X1::supply_grid_power' ...
  * loc_techs_milp                        (loc_techs_milp) object 'X1::chp'
  * loc_techs_supply_plus                 (loc_techs_supply_plus) object 'X3::pv' ...
  * carriers                              (carriers) object 'gas' 'heat' ...
  * coordinates                           (coordinates) object 'y' 'x'
  * loc_carriers                          (loc_carriers) object 'X2::electricity' ...
  * loc_techs_non_conversion              (loc_techs_non_conversion) object 'X2::demand_heat' ...
  * locs                                  (locs) object 'N1' 'X2' 'X1' 'X3'
  * techs                                 (techs) object 'heat_pipes' ...
  * loc_tech_carriers_conversion_plus     (loc_tech_carriers_conversion_plus) object 'X1::chp::gas' ...
  * loc_techs_conversion_plus             (loc_techs_conversion_plus) <U7 'X1::chp' ...
  * loc_techs_area                        (loc_techs_area) object 'X3::pv' ...
  * timesteps                             (timesteps) datetime64[ns] 2005-07-01 ...
  * loc_techs_om_cost                     (loc_techs_om_cost) object 'X1::supply_grid_power' ...
  * loc_techs_conversion                  (loc_techs_conversion) <U10 'X3::boiler' ...
  * loc_techs                             (loc_techs) object 'X2::demand_heat' ...
  * loc_techs_transmission                (loc_techs_transmission) object 'X3::heat_pipes:N1' ...
Data variables:
    energy_cap_min_use                    (loc_techs) float64 nan nan nan ...
    resource_unit                         (loc_techs_finite_resource) <U15 'energy' ...
    resource_area_per_energy_cap          (loc_techs_area) int64 7 7 7
    energy_cap_max                        (loc_techs) float64 nan 2e+03 ...
    energy_eff                            (loc_techs) float64 nan nan nan ...
    parasitic_eff                         (loc_techs_supply_plus) float64 0.85 ...
    lifetime                              (loc_techs) float64 nan 25.0 25.0 ...
    resource_eff                          (loc_techs_finite_resource) float64 nan ...
    energy_prod                           (loc_techs) float64 nan 1.0 1.0 ...
    units_max                             (loc_techs_milp) int64 4
    energy_cap_per_unit                   (loc_techs) float64 nan nan nan ...
    energy_con                            (loc_techs) float64 1.0 nan nan ...
    resource_area_max                     (loc_techs_area) int64 1500 1500 1500
    force_resource                        (loc_techs_finite_resource) bool True ...
    export_carrier                        (loc_techs_export) <U11 'electricity' ...
    resource                              (loc_techs_finite_resource, timesteps) float64 -64.73 ...
    reserve_margin                        (carriers) float64 nan nan nan
    cost_export                           (costs, loc_techs_om_cost, timesteps) float64 nan ...
    cost_om_con                           (costs, loc_techs_om_cost) float64 0.1 ...
    cost_energy_cap                       (costs, loc_techs_investment_cost) float64 100.0 ...
    cost_om_prod                          (costs, loc_techs_om_cost) float64 nan ...
    cost_om_annual                        (costs, loc_techs_om_cost) float64 nan ...
    cost_depreciation_rate                (costs, loc_techs_investment_cost) float64 0.1102 ...
    cost_purchase                         (costs, loc_techs_investment_cost) float64 nan ...
    distance                              (loc_techs_transmission) float64 4.0 ...
    lookup_remotes                        (loc_techs_transmission) <U18 'N1::heat_pipes:X3' ...
    available_area                        (locs) float64 nan 1.3e+03 500.0 900.0
    loc_coordinates                       (coordinates, locs) int64 7 7 7 3 ...
    colors                                (techs) <U7 '#823739' '#6783E3' ...
    inheritance                           (techs) <U29 'transmission' ...
    names                                 (techs) <U29 'District heat distribution' ...
    carrier_ratios                        (carrier_tiers, loc_tech_carriers_conversion_plus) float64 1.0 ...
    carrier_ratios_min                    (carrier_tiers, loc_techs_conversion_plus) float64 0.8 ...
    lookup_loc_carriers                   (loc_carriers) <U175 'X2::demand_electricity::electricity,X2::pv::electricity,X2::power_lines:X1::electricity' ...
    lookup_loc_techs                      (loc_techs_non_conversion) <U35 'X2::demand_heat::heat' ...
    lookup_loc_techs_conversion           (carrier_tiers, loc_techs_conversion) object None ...
    lookup_primary_loc_tech_carriers_in   (loc_techs_conversion_plus) <U12 'X1::chp::gas' ...
    lookup_primary_loc_tech_carriers_out  (loc_techs_conversion_plus) <U20 'X1::chp::electricity' ...
    lookup_loc_techs_conversion_plus      (carrier_tiers, loc_techs_conversion_plus) object 'X1::chp::heat' ...
    lookup_loc_techs_export               (loc_techs_export) <U20 'X3::pv::electricity' ...
    lookup_loc_techs_area                 (locs) <U6 '' 'X2::pv' 'X1::pv' ...
    timestep_resolution                   (timesteps) float64 1.0 1.0 1.0 ...
    timestep_weights                      (timesteps) float64 1.0 1.0 1.0 ...
    max_demand_timesteps                  (carriers) datetime64[ns] 2005-07-01 ...
Attributes:
    model.calliope_version:            0.6.3
    model.name:                        Urban-scale example model with MILP
    model.subset_time:                 ['2005-07-01', '2005-07-02']
    model.timeseries_dateformat:       %Y-%m-%d %H:%M:%S
    run.backend:                       pyomo
    run.bigM:                          1000000.0
    run.cyclic_storage:                True
    run.ensure_feasibility:            True
    run.mode:                          plan
    run.objective:                     minmax_cost_optimization
    run.objective_options.cost_class:  monetary
    run.objective_options.sense:       minimize
    run.operation.use_cap_results:     False
    run.solver:                        glpk
    run.zero_threshold:                1e-10
    calliope_version:                  0.6.3
    applied_overrides:                 milp
    scenario:                          milp
    defaults:                          available_area: null\ncarrier_ratios: ...
    allow_operate_mode:                1
In [4]:
# Individual data variables can be accessed easily, `to_pandas()` reformats the data to look nicer
# Here we look at one of the MILP overrides that we have added, the fixed `purchase` cost
model.inputs.cost_purchase.to_pandas().dropna(axis=1)
Out[4]:
loc_techs_investment_cost X2::boiler X3::boiler X1::chp
costs
monetary 2000.0 2000.0 40000.0
In [5]:
# Solve the model. Results are loaded into `model.results`. 
# By including logging (see package importing), we can see the timing of parts of the run, as well as the solver's log
model.run()
[2018-10-04 16:31:03] INFO: Backend: starting model run
[2018-10-04 16:31:04] INFO: Backend: model generated. Time since start: 0:00:04.579031
[2018-10-04 16:31:04] INFO: Backend: sending model to solver
[2018-10-04 16:31:06] INFO: Backend: solver finished running. Time since start: 0:00:06.046883
[2018-10-04 16:31:06] INFO: Backend: loaded results
[2018-10-04 16:31:06] INFO: Backend: generated solution array. Time since start: 0:00:06.134792
[2018-10-04 16:31:06] INFO: Postprocessing: started
[2018-10-04 16:31:06] INFO: Postprocessing: All values < 1e-10 set to 0 in unmet_demand
[2018-10-04 16:31:06] INFO: Postprocessing: Model was feasible, deleting unmet_demand variable
[2018-10-04 16:31:06] INFO: Postprocessing: ended. Time since start: 0:00:06.585246
In [6]:
# Model results are held in the same structure as model inputs. 
# The results consist of the optimal values for all decision variables, including capacities and carrier flow
# There are also results, like system capacity factor and levelised costs, which are calculated in postprocessing
# before being added to the results Dataset

model.results
Out[6]:
<xarray.Dataset>
Dimensions:                     (carriers: 3, costs: 1, loc_tech_carriers_con: 19, loc_tech_carriers_export: 4, loc_tech_carriers_prod: 21, loc_techs: 26, loc_techs_area: 3, loc_techs_cost: 20, loc_techs_investment_cost: 20, loc_techs_milp: 1, loc_techs_om_cost: 9, loc_techs_purchase: 2, loc_techs_supply_plus: 3, techs: 9, timesteps: 48)
Coordinates:
  * costs                       (costs) object 'monetary'
  * loc_techs_investment_cost   (loc_techs_investment_cost) object 'X1::supply_grid_power' ...
  * loc_techs_milp              (loc_techs_milp) object 'X1::chp'
  * loc_tech_carriers_con       (loc_tech_carriers_con) object 'X1::power_lines:X3::electricity' ...
  * loc_techs_supply_plus       (loc_techs_supply_plus) object 'X3::pv' ...
  * loc_tech_carriers_export    (loc_tech_carriers_export) object 'X1::chp::electricity' ...
  * carriers                    (carriers) object 'gas' 'heat' 'electricity'
  * techs                       (techs) object 'heat_pipes' 'power_lines' ...
  * loc_techs_purchase          (loc_techs_purchase) object 'X3::boiler' ...
  * loc_techs_area              (loc_techs_area) object 'X3::pv' 'X2::pv' ...
  * timesteps                   (timesteps) datetime64[ns] 2005-07-01 ...
  * loc_techs_om_cost           (loc_techs_om_cost) object 'X1::supply_grid_power' ...
  * loc_tech_carriers_prod      (loc_tech_carriers_prod) object 'X2::pv::electricity' ...
  * loc_techs_cost              (loc_techs_cost) object 'X1::supply_grid_power' ...
  * loc_techs                   (loc_techs) object 'X2::demand_heat' ...
Data variables:
    energy_cap                  (loc_techs) float64 365.1 195.3 740.7 468.4 ...
    carrier_prod                (loc_tech_carriers_prod, timesteps) float64 0.0 ...
    carrier_con                 (loc_tech_carriers_con, timesteps) float64 -19.15 ...
    cost                        (costs, loc_techs_cost) float64 0.0 18.69 ...
    resource_area               (loc_techs_area) float64 350.0 0.0 0.0
    resource_con                (loc_techs_supply_plus, timesteps) float64 0.0 ...
    resource_cap                (loc_techs_supply_plus) float64 38.95 0.0 0.0
    carrier_export              (loc_tech_carriers_export, timesteps) float64 0.0 ...
    cost_var                    (costs, loc_techs_om_cost, timesteps) float64 0.0 ...
    cost_investment             (costs, loc_techs_investment_cost) float64 0.0 ...
    purchased                   (loc_techs_purchase) float64 0.0 1.0
    units                       (loc_techs_milp) float64 1.0
    operating_units             (loc_techs_milp, timesteps) float64 1.0 1.0 ...
    capacity_factor             (loc_tech_carriers_prod, timesteps) float64 0.0 ...
    systemwide_capacity_factor  (techs, carriers) float64 nan nan nan nan ...
    systemwide_levelised_cost   (carriers, techs, costs) float64 nan nan nan ...
    total_levelised_cost        (carriers, costs) float64 0.03351 0.02532 ...
Attributes:
    model.calliope_version:            0.6.3
    model.name:                        Urban-scale example model with MILP
    model.subset_time:                 ['2005-07-01', '2005-07-02']
    model.timeseries_dateformat:       %Y-%m-%d %H:%M:%S
    run.backend:                       pyomo
    run.bigM:                          1000000.0
    run.cyclic_storage:                True
    run.ensure_feasibility:            True
    run.mode:                          plan
    run.objective:                     minmax_cost_optimization
    run.objective_options.cost_class:  monetary
    run.objective_options.sense:       minimize
    run.operation.use_cap_results:     False
    run.solver:                        glpk
    run.zero_threshold:                1e-10
    calliope_version:                  0.6.3
    applied_overrides:                 milp
    scenario:                          milp
    defaults:                          available_area: null\ncarrier_ratios: ...
    allow_operate_mode:                1
    termination_condition:             optimal
    solution_time:                     2.880736
    time_finished:                     2018-10-04 16:31:06
In [7]:
# We can sum operating units of CHP over all locations and turn the result into a pandas DataFrame
df_units = model.get_formatted_array('operating_units').sum('locs').to_pandas().T

#The information about the dataframe tells us about the amount of data it holds in the index and in each column
df_units.info()
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 48 entries, 2005-07-01 00:00:00 to 2005-07-02 23:00:00
Data columns (total 1 columns):
chp    48 non-null float64
dtypes: float64(1)
memory usage: 768.0 bytes
In [8]:
# Using .head() to see the first few rows of operating units

df_units.head()
Out[8]:
techs chp
timesteps
2005-07-01 00:00:00 1.0
2005-07-01 01:00:00 1.0
2005-07-01 02:00:00 1.0
2005-07-01 03:00:00 1.0
2005-07-01 04:00:00 1.0
In [9]:
# We can plot this by using the timeseries plotting functionality.
# The top-left dropdown gives us the chance to scroll through other timeseries data too.

model.plot.timeseries()
In [10]:
# plot.capacities gives a graphical view of the non-timeseries variables, both input and output

# Note, because we fix unit size, CHP now has a maximum capacity of 300kW, 
# compared to 260kW in the non-MILP case

model.plot.capacity()