Calliope logging examples¶

In this notebook, we will look at ways of capturing calliope logging outputs and printing them to the console or to file.

In [1]:

Copied!

import logging
from pathlib import Path

import calliope
import logging
from pathlib import Path

import calliope

Using internal Calliope functionality¶

The calliope.set_log_verbosity method allows you to quickly set the logging level across all calliope loggers. It doesn't require you to know anything about the logging package, just the available logging levels.

This is the default log verbosity that is set on importing calliope. It will print the WARNING and INFO log levels to the console and it will print the solver output (which is otherwise at the DEBUG log level)

In [2]:

Copied!





calliope.set_log_verbosity("info")
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])
m.build()
m.solve()
calliope.set_log_verbosity("info")
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])
m.build()
m.solve()

[2024-01-27 14:22:56] INFO     Model: initialising

[2024-01-27 14:22:56] INFO     Model: preprocessing stage 1 (model_run)

[2024-01-27 14:22:59] INFO     Model: preprocessing stage 2 (model_data)

[2024-01-27 14:22:59] INFO     Model: preprocessing complete

[2024-01-27 14:22:59] INFO     Optimisation Model | parameters | Generated.

[2024-01-27 14:23:00] INFO     Optimisation Model | variables | Generated.

[2024-01-27 14:23:01] INFO     Optimisation Model | global_expressions | Generated.

[2024-01-27 14:23:03] INFO     Optimisation Model | constraints | Generated.

[2024-01-27 14:23:03] INFO     Optimisation Model | objectives | Generated.

[2024-01-27 14:23:03] INFO     Optimisation model | starting model in plan mode.

[2024-01-27 14:23:03] DEBUG    Welcome to the CBC MILP Solver 
Version: 2.10.10 
Build Date: Apr 19 2023 

command line - /home/docs/checkouts/readthedocs.org/user_builds/calliope/conda/v0.7.0.dev2/bin/cbc -printingOptions all -import /tmp/tmpyiimmz_0.pyomo.lp -stat=1 -solve -solu /tmp/tmpyiimmz_0.pyomo.soln (default strategy 1)
Option for printingOptions changed from normal to all
Presolve 700 (-583) rows, 861 (-177) columns and 2384 (-1088) elements
Statistics for presolved model


Problem has 700 rows, 861 columns (399 with objective) and 2384 elements
There are 240 singletons with objective 
Column breakdown:
732 of type 0.0->inf, 9 of type 0.0->up, 0 of type lo->inf, 
0 of type lo->up, 0 of type free, 0 of type fixed, 
120 of type -inf->0.0, 0 of type -inf->up, 0 of type 0.0->1.0 
Row breakdown:
224 of type E 0.0, 0 of type E 1.0, 0 of type E -1.0, 
48 of type E other, 0 of type G 0.0, 0 of type G 1.0, 
0 of type G other, 428 of type L 0.0, 0 of type L 1.0, 
0 of type L other, 0 of type Range 0.0->1.0, 0 of type Range other, 
0 of type Free 
Presolve 700 (-583) rows, 861 (-177) columns and 2384 (-1088) elements
Perturbing problem by 0.001% of 66.616365 - largest nonzero change 0.00016650948 ( 19.901606%) - largest zero change 0.00016401159
0  Obj 0 Primal inf 708537.92 (48)
89  Obj 64.743123 Primal inf 754880.64 (48)
178  Obj 5172.2946 Primal inf 1817673 (91)
267  Obj 24242.276 Primal inf 1092822.4 (156)
356  Obj 36907.542 Primal inf 167562.17 (121)
445  Obj 39158.214 Primal inf 1.4289021 (15)
460  Obj 39158.345
Optimal - objective value 38988.744
After Postsolve, objective 38988.744, infeasibilities - dual 0 (0), primal 0 (0)
Optimal objective 38988.7442 - 460 iterations time 0.022, Presolve 0.00
Total time (CPU seconds):       0.03   (Wallclock seconds):       0.03

[2024-01-27 14:23:03] INFO     Backend: solver finished running. Time since start of solving optimisation problem: 0:00:00.256690

[2024-01-27 14:23:03] INFO     Postprocessing: started

[2024-01-27 14:23:03] INFO     Postprocessing: All values < 1e-10 set to 0 in flow_cap

[2024-01-27 14:23:03] INFO     Postprocessing: ended. Time since start of solving optimisation problem: 0:00:00.429772

[2024-01-27 14:23:03] INFO     Model: loaded model_data

This will print WARNING and INFO log levels to the console but NOT the log solver output

In [3]:

Copied!





calliope.set_log_verbosity("info", include_solver_output=False)
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])
m.build()
m.solve()
calliope.set_log_verbosity("info", include_solver_output=False)
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])
m.build()
m.solve()

[2024-01-27 14:23:03] INFO     Model: initialising

[2024-01-27 14:23:03] INFO     Model: preprocessing stage 1 (model_run)

[2024-01-27 14:23:06] INFO     Model: preprocessing stage 2 (model_data)

[2024-01-27 14:23:06] INFO     Model: preprocessing complete

[2024-01-27 14:23:07] INFO     Optimisation Model | parameters | Generated.

[2024-01-27 14:23:07] INFO     Optimisation Model | variables | Generated.

[2024-01-27 14:23:08] INFO     Optimisation Model | global_expressions | Generated.

[2024-01-27 14:23:10] INFO     Optimisation Model | constraints | Generated.

[2024-01-27 14:23:10] INFO     Optimisation Model | objectives | Generated.

[2024-01-27 14:23:10] INFO     Optimisation model | starting model in plan mode.

[2024-01-27 14:23:11] INFO     Backend: solver finished running. Time since start of solving optimisation problem: 0:00:00.249769

[2024-01-27 14:23:11] INFO     Postprocessing: started

[2024-01-27 14:23:11] INFO     Postprocessing: All values < 1e-10 set to 0 in flow_cap

[2024-01-27 14:23:11] INFO     Postprocessing: ended. Time since start of solving optimisation problem: 0:00:00.418622

[2024-01-27 14:23:11] INFO     Model: loaded model_data

You can set the log verbosity to print all DEBUG level logs to the console

In [4]:

Copied!





calliope.set_log_verbosity("debug")
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])
m.build()
m.solve()
calliope.set_log_verbosity("debug")
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])
m.build()
m.solve()

[2024-01-27 14:23:11] INFO     Model: initialising

[2024-01-27 14:23:11] INFO     Model: preprocessing stage 1 (model_run)

[2024-01-27 14:23:11] DEBUG    (data_sources, time_varying_parameters) | Updating non-NaN values of parameter `sink_use_equals` to float type.

[2024-01-27 14:23:11] DEBUG    (data_sources, time_varying_parameters) | Updating non-NaN values of parameter `source_use_max` to float type.

[2024-01-27 14:23:11] DEBUG    (data_sources, time_varying_parameters) | Loaded arrays:
<xarray.Dataset>
Dimensions:          (timesteps: 8760, nodes: 5, techs: 2)
Coordinates:
  * timesteps        (timesteps) object '2005-01-01 00:00' ... '2005-12-31 23...
  * nodes            (nodes) object 'region1' 'region1_1' ... 'region2'
  * techs            (techs) object 'csp' 'demand_power'
Data variables:
    source_use_max   (timesteps, nodes, techs) float64 nan nan 0.0 ... nan nan
    sink_use_equals  (timesteps, nodes, techs) float64 nan ... 2.295e+03.

[2024-01-27 14:23:11] DEBUG    (data_sources, time_varying_parameters) | Updating `timesteps` dimension index values to datetime format.

[2024-01-27 14:23:14] DEBUG    Link distance matrix automatically computed from lat/lon coordinates.

[2024-01-27 14:23:14] INFO     Model: preprocessing stage 2 (model_data)

[2024-01-27 14:23:14] INFO     Model: preprocessing complete

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:available_area | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:area_use_min | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:area_use_per_flow_cap | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:area_use_per_flow_cap | Component not added; no data found in array.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:cap_method | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:cost_depreciation_rate | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:cost_export | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:cost_flow_cap_per_distance | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:cost_om_annual | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:cost_om_annual_investment_fraction | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:cost_purchase | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:cost_purchase_per_distance | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:cyclic_storage | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:export_max | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:flow_cap_min | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:flow_cap_min_systemwide | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:flow_cap_per_storage_cap_min | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:flow_cap_per_unit | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:flow_cap_per_unit | Component not added; no data found in array.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:flow_in_eff_per_distance | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:flow_out_eff_per_distance | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:flow_out_min_relative | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:force_async_flow | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:integer_dispatch | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:one_way | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:purchased_units_max | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:purchased_units_max_systemwide | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:purchased_units_min | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:purchased_units_min_systemwide | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:sink_unit | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:sink_use_max | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:sink_use_min | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:source_cap_equals_flow_cap | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:source_cap_max | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:source_cap_min | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:source_eff | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:source_use_equals | Component not defined; using default value.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:source_use_equals | Component not added; no data found in array.

[2024-01-27 14:23:14] DEBUG    Optimisation model | parameters:source_use_min | Component not defined; using default value.

[2024-01-27 14:23:15] DEBUG    Optimisation model | parameters:storage_cap_min | Component not defined; using default value.

[2024-01-27 14:23:15] DEBUG    Optimisation model | parameters:storage_cap_per_unit | Component not defined; using default value.

[2024-01-27 14:23:15] DEBUG    Optimisation model | parameters:storage_cap_per_unit | Component not added; no data found in array.

[2024-01-27 14:23:15] DEBUG    Optimisation model | parameters:storage_discharge_depth | Component not defined; using default value.

[2024-01-27 14:23:15] DEBUG    Optimisation model | parameters:storage_initial | Component not defined; using default value.

[2024-01-27 14:23:15] INFO     Optimisation Model | parameters | Generated.

[2024-01-27 14:23:15] DEBUG    Optimisation model | variables:flow_cap | Applying bound according to the flow_cap_max parameter values.

[2024-01-27 14:23:15] DEBUG    Optimisation model | variables:flow_cap | Applying bound according to the flow_cap_min parameter values.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:flow_cap | Built in 0.0180s

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:link_flow_cap | Built in 0.0147s

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:flow_out | Built in 0.0184s

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:flow_in | Built in 0.0185s

[2024-01-27 14:23:15] DEBUG    Math parsing | variables:flow_export | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:flow_export | Built in 0.0114s

[2024-01-27 14:23:15] DEBUG    Optimisation model | variables:area_use | Applying bound according to the area_use_max parameter values.

[2024-01-27 14:23:15] DEBUG    Optimisation model | variables:area_use | Applying bound according to the area_use_min parameter values.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:area_use | Built in 0.0258s

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:source_use | Built in 0.0161s

[2024-01-27 14:23:15] DEBUG    Optimisation model | variables:source_cap | Applying bound according to the source_cap_max parameter values.

[2024-01-27 14:23:15] DEBUG    Optimisation model | variables:source_cap | Applying bound according to the source_cap_min parameter values.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:source_cap | Built in 0.0197s

[2024-01-27 14:23:15] DEBUG    Optimisation model | variables:storage_cap | Applying bound according to the storage_cap_max parameter values.

[2024-01-27 14:23:15] DEBUG    Optimisation model | variables:storage_cap | Applying bound according to the storage_cap_min parameter values.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:storage_cap | Built in 0.0212s

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:storage | Built in 0.0195s

[2024-01-27 14:23:15] DEBUG    Math parsing | variables:purchased_units | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:purchased_units | Built in 0.0092s

[2024-01-27 14:23:15] DEBUG    Math parsing | variables:operating_units | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:operating_units | Built in 0.1000s

[2024-01-27 14:23:15] DEBUG    Math parsing | variables:available_flow_cap | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:available_flow_cap | Built in 0.0149s

[2024-01-27 14:23:15] DEBUG    Math parsing | variables:async_flow_switch | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:async_flow_switch | Built in 0.0093s

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:unmet_demand | Built in 0.0158s

[2024-01-27 14:23:15] DEBUG    Optimisation Model | variables:unused_supply | Built in 0.0170s

[2024-01-27 14:23:15] INFO     Optimisation Model | variables | Generated.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | global_expressions:flow_out_inc_eff | Built in 0.0922s

[2024-01-27 14:23:15] DEBUG    Optimisation Model | global_expressions:flow_in_inc_eff | Built in 0.0801s

[2024-01-27 14:23:15] DEBUG    Math parsing | global_expressions:cost_var:0-cost_flow_out:0-cost_export:0-cost_flow_in:0 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:15] DEBUG    Math parsing | global_expressions:cost_var:0-cost_flow_out:0-cost_export:0-cost_flow_in:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:15] DEBUG    Optimisation Model | global_expressions:cost_var | Built in 0.1943s

[2024-01-27 14:23:15] DEBUG    Optimisation Model | global_expressions:cost_investment_flow_cap | Built in 0.0989s

[2024-01-27 14:23:15] DEBUG    Optimisation Model | global_expressions:cost_investment_storage_cap | Built in 0.0427s

[2024-01-27 14:23:16] DEBUG    Optimisation Model | global_expressions:cost_investment_source_cap | Built in 0.0417s

[2024-01-27 14:23:16] DEBUG    Optimisation Model | global_expressions:cost_investment_area_use | Built in 0.0406s

[2024-01-27 14:23:16] DEBUG    Math parsing | global_expressions:cost_investment_purchase | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | global_expressions:cost_investment_purchase | Built in 0.0112s

[2024-01-27 14:23:16] DEBUG    Math parsing | global_expressions:cost_investment:0-annualisation_weight:0-depreciation_rate:0 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Math parsing | global_expressions:cost_investment:0-annualisation_weight:0-depreciation_rate:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | global_expressions:cost_investment | Built in 0.4012s

[2024-01-27 14:23:16] DEBUG    Math parsing | global_expressions:cost:0-cost_var_sum:0-cost_investment:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Math parsing | global_expressions:cost:0-cost_var_sum:1-cost_investment:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | global_expressions:cost | Built in 0.1118s

[2024-01-27 14:23:16] INFO     Optimisation Model | global_expressions | Generated.

[2024-01-27 14:23:16] DEBUG    Math parsing | constraints:flow_capacity_per_storage_capacity_min | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:flow_capacity_per_storage_capacity_min | Built in 0.0114s

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:flow_capacity_per_storage_capacity_max | Built in 0.0405s

[2024-01-27 14:23:16] DEBUG    Math parsing | constraints:source_capacity_equals_flow_capacity | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:source_capacity_equals_flow_capacity | Built in 0.0122s

[2024-01-27 14:23:16] DEBUG    Math parsing | constraints:force_zero_area_use | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:force_zero_area_use | Built in 0.0109s

[2024-01-27 14:23:16] DEBUG    Math parsing | constraints:area_use_per_flow_capacity | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:area_use_per_flow_capacity | Built in 0.0098s

[2024-01-27 14:23:16] DEBUG    Math parsing | constraints:area_use_capacity_per_loc | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:area_use_capacity_per_loc | Built in 0.0097s

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:flow_capacity_systemwide_max | Built in 0.0407s

[2024-01-27 14:23:16] DEBUG    Math parsing | constraints:flow_capacity_systemwide_min | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:flow_capacity_systemwide_min | Built in 0.0090s

[2024-01-27 14:23:16] DEBUG    Math parsing | constraints:balance_conversion | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:balance_conversion | Built in 0.0132s

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:flow_out_max | Built in 0.0543s

[2024-01-27 14:23:16] DEBUG    Math parsing | constraints:flow_out_min | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:flow_out_min | Built in 0.0106s

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:flow_in_max | Built in 0.1236s

[2024-01-27 14:23:16] DEBUG    Optimisation Model | constraints:source_max | Built in 0.0416s

[2024-01-27 14:23:17] DEBUG    Optimisation Model | constraints:storage_max | Built in 0.0398s

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:storage_discharge_depth_limit | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Optimisation Model | constraints:storage_discharge_depth_limit | Built in 0.0113s

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:system_balance:0-unmet_demand_and_unused_supply:0-flow_export:0 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:system_balance:0-unmet_demand_and_unused_supply:1-flow_export:0 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:system_balance:0-unmet_demand_and_unused_supply:1-flow_export:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Optimisation Model | constraints:system_balance | Built in 0.1277s

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:balance_demand:0-sink_scaler:0 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:balance_demand:0-sink_scaler:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:balance_demand:1-sink_scaler:0 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:balance_demand:1-sink_scaler:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:balance_demand:1-sink_scaler:2 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Optimisation Model | constraints:balance_demand | Built in 0.1200s

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:balance_demand_min_use | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Optimisation Model | constraints:balance_demand_min_use | Built in 0.0150s

[2024-01-27 14:23:17] DEBUG    Optimisation Model | constraints:balance_supply_no_storage | Built in 0.0467s

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:balance_supply_with_storage:0-storage_previous_step:0 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:balance_supply_with_storage:0-storage_previous_step:2 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Optimisation Model | constraints:balance_supply_with_storage | Built in 0.2941s

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:source_availability_supply:0-source_scaler:0 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:source_availability_supply:0-source_scaler:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:source_availability_supply:0-source_scaler:2 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:source_availability_supply:1-source_scaler:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:source_availability_supply:1-source_scaler:2 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Optimisation Model | constraints:source_availability_supply | Built in 0.1195s

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:balance_supply_min_use | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:17] DEBUG    Optimisation Model | constraints:balance_supply_min_use | Built in 0.0121s

[2024-01-27 14:23:17] DEBUG    Math parsing | constraints:balance_storage:0-storage_previous_step:0 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:balance_storage:0-storage_previous_step:2 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:balance_storage | Built in 0.2297s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:set_storage_initial | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:set_storage_initial | Built in 0.0121s

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:balance_transmission | Built in 0.0523s

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:symmetric_transmission | Built in 0.0424s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:export_balance | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:export_balance | Built in 0.0095s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:flow_export_max | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:flow_export_max | Built in 0.0099s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:unit_commitment_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:unit_commitment_milp | Built in 0.0097s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:available_flow_cap_binary | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:available_flow_cap_binary | Built in 0.0107s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:available_flow_cap_continuous | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:available_flow_cap_continuous | Built in 0.0090s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:available_flow_cap_max_binary_continuous_switch | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:available_flow_cap_max_binary_continuous_switch | Built in 0.0086s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:flow_out_max_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:flow_out_max_milp | Built in 0.0123s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:flow_in_max_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:flow_in_max_milp | Built in 0.0124s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:flow_out_min_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:flow_out_min_milp | Built in 0.0137s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:storage_capacity_units_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:storage_capacity_units_milp | Built in 0.0109s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:flow_capacity_units_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:flow_capacity_units_milp | Built in 0.0077s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:flow_capacity_max_purchase_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:flow_capacity_max_purchase_milp | Built in 0.0108s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:flow_capacity_min_purchase_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:flow_capacity_min_purchase_milp | Built in 0.0105s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:storage_capacity_max_purchase_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:storage_capacity_max_purchase_milp | Built in 0.0104s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:storage_capacity_min_purchase_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:storage_capacity_min_purchase_milp | Built in 0.0087s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:unit_capacity_max_systemwide_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:unit_capacity_max_systemwide_milp | Built in 0.0093s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:unit_capacity_min_systemwide_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:unit_capacity_min_systemwide_milp | Built in 0.0101s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:async_flow_in_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:async_flow_in_milp | Built in 0.0879s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:async_flow_out_milp | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:async_flow_out_milp | Built in 0.0100s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:ramping_up:0-flow:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:ramping_up:0-flow:2 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:ramping_up | Built in 0.1190s

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:ramping_down:0-flow:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Math parsing | constraints:ramping_down:0-flow:2 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | constraints:ramping_down | Built in 0.1221s

[2024-01-27 14:23:18] INFO     Optimisation Model | constraints | Generated.

[2024-01-27 14:23:18] DEBUG    Optimisation model | objectives:min_cost_optimisation | Objective activated.

[2024-01-27 14:23:18] DEBUG    Math parsing | objectives:min_cost_optimisation:0-unmet_demand:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Math parsing | objectives:min_cost_optimisation:1-unmet_demand:0 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Math parsing | objectives:min_cost_optimisation:1-unmet_demand:1 | Component not added; 'where' does not apply anywhere.

[2024-01-27 14:23:18] DEBUG    Optimisation Model | objectives:min_cost_optimisation | Built in 0.1156s

[2024-01-27 14:23:18] INFO     Optimisation Model | objectives | Generated.

[2024-01-27 14:23:18] INFO     Optimisation model | starting model in plan mode.

[2024-01-27 14:23:18] DEBUG    Welcome to the CBC MILP Solver 
Version: 2.10.10 
Build Date: Apr 19 2023 

command line - /home/docs/checkouts/readthedocs.org/user_builds/calliope/conda/v0.7.0.dev2/bin/cbc -printingOptions all -import /tmp/tmp7de74lgq.pyomo.lp -stat=1 -solve -solu /tmp/tmp7de74lgq.pyomo.soln (default strategy 1)
Option for printingOptions changed from normal to all
Presolve 700 (-583) rows, 861 (-177) columns and 2384 (-1088) elements
Statistics for presolved model


Problem has 700 rows, 861 columns (399 with objective) and 2384 elements
There are 240 singletons with objective 
Column breakdown:
732 of type 0.0->inf, 9 of type 0.0->up, 0 of type lo->inf, 
0 of type lo->up, 0 of type free, 0 of type fixed, 
120 of type -inf->0.0, 0 of type -inf->up, 0 of type 0.0->1.0 
Row breakdown:
224 of type E 0.0, 0 of type E 1.0, 0 of type E -1.0, 
48 of type E other, 0 of type G 0.0, 0 of type G 1.0, 
0 of type G other, 428 of type L 0.0, 0 of type L 1.0, 
0 of type L other, 0 of type Range 0.0->1.0, 0 of type Range other, 
0 of type Free 
Presolve 700 (-583) rows, 861 (-177) columns and 2384 (-1088) elements
Perturbing problem by 0.001% of 66.616365 - largest nonzero change 0.00016650948 ( 19.901606%) - largest zero change 0.00016401159
0  Obj 0 Primal inf 708537.92 (48)
89  Obj 64.743123 Primal inf 754880.64 (48)
178  Obj 5172.2946 Primal inf 1817673 (91)
267  Obj 24242.276 Primal inf 1092822.4 (156)
356  Obj 36907.542 Primal inf 167562.17 (121)
445  Obj 39158.214 Primal inf 1.4289021 (15)
460  Obj 39158.345
Optimal - objective value 38988.744
After Postsolve, objective 38988.744, infeasibilities - dual 0 (0), primal 0 (0)
Optimal objective 38988.7442 - 460 iterations time 0.032, Presolve 0.01
Total time (CPU seconds):       0.06   (Wallclock seconds):       0.03

[2024-01-27 14:23:19] INFO     Backend: solver finished running. Time since start of solving optimisation problem: 0:00:00.250475

[2024-01-27 14:23:19] INFO     Postprocessing: started

[2024-01-27 14:23:19] INFO     Postprocessing: All values < 1e-10 set to 0 in flow_cap

[2024-01-27 14:23:19] INFO     Postprocessing: ended. Time since start of solving optimisation problem: 0:00:00.422461

[2024-01-27 14:23:19] INFO     Model: loaded model_data

Adding your own console logging handler¶

If the calliope.set_log_verbosity method isn't providing you with enough flexibility then you can add your own logging handlers

In [5]:

Copied!





# Grab the calliope logger, which will also automatically get the child loggers (e.g., `calliope.model`).
logger = logging.getLogger("calliope")

# Remove existing handlers (i.e., those introduced by `calliope.set_log_verbosity` above)
logger.handlers.clear()

# You can define your own custom formatter.
# See https://docs.python.org/3/library/logging.html#logrecord-attributes for available attributes.
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

# Add a ConsoleHandler (this is what `calliope.set_log_verbosity` is doing under the hood)
console_handler = logging.StreamHandler()
console_handler.setLevel(
    logging.INFO
)  # In this example, we only want to see warnings in the console
console_handler.setFormatter(formatter)
logger.addHandler(console_handler)

# You can also use logging in your scripts to add more information
logger.info("Loading the national-scale example model")
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])

logger.info("Building the national-scale example model optimisation problem")
m.build()

logger.info("Solving the national-scale example model optimisation problem")
m.solve()
# Grab the calliope logger, which will also automatically get the child loggers (e.g., `calliope.model`).
logger = logging.getLogger("calliope")

# Remove existing handlers (i.e., those introduced by `calliope.set_log_verbosity` above)
logger.handlers.clear()

# You can define your own custom formatter.
# See https://docs.python.org/3/library/logging.html#logrecord-attributes for available attributes.
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

# Add a ConsoleHandler (this is what `calliope.set_log_verbosity` is doing under the hood)
console_handler = logging.StreamHandler()
console_handler.setLevel(
    logging.INFO
)  # In this example, we only want to see warnings in the console
console_handler.setFormatter(formatter)
logger.addHandler(console_handler)

# You can also use logging in your scripts to add more information
logger.info("Loading the national-scale example model")
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])

logger.info("Building the national-scale example model optimisation problem")
m.build()

logger.info("Solving the national-scale example model optimisation problem")
m.solve()

2024-01-27 14:23:19,215 - calliope - INFO - Loading the national-scale example model

2024-01-27 14:23:19,216 - calliope.model - INFO - Model: initialising

2024-01-27 14:23:19,334 - calliope.model - INFO - Model: preprocessing stage 1 (model_run)

2024-01-27 14:23:22,299 - calliope.model - INFO - Model: preprocessing stage 2 (model_data)

2024-01-27 14:23:22,316 - calliope.model - INFO - Model: preprocessing complete

2024-01-27 14:23:22,317 - calliope - INFO - Building the national-scale example model optimisation problem

2024-01-27 14:23:22,856 - calliope.backend.backend_model - INFO - Optimisation Model | parameters | Generated.

2024-01-27 14:23:23,254 - calliope.backend.backend_model - INFO - Optimisation Model | variables | Generated.

2024-01-27 14:23:24,362 - calliope.backend.backend_model - INFO - Optimisation Model | global_expressions | Generated.

2024-01-27 14:23:26,319 - calliope.backend.backend_model - INFO - Optimisation Model | constraints | Generated.

2024-01-27 14:23:26,431 - calliope.backend.backend_model - INFO - Optimisation Model | objectives | Generated.

2024-01-27 14:23:26,432 - calliope - INFO - Solving the national-scale example model optimisation problem

2024-01-27 14:23:26,433 - calliope.model - INFO - Optimisation model | starting model in plan mode.

2024-01-27 14:23:26,682 - calliope.model - INFO - Backend: solver finished running. Time since start of solving optimisation problem: 0:00:00.248863

2024-01-27 14:23:26,683 - calliope.postprocess.postprocess - INFO - Postprocessing: started

2024-01-27 14:23:26,849 - calliope.postprocess.postprocess - INFO - Postprocessing: All values < 1e-10 set to 0 in flow_cap

2024-01-27 14:23:26,852 - calliope.postprocess.postprocess - INFO - Postprocessing: ended. Time since start of solving optimisation problem: 0:00:00.418241

2024-01-27 14:23:26,856 - calliope.model - INFO - Model: loaded model_data

Adding your own file logging handler¶

You may find it more practical to store logging information in files, particularly if you are running your model on a remote device or if you have a very large model.

Then, you can search through your logs using your favourite IDE.

In [6]:

Copied!

log_filepath = Path(".") / "outputs" / "5_logging"
log_filepath.mkdir(parents=True, exist_ok=True)
log_filepath = Path(".") / "outputs" / "5_logging"
log_filepath.mkdir(parents=True, exist_ok=True)

In [7]:

Copied!





# Grab the calliope logger, which will also automatically get the child loggers (e.g., `calliope.model`).
logger = logging.getLogger("calliope")

# Remove existing handlers (i.e., those introduced earlier in this notebook)
logger.handlers.clear()

# You can define your own custom formatter.
# See https://docs.python.org/3/library/logging.html#logrecord-attributes for available attributes.
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

# Set up a file handler, which will store log outputs in a file
file_handler = logging.FileHandler(log_filepath / "calliope.log")
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)

# You can also use logging in your scripts to add more information
logger.info("Loading the national-scale example model")
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])

logger.info("Building the national-scale example model optimisation problem")
m.build()

logger.info("Solving the national-scale example model optimisation problem")
m.solve()
# Grab the calliope logger, which will also automatically get the child loggers (e.g., `calliope.model`).
logger = logging.getLogger("calliope")

# Remove existing handlers (i.e., those introduced earlier in this notebook)
logger.handlers.clear()

# You can define your own custom formatter.
# See https://docs.python.org/3/library/logging.html#logrecord-attributes for available attributes.
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

# Set up a file handler, which will store log outputs in a file
file_handler = logging.FileHandler(log_filepath / "calliope.log")
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)

# You can also use logging in your scripts to add more information
logger.info("Loading the national-scale example model")
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])

logger.info("Building the national-scale example model optimisation problem")
m.build()

logger.info("Solving the national-scale example model optimisation problem")
m.solve()

Notice that there is no logging to the console here, but that there is now a file outputs/calliope.log that contains the logging information.

We can also log both to the console at one level and to file at another:

In [8]:

Copied!





# Grab the calliope logger, which will also automatically get the child loggers (e.g., `calliope.model`).
logger = logging.getLogger("calliope")

# Remove existing handlers (i.e., those introduced earlier in this notebook)
logger.handlers.clear()

# You can define your own custom formatter.
# See https://docs.python.org/3/library/logging.html#logrecord-attributes for available attributes.
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

# Add a ConsoleHandler (this is what `calliope.set_log_verbosity` is doing under the hood)
console_handler = logging.StreamHandler()
# Log to console at the INFO level
console_handler.setLevel(logging.INFO)
console_handler.setFormatter(formatter)
logger.addHandler(console_handler)

# Set up a file handler, which will store log outputs in a file
file_handler = logging.FileHandler(log_filepath / "calliope.log")
# Log to file at the DEBUG level
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)

# You can also use logging in your scripts to add more information
logger.info("Loading the national-scale example model")
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])

logger.info("Building the national-scale example model optimisation problem")
m.build()

logger.info("Solving the national-scale example model optimisation problem")
m.solve()
# Grab the calliope logger, which will also automatically get the child loggers (e.g., `calliope.model`).
logger = logging.getLogger("calliope")

# Remove existing handlers (i.e., those introduced earlier in this notebook)
logger.handlers.clear()

# You can define your own custom formatter.
# See https://docs.python.org/3/library/logging.html#logrecord-attributes for available attributes.
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

# Add a ConsoleHandler (this is what `calliope.set_log_verbosity` is doing under the hood)
console_handler = logging.StreamHandler()
# Log to console at the INFO level
console_handler.setLevel(logging.INFO)
console_handler.setFormatter(formatter)
logger.addHandler(console_handler)

# Set up a file handler, which will store log outputs in a file
file_handler = logging.FileHandler(log_filepath / "calliope.log")
# Log to file at the DEBUG level
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)

# You can also use logging in your scripts to add more information
logger.info("Loading the national-scale example model")
m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])

logger.info("Building the national-scale example model optimisation problem")
m.build()

logger.info("Solving the national-scale example model optimisation problem")
m.solve()

2024-01-27 14:23:34,481 - calliope - INFO - Loading the national-scale example model

2024-01-27 14:23:34,482 - calliope.model - INFO - Model: initialising

2024-01-27 14:23:34,600 - calliope.model - INFO - Model: preprocessing stage 1 (model_run)

2024-01-27 14:23:37,562 - calliope.model - INFO - Model: preprocessing stage 2 (model_data)

2024-01-27 14:23:37,579 - calliope.model - INFO - Model: preprocessing complete

2024-01-27 14:23:37,580 - calliope - INFO - Building the national-scale example model optimisation problem

2024-01-27 14:23:38,118 - calliope.backend.backend_model - INFO - Optimisation Model | parameters | Generated.

2024-01-27 14:23:38,433 - calliope.backend.backend_model - INFO - Optimisation Model | variables | Generated.

2024-01-27 14:23:39,617 - calliope.backend.backend_model - INFO - Optimisation Model | global_expressions | Generated.

2024-01-27 14:23:41,576 - calliope.backend.backend_model - INFO - Optimisation Model | constraints | Generated.

2024-01-27 14:23:41,690 - calliope.backend.backend_model - INFO - Optimisation Model | objectives | Generated.

2024-01-27 14:23:41,691 - calliope - INFO - Solving the national-scale example model optimisation problem

2024-01-27 14:23:41,692 - calliope.model - INFO - Optimisation model | starting model in plan mode.

2024-01-27 14:23:41,940 - calliope.model - INFO - Backend: solver finished running. Time since start of solving optimisation problem: 0:00:00.247723

2024-01-27 14:23:41,941 - calliope.postprocess.postprocess - INFO - Postprocessing: started

2024-01-27 14:23:42,107 - calliope.postprocess.postprocess - INFO - Postprocessing: All values < 1e-10 set to 0 in flow_cap

2024-01-27 14:23:42,110 - calliope.postprocess.postprocess - INFO - Postprocessing: ended. Time since start of solving optimisation problem: 0:00:00.418050

2024-01-27 14:23:42,113 - calliope.model - INFO - Model: loaded model_data

The log file will contain all calliope child logger outputs in one place. You will notice the name of the logger, which corresponds to the file where the log was recorded, at the second level of the log messages.

We can store each of these child loggers to a different file if we like:

In [9]:

Copied!





# Grab the calliope logger, which will also automatically get the child loggers (e.g., `calliope.model`).
logger = logging.getLogger("calliope")

# Remove existing handlers (i.e., those introduced earlier in this notebook)
logger.handlers.clear()

# You can define your own custom formatter.
# Here we don't include the logger name, as the filename will contain that information.
# See https://docs.python.org/3/library/logging.html#logrecord-attributes for available attributes.
formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")

for logger_name in logging.root.manager.loggerDict.keys():
    if not logger_name.startswith("calliope"):
        # There are lots of other loggers that calliope imports from its dependencies which we will ignore.
        # You can also dump these log to files if you like, by removing this conditional statement.
        continue
    # Set up a file handler, which will store log outputs in a file
    file_handler = logging.FileHandler(log_filepath / f"{logger_name}.log")
    # Log to file at the DEBUG level
    file_handler.setLevel(logging.DEBUG)
    file_handler.setFormatter(formatter)
    logging.getLogger(logger_name).addHandler(file_handler)

m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])
m.build()
m.solve()
# Grab the calliope logger, which will also automatically get the child loggers (e.g., `calliope.model`).
logger = logging.getLogger("calliope")

# Remove existing handlers (i.e., those introduced earlier in this notebook)
logger.handlers.clear()

# You can define your own custom formatter.
# Here we don't include the logger name, as the filename will contain that information.
# See https://docs.python.org/3/library/logging.html#logrecord-attributes for available attributes.
formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")

for logger_name in logging.root.manager.loggerDict.keys():
    if not logger_name.startswith("calliope"):
        # There are lots of other loggers that calliope imports from its dependencies which we will ignore.
        # You can also dump these log to files if you like, by removing this conditional statement.
        continue
    # Set up a file handler, which will store log outputs in a file
    file_handler = logging.FileHandler(log_filepath / f"{logger_name}.log")
    # Log to file at the DEBUG level
    file_handler.setLevel(logging.DEBUG)
    file_handler.setFormatter(formatter)
    logging.getLogger(logger_name).addHandler(file_handler)

m = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-01"])
m.build()
m.solve()