Reference

DataValidationException

Exception related to data validation of the Tulipa Energy Model input data.

EnergyProblem

Structure to hold all parts of an energy problem. It is a wrapper around various other relevant structures. It hides the complexity behind the energy problem, making the usage more friendly, although more verbose.

Fields

• db_connection: A DuckDB connection to the input tables in the model.
• variables: A dictionary of TulipaVariables containing the variables of the model.
• expressions: A dictionary of TulipaExpressions containing the expressions of the model attached to tables.
• constraints: A dictionary of TulipaConstraints containing the constraints of the model.
• profiles: Holds the profiles per rep_period or over_clustered_year in dictionary format. See ProfileLookup.
• model_parameters: A ModelParameters structure to store all the parameters that are exclusive of the model.
• model: A JuMP.Model object representing the optimization model.
• solved: A boolean indicating whether the model has been solved or not.
• objective_value: The objective value of the solved problem (Float64).
• termination_status: The termination status of the optimization model.

Constructor

• EnergyProblem(connection): Constructs a new EnergyProblem object with the given connection.

The constraints_partitions field is computed from the representative_periods, and the other fields are initialized with default values.

See the basic example tutorial to see how these can be used.

ModelParameters(;key = value, ...)
ModelParameters(path; ...)
ModelParameters(connection; ...)
ModelParameters(connection, path; ...)

Structure to hold the model parameters. Some values are defined by default and some required explicit definition.

If path is passed, it is expected to be a string pointing to a TOML file with a key = value list of parameters. Explicit keyword arguments take precedence.

If connection is passed, the default discount_year is set to the minimum of all milestone years. In other words, we check for the table year_data for the column year where the column is_milestone is true. Explicit keyword arguments take precedence.

If both are passed, then path has preference. Explicit keyword arguments take precedence.

Parameters

• discount_rate::Float64 = 0.0: The model discount rate.
• discount_year::Int: The model discount year.

Structure to hold the dictionaries of profiles.

Structure to hold the JuMP constraints for the TulipaEnergyModel

Structure to hold some JuMP expressions that are not attached to constraints but are attached to a table.

TulipaTabularIndex

Abstract structure for TulipaVariable, TulipaConstraint and TulipaExpression. All deriving types must satisfy:

• Have fields
  • indices::DuckDB.QueryResult
  • table_name::String

Structure to hold the JuMP variables for the TulipaEnergyModel

_check_if_table_exists(connection, table_name)

Check if table table_name exists in the connection.

_create_group_table_if_not_exist!(
    connection,
    table_name,
    grouped_table_name,
    group_by_columns,
    array_agg_columns;
    rename_columns = Dict(),
    order_agg_by = "id",
)

Create a grouped table grouping the table_name into the grouped_table_name. The group_by_columns are the columns that are used in the group by (e.g., asset, year, repperiod), and the `arrayaggcolumns` are the columns that are aggregated into arrays (e.g., id, timeblockstart, timeblock_end).

It is expected that the original table has an id column, which is used in the ordering of the array_agg_columns. Otherwise, please pass the argument order_agg_by with the column that should be used for this ordering.

If one of the columns has to be renamed, use the rename_columns dictionary.

_profile_aggregate(profiles, tuple_key, time_block, agg_functions, default_value)

Aggregates the profiles[tuple_key] over the time_block using the agg_function function. If the profile does not exist, uses default_value instead of each profile value.

profiles should be a dictionary of profiles, and tuple_key should be either (profile_name, year, rep_period) for the profiles of representative periods or (profile_name, year) for the profiles over clustered years.

If profiles[tuple_key] exists, then this function computes the aggregation of V = profiles[tuple_key] over the range time_block using the aggregator agg_function, i.e., agg_function(V[time_block]). If it does not exist, then V[time_block] is substituted by a vector of the corresponding size and default_value.

add_capacity_constraints!(connection, model, expressions, constraints, profiles)

Adds the capacity constraints for all asset types to the model

add_consumer_constraints!(connection, model, constraints, profiles)

Adds the consumer asset constraints to the model.

add_conversion_constraints!(model, constraints)

Adds the conversion asset constraints to the model.

add_energy_constraints!(connection, model, constraints, profiles)

Adds the energy constraints for assets within the period blocks of the timeframe (inter-temporal) to the model.

add_expression_terms_over_clustered_year_constraints!(
    connection,
    cons,
    flow,
    profiles;
    is_storage_level = false,
)

Computes the incoming and outgoing expressions per row of df_inter for the constraints that are between (inter) the representative periods.

This function is only used internally in the model.

add_expression_terms_rep_period_constraints!(
    connection,
    cons,
    flow;
    use_highest_resolution = true,
    multiply_by_duration = true,
    add_min_outgoing_flow_duration = false,
)

Computes the incoming and outgoing expressions per row of cons for the constraints that are within (intra) the representative periods.

This function is only used internally in the model.

This strategy is based on the replies in this discourse thread:

• https://discourse.julialang.org/t/help-improving-the-speed-of-a-dataframes-operation/107615/23

Implementation

This expression computation uses a workspace to store all variables defined for each timestep. The idea of this algorithm is to append all variables defined at time timestep in workspace[timestep] and then aggregate then for the constraint time block.

The algorithm works like this:

1. Loop over each group of (asset, year, rep_period)

1.1. Loop over each variable in the group: (varid, vartimeblockstart, vartimeblockend) 1.1.1. Loop over each timestep in vartimeblockstart:vartimeblockend 1.1.1.1. Compute the coefficient of the variable based on the repperiod resolution and the variable efficiency 1.1.1.2. Store (varid, coefficient) in workspace[timestep] 1.2. Loop over each constraint in the group: (consid, constimeblockstart, constimeblockend) 1.2.1. Aggregate all variables in workspace[timestep] for timestep in the time block to create a list of variable ids and their coefficients [(var_id1, coef1), ...] 1.2.2. Compute the expression using the variable container, the ids and coefficients

Notes:

• On step 1.2.1, the aggregation can be either by uniqueness of not, i.e., if the variable happens in more that one workspace[timestep], should we add up the coefficients or not. This is defined by the keyword

multiply_by_duration

add_flow_variables!(connection, model, variables)

Adds flow variables to the optimization model based on data from the variables. The flow variables are created using the @variable macro for each row in the :flows table.

add_group_constraints!(connection, model, variables, constraints)

Adds group constraints for assets that share a common limits or bounds.

add_hub_constraints!(model, constraints)

Adds the hub asset constraints to the model.

add_investment_variables!(model, variables)

Adds investment, decommission, and energy-related variables to the optimization model, and sets integer constraints on selected variables based on the input data.

add_ramping_and_unit_commitment_constraints!(
    connection,
    model,
    variables,
    expressions,
    constraints,
    profiles
)

Adds the ramping constraints for producer and conversion assets where ramping = true in assets_data

add_storage_constraints!(connection, model, variables, expressions, constraints, profiles)

Adds the storage asset constraints to the model.

add_storage_variables!(connection, model, variables)

Adds storage-related variables to the optimization model, including storage levels for both intra-representative periods and inter-representative periods, as well as charging state variables. The function also optionally sets binary constraints for certain charging variables based on storage methods.

add_transport_constraints!(connection, model, variables, expressions, constraints, profiles)

Adds the transport flow constraints to the model.

add_unit_commitment_variables!(model, variables)

Adds unit commitment variables to the optimization model based on the :units_on indices. Additionally, variables are constrained to be integers based on the unit_commitment_integer property.

attach_coefficient!(cons, name, container)

Attach a coefficient named name stored in container. This checks that the container length matches the stored indices number of rows.

attach_constraint!(model, cons, name, container)

Attach a constraint named name stored in container, and set model[name] = container. This checks that the container length matches the stored indices number of rows.

attach_expression!(cons_or_expr, name, container)
attach_expression!(model, cons_or_expr, name, container)

Attach a expression named name stored in container, and optionally set model[name] = container. This checks that the container length matches the stored indices number of rows.

attach_expression_on_constraints_grouping_variables!(
    connection,
    constraint,
    variable,
    expr_name;
    agg_strategy
)

Computes the intersection of the constraint and variable grouping by (:asset, :year, :rep_period).

The intersection is made on time blocks, so both the constraint table and the variable table must have columns id, timeblockstart and timeblockend.

The variable expr_name will be the name of the attached expression.

The agg_strategy must be either :sum, :mean, and :unique_sum, indicating how to aggregate the variables for a given constraint time block.

Implementation

The expression uses a workspace to store all variables defined for each timestep. The idea of this algorithm is to append all variables defined at time timestep in workspace[timestep] and then aggregate then for the constraint time block.

The algorithm works like this:

1. Loop over each group of (asset, year, rep_period)

1.1. Loop over each variable in the group: (varid, vartimeblockstart, vartimeblockend) 1.1.1. Loop over each timestep in vartimeblockstart:vartimeblockend 1.1.1.1. Compute the coefficient of the variable based on the repperiod resolution and the variable efficiency 1.1.1.2. Store (varid, coefficient) in workspace[timestep] 1.2. Loop over each constraint in the group: (consid, constimeblockstart, constimeblockend) 1.2.1. Aggregate all variables in workspace[timestep] for timestep in the time block to create a list of variable ids and their coefficients [(var_id1, coef1), ...] 1.2.2. Compute the expression using the variable container, the ids and coefficients

Notes:

• On step 1.2.1, the aggregation can be by either
  • :sum - add the coefficients
  • :mean - add the coefficients and divide by number of times the variable appears
  • :unique_sum - use 1.0 for the coefficient (this is not robust)

compute_dual_variables!(model)

Compute the dual variables for the given model.

If the model does not have dual variables, this function fixes the discrete variables, optimizes the model, and then computes the dual variables.

Arguments

• model: The model for which to compute the dual variables.

Returns

A named tuple containing the dual variables of selected constraints.

create_internal_tables!(connection)

Creates internal tables.

create_model!(energy_problem; kwargs...)

Create the internal model of an TulipaEnergyModel.EnergyProblem. Any keyword argument will be passed to the underlyting create_model.

model = create_model(
    connection,
    variables,
    expressions,
    constraints,
    profiles,
    model_parameters;
    write_lp_file = false,
    enable_names = true
)

Create the energy model manually. We recommend using create_model! instead.

create_unrolled_partition_table!(connection)

Create unrolled partitions tables from existing DuckDB tables, i.e., adds the time block start and end information.

Input

The following tables are expected to exist in the connection, containing the partition of some, or all, assets or flows, with their respective time information.

• assets_rep_periods_partitions
• flows_rep_periods_partitions
• assets_timeframe_partitions

Output

The generated tables are the unrolled version of the tables above. It transforms each row in (possibly) multiple rows. The columns specification and partition are used to determine the time blocks, and are replaced by columns time_block_start and time_block_end.

• asset_time_resolution_rep_period
• flow_time_resolution_rep_period
• asset_time_resolution_over_clustered_year

default_parameters(Val(optimizer_name_symbol))
default_parameters(optimizer)
default_parameters(optimizer_name_symbol)
default_parameters(optimizer_name_string)

Returns the default parameters for a given JuMP optimizer. Falls back to Dict() for undefined solvers.

Arguments

There are four ways to use this function:

• Val(optimizer_name_symbol): This uses type dispatch with the special Val type. Pass the solver name as a Symbol (e.g., Val(:HiGHS)).
• optimizer: The JuMP optimizer type (e.g., HiGHS.Optimizer).
• optimizer_name_symbol or optimizer_name_string: Pass the name in Symbol or String format and it will be converted to Val.

Using Val is necessary for the dispatch. All other cases will convert the argument and call the Val version, which might lead to type instability.

Examples

using HiGHS
default_parameters(HiGHS.Optimizer)

# output

Dict{String, Any} with 1 entry:
  "output_flag" => false

Another case

default_parameters(Val(:Cbc))

# output

Dict{String, Any} with 1 entry:
  "logLevel" => 0

default_parameters(:Cbc) == default_parameters("Cbc") == default_parameters(Val(:Cbc))

# output

true

export_solution_to_csv_files(output_file, connection, variables, constraints)

Saves the solution in CSV files inside output_folder. Notice that this assumes that the solution has been computed by save_solution!.

export_solution_to_csv_files(output_folder, energy_problem)

Saves the solution from energy_problem in CSV files inside output_file. Notice that this assumes that the solution has been computed by save_solution!.

read_parameters_from_file(filepath)

Parse the parameters from a file into a dictionary. The keys and values are NOT checked to be valid parameters for any specific solvers.

The file should contain a list of lines of the following type:

key = value

The file is parsed as TOML, which is intuitive. See the example below.

Example

# Creating file
filepath, io = mktemp()
println(io,
  """
    true_or_false = true
    integer_number = 5
    real_number1 = 3.14
    big_number = 6.66E06
    small_number = 1e-8
    string = "something"
  """
)
close(io)
# Reading
read_parameters_from_file(filepath)

# output

Dict{String, Any} with 6 entries:
  "string"         => "something"
  "integer_number" => 5
  "small_number"   => 1.0e-8
  "true_or_false"  => true
  "real_number1"   => 3.14
  "big_number"     => 6.66e6

energy_problem = run_scenario(connection; output_folder, optimizer, parameters, model_parameters_file, write_lp_file, enable_names, log_file, show_log)

Run the scenario in the given connection and return the energy problem.

The optimizer and parameters keyword arguments can be used to change the optimizer (the default is HiGHS) and its parameters. The variables are passed to the solve_model function.

Set write_lp_file = true to export the problem that is sent to the solver to a file for viewing. Set enable_names = false to turn off variable and constraint names (faster model creation). Set show_log = false to silence printing the log while running.

Specify a output_folder name to export the solution to CSV files. Specify a model_parameters_file name to load the model parameters from a TOML file. Specify a log_file name to export the log to a file.

save_solution!(connection, model, variables, constraints; compute_duals = true)

Saves the primal and dual variables values, in the following way:

• For each variable in variables, get the solution value and save it in a

column named solution in the corresponding dataset.

• For each constraint in constraints, get the dual of each attached

constraint listed in constraint.constraint_names and save it to the dictionary constraint.duals with the key given by the name.

Notice that the duals are only computed if compute_duals is true.

save_solution!(energy_problem; compute_duals = true)

solve_model(model[, optimizer; parameters])

Solve the JuMP model. The optimizer argument should be an MILP solver from the JuMP list of supported solvers. By default we use HiGHS.

The keyword argument parameters should be passed as a list of key => value pairs. These can be created manually, obtained using default_parameters, or read from a file using read_parameters_from_file.

Examples

parameters = Dict{String,Any}("presolve" => "on", "time_limit" => 60.0, "output_flag" => true)
solve_model(model, HiGHS.Optimizer; parameters = parameters)

solve_model!(energy_problem[, optimizer; parameters, save_solution = true])

Solve the internal model of an energy_problem. If save_solution, then the solution and dual variables are computed and saved by save_solution!.

validate_data!(connection)

Raises an error if the data is not valid.