Running, Debugging, and Posting-processing

• Running a Scenario
• Finding an input parameter
• Running automatic tests
• Input and Output
• • Input
  • Output
• Cost breakdown in post-processing
• • Investment cost (CAPEX) per asset
  • Fixed O&M cost per asset
  • Flows operational cost per flow
• Setting the solver and its parameters
• Exploring infeasibility
• Speeding up model creation
• • Disable names of variables and constraints
  • Create a direct model

This section assumes users have already followed the basic Tutorials and are looking for specific instructions for certain features.

Running a Scenario

To run a scenario, use the function:

• run_scenario(connection)
• run_scenario(connection; output_folder)

The connection should have been created and the data loaded into it using TulipaIO. See the Workflow Tutorial for a complete guide on how to achieve this. The output_folder is optional if the user wants to export the output.

Finding an input parameter

Running automatic tests

To run the automatic tests on your installation of TulipaEnergyModel:

• Enter package mode (press "]")

pkg> test TulipaEnergyModel
# This takes a minute or two...

All tests should pass. (If you have an error in your analysis, it is probably not caused by TulipaEnergyModel.)

Input and Output

Tulipa runs from tables in DuckDB, which can be loaded from many formats (CSV, Parquet, etc). See the workflow tutorial for more information on inputting data.

Input

Tulipa runs from strictly defined files that follow the Schemas. See the workflow section for more information on how to work with the schema.

You can check the test/inputs folder for examples of different predefined energy systems and features. Moreover, Tulipa's Offshore Bidding Zone Case Study can be found in https://github.com/TulipaEnergy/Tulipa-OBZ-CaseStudy. It shows how to start from user-friendly files and transform the data into the input files in the Schemas through different functions.

Output

Outputs are sent from Tulipa to DuckDB and can be exported to various file formats.

To save the solution to CSV files, you can use export_solution_to_csv_files. See the Workflow Tutorial for an example showcasing this function.

Cost breakdown in post-processing

After solving the model, you can compute a detailed cost breakdown per asset or per flow directly from the solved variable and expression values. This does not modify the optimization model — it is purely a post-processing step.

The total objective is composed of several cost components (investment costs, fixed O&M costs, operational costs, etc.). Each component is stored as an aggregated value in the obj_breakdown table. The examples below show how to disaggregate these components by asset or by flow.

The general approach to construct these breakdowns is:

1. Identify the source: Look at the corresponding objective function in src/objectives/ to understand what variables, expressions, and cost coefficients are involved.
2. Get the solved values: Variable solutions are stored in the solution column of var_* tables. Expression values (like available units) must be evaluated via JuMP.value() on the in-memory expressions.
3. Get the cost coefficients: Join with the same tables the objective uses (t_objective_assets, t_objective_flows, asset_commission, etc.) to obtain discounted cost coefficients.
4. Combine: Multiply solved values by cost coefficients, applying the (1 - lambda) risk aversion weight when necessary.

DuckDB.query(connection, "SELECT * FROM obj_breakdown")

To save any result as a CSV file, register it as a DuckDB table and use COPY:

DuckDB.register_table(connection, result, "my_table_name")
output_file = joinpath(output_folder, "my_table_name.csv")
DuckDB.execute(connection, "COPY my_table_name TO '$output_file' (HEADER, DELIMITER ',')")

Investment cost (CAPEX) per asset

obj_assets_investment_cost = DuckDB.query(
    connection,
    "SELECT
        var.asset,
        var.milestone_year,
        obj.weight_for_asset_investment_discount
            * obj.investment_cost
            * obj.capacity
            AS cost_per_unit_invested,
        var.solution AS units_invested,
        (1 - mp.risk_aversion_weight_lambda)
            * cost_per_unit_invested
            * var.solution
            AS investment_cost,
    FROM var_assets_investment AS var
    LEFT JOIN t_objective_assets AS obj
        ON var.asset = obj.asset
        AND var.milestone_year = obj.milestone_year
    CROSS JOIN model_parameters AS mp
    ORDER BY var.asset, var.milestone_year
    ",
)

DuckDB.register_table(connection, obj_assets_investment_cost, "obj_assets_investment_cost")
output_file = joinpath(output_folder, "obj_assets_investment_cost.csv")
DuckDB.execute(connection, "COPY obj_assets_investment_cost TO '$output_file' (HEADER, DELIMITER ',')")

Fixed O&M cost per asset

using JuMP: value

# Evaluate the available units expressions (one value per row in the expr table)
expr_agg = energy_problem.expressions[:available_asset_units_aggregated_vintage_method]
avail_units = value.(expr_agg.expressions[:assets])

# Get cost coefficients from DuckDB (same query the objective uses, plus key columns)
cost_data = DuckDB.query(
    connection,
    "SELECT
        expr.id,
        expr.asset,
        expr.milestone_year,
        expr.commission_year,
        obj.weight_for_operation_discounts
            * asset_commission.fixed_cost
            * obj.capacity
            AS cost_coefficient,
    FROM expr_available_asset_units_aggregated_vintage_method AS expr
    LEFT JOIN asset_commission
        ON expr.asset = asset_commission.asset
        AND expr.commission_year = asset_commission.commission_year
    LEFT JOIN t_objective_assets AS obj
        ON expr.asset = obj.asset
        AND expr.milestone_year = obj.milestone_year
    ORDER BY expr.id
    ",
)

# Combine and build the result table
lambda = first(
    DuckDB.query(connection, "SELECT risk_aversion_weight_lambda FROM model_parameters"),
).risk_aversion_weight_lambda

obj_assets_fixed_cost = [
    (
        asset = row.asset,
        milestone_year = row.milestone_year,
        commission_year = row.commission_year,
        available_units = avail_units[row.id],
        cost_coefficient = row.cost_coefficient,
        fixed_cost = (1 - lambda) * avail_units[row.id] * row.cost_coefficient,
    ) for row in cost_data
]

# Register and export
DuckDB.register_table(connection, obj_assets_fixed_cost, "obj_assets_fixed_cost")
output_file = joinpath(output_folder, "obj_assets_fixed_cost.csv")
DuckDB.execute(connection, "COPY obj_assets_fixed_cost TO '$output_file' (HEADER, DELIMITER ',')")

Flows operational cost per flow

table_name = "obj_flows_operational_cost"
DuckDB.execute(
    connection,
    "CREATE OR REPLACE TABLE $table_name AS
    SELECT
        var.from_asset,
        var.to_asset,
        var.milestone_year,
        (1 - mp.risk_aversion_weight_lambda) * SUM(
            ss.probability
            * obj.weight_for_operation_discounts
            * rp_weight.total_weight_per_scenario
            * rp_res.resolution
            * obj.total_variable_cost
            * (var.time_block_end - var.time_block_start + 1)
            * var.solution
        ) AS operational_cost,
    FROM var_flow AS var
    LEFT JOIN t_objective_flows AS obj
        ON var.from_asset = obj.from_asset
        AND var.to_asset = obj.to_asset
        AND var.milestone_year = obj.milestone_year
    LEFT JOIN (
        SELECT milestone_year, rep_period, scenario,
               SUM(weight) AS total_weight_per_scenario
        FROM rep_periods_mapping
        GROUP BY milestone_year, rep_period, scenario
    ) AS rp_weight
        ON var.milestone_year = rp_weight.milestone_year
        AND var.rep_period = rp_weight.rep_period
    LEFT JOIN (
        SELECT milestone_year, rep_period, ANY_VALUE(resolution) AS resolution
        FROM rep_periods_data
        GROUP BY milestone_year, rep_period
    ) AS rp_res
        ON var.milestone_year = rp_res.milestone_year
        AND var.rep_period = rp_res.rep_period
    LEFT JOIN stochastic_scenario AS ss
        ON rp_weight.scenario = ss.scenario
    LEFT JOIN asset
        ON asset.asset = var.from_asset
    CROSS JOIN model_parameters AS mp
    WHERE asset.vintage_method != 'compact_efficiencies'
    GROUP BY var.from_asset, var.to_asset, var.milestone_year, mp.risk_aversion_weight_lambda
    ORDER BY var.from_asset, var.to_asset, var.milestone_year
    ",
)

Setting the solver and its parameters

By default, the model is solved using the HiGHS optimizer (or solver). To change this, you can give the functions run_scenario or create_model! a different optimizer.

Here is an example running the Tiny case using the GLPK optimizer:

using DuckDB, TulipaIO, TulipaEnergyModel, GLPK

input_dir = "../../test/inputs/Tiny" # you path will be different
connection = DBInterface.connect(DuckDB.DB)
read_csv_folder(connection, input_dir; schemas = TulipaEnergyModel.schema_per_table_name)
energy_problem = run_scenario(connection; optimizer = GLPK.Optimizer)
#OR create_model!(energy_problem; optimizer = GLPK.Optimizer)

In both cases above, the GLPK optimizer uses its default parameters, which you can query using default_parameters. To change any optimizer parameters, you can pass a dictionary to the optimizer_parameters keyword argument. The example below changes the maximum allowed runtime for GLPK to 1 second, which will probably cause it to fail to converge in time.

# change the optimizer parameters
parameter_dict = Dict("tm_lim" => 1) # list optimizer parameters as comma-separated parameter=>value pairs
energy_problem = run_scenario(connection; optimizer = GLPK.Optimizer, optimizer_parameters = parameter_dict)
#OR create_model!(energy_problem; optimizer = GLPK.Optimizer, optimizer_parameters = parameter_dict)
energy_problem.termination_status

If direct_model = false you can change the optimizer and parameters after creating the model (but before solving it) using the JuMP commands demonstrated below. For more information on direct_model, see Speed improvements in the model creation.

# create the model and solve with the default optimizer and optimizer parameters
energy_problem = EnergyProblem(connection)
create_model!(energy_problem)
solve_model(energy_problem)

# change the solver and parameters and resolve:
parameter_dict = Dict("tm_lim" => 1) # list optimizer parameters as comma-separated parameter=>value pairs

JuMP.set_optimizer(energy_problem.model, GLPK.Optimizer) # change the optimizer
for (k, v) in optimizer_parameters
    JuMP.set_attribute(energy_problem.model, k, v) # change the optimizer_parameters
end

solve_model(energy_problem) # solve the model with new optimizer & optimizer_parameters

For the complete list of parameters, check your chosen optimizer.

You can also pass these parameters via a file using the read_parameters_from_file function.

Exploring infeasibility

If your model is infeasible, you can try exploring the infeasibility with JuMP.compute_conflict! and JuMP.copy_conflict.

Use energy_problem.model for the model argument. For instance:

import MathOptInterface as MOI
using JuMP

if JuMP.termination_status(energy_problem.model) == MOI.INFEASIBLE
    JuMP.compute_conflict!(energy_problem.model)
    iis_model, reference_map = JuMP.copy_conflict(energy_problem.model)
    print(iis_model)
end

Speeding up model creation

Disable names of variables and constraints

If you want to speed-up model creation, consider disabling the naming of variables and constraints. Of course, removing the names will make debugging difficult (or impossible) - so enable/disable naming as needed for your analysis.

# Disable names while using run_scenario
run_scenario(connection; enable_names = false)

# OR while using create_model!
create_model!(energy_problem; enable_names = false)

For more information, see the JuMP documentation for Disable string names.

Create a direct model

If you want to reduce memory usage, consider using direct_model = true. This restricts certain actions after model creation, such as changing the optimizer.

# Create direct model with run_scenario
run_scenario(connection; direct_model = true)

# OR while using create_model!
create_model!(energy_problem; direct_model = true)

For more information, see the JuMP documentation for direct_model.