Getting started¶
The Helyx Backtest Service replays a selected intraday trading policy against historical Nord Pool order-book data using your portfolio positions.
You provide three inputs:
- Position Manager targets
- Cleared day-ahead positions
- Realised production
The service returns detailed 15-minute economics and aggregated results for each run.
Helyx market data and the trading-engine implementation remain internal to the service. The backtest uses only the client datasets uploaded for the requested run.
A standard workflow is:
Validate → Upload → Request → Monitor → Fetch → Compare
Before starting, you need:
- the service-account JSON key provided by Helyx;
- your assigned bucket name, for example
helyx-client-esg; - Python 3.11 or later;
- input data for the requested backtest period.
Step 1 — Install and authenticate¶
Install the helyx command-line tool once per machine and point it at your key and bucket.
pip install keyring keyrings.google-artifactregistry-auth
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/your-key.json
pip install helyx-sdk --extra-index-url https://europe-west4-python.pkg.dev/helyx-intraday-prod/helyx-sdk/simple/
export HELYX_BUCKET=helyx-client-esg
Verify the installation:
helyx coverage # market-data period available for backtests
helyx policies # published policy versions and the parameters each exposes
helyx docs # this guide, for the installed version
helyx coverage returns a start date, an end date and any gaps. Every run must fall inside that range. Add the two export lines to your shell profile so they persist. The same operations are available in Python via from helyx_sdk import Client.
Step 2 — Prepare three files¶
All three datasets must use 15-minute delivery periods in UTC.
| File | Purpose | Format | Required fields |
|---|---|---|---|
pm_targets.jsonl |
Position Manager instructions and their revisions | JSON Lines, the Position Manager export as is; no other format is accepted | see Formats |
da_cleared.csv |
Cleared day-ahead position | CSV or Parquet | delivery_from_utc, cleared_mwh |
delivered.csv |
Realised production | CSV or Parquet | delivery_from_utc, delivered_mwh |
Sign convention
| Field | Positive | Negative |
|---|---|---|
cleared_mwh |
day-ahead buy | day-ahead sell |
delivered_mwh |
production | consumption |
quantity_mw |
intraday buy target | intraday sell target |
For a solar portfolio that sold its expected production day-ahead, cleared_mwh will normally be negative while delivered_mwh will normally be positive.
Formats
pm_targets.jsonlis your Position Manager's own export, unchanged: one JSON record per target revision, withportfolio_id,delivery_from_utc,delivery_to_utc,created_utc,target(the price–volume levels) andda_position_delta. Records for the same delivery period with latercreated_utcare revisions; the backtest replays them in chronological order.created_utcmust be earlier thandelivery_from_utc; this prevents look-ahead.- Timestamps, in every file: UTC, written as
YYYY-MM-DD HH:MM:SS(for example2026-03-02 12:00:00) or as ISO 8601 with an explicitZor offset. A timestamp without a zone is taken as UTC. Delivery periods must start on quarter hours. - CSV files: a header row, comma-separated,
.as the decimal separator, no thousands separators. Parquet is accepted for the two volume files with the same column names.
Exact schemas are on the Formats page. A complete, valid one-day example set is available: helyx examples ./helyx-examples writes it to a folder, or download the files from intraday-docs.helyx-energy.com/examples/ (pm_targets.jsonl, da_cleared.csv, delivered.csv, config.yaml, request.json). Use it to check your own export against a known-good shape.
The da_cleared and delivered datasets must cover every delivery period required by the Position Manager stream, on a complete 15-minute grid with no duplicates.
The da_cleared dataset must correspond to the same day-ahead position against which the Position Manager stream was generated. This relationship cannot be validated from the input files.
Step 3 — Validate¶
Run the same checks the service applies, locally and before any upload:
helyx validate --pm-targets pm_targets.jsonl --da-cleared da_cleared.csv --delivered delivered.csv
Each dataset receives a list of [ok] / [FAIL] checks and a VALID or INVALID verdict. A FAIL line names the problem (for example, "12 periods missing between …" or "3 revisions created at/after delivery start"). Correct the file and validate again until every dataset is VALID. A dataset that passes locally passes on upload, except for the server-side checks listed under Troubleshooting.
Step 4 — Upload¶
Upload each dataset under a name of your choice. The name is how the dataset is referenced in a run request.
helyx push delivered delivered.csv --name realised
helyx push da_cleared da_cleared.csv --name actual_da
helyx push pm_targets pm_targets.jsonl --name improved_floor20
Each upload validates first and refuses an invalid dataset. On success, the dataset is stored as a fixed version that can be referenced reproducibly by later runs. The three datasets change at different rates: delivered is a metered fact and is normally uploaded once; da_cleared changes with the day-ahead bidding strategy; pm_targets changes per intraday strategy. Upload a modified dataset under a new name rather than over an existing one, so that earlier runs remain comparable.
Step 5 — Request a run¶
A run specifies:
- the three input datasets;
- a policy version;
- the requested delivery period;
- an optional configuration;
- a unique request ID.
helyx request \
--id esg-2026-09-24-floor20 \
--pm-targets improved_floor20 \
--da-cleared actual_da \
--delivered realised \
--policy v1.3.0 \
--start 2026-03-01 \
--end 2026-05-31
The start and end dates are inclusive UTC delivery dates and must fall within the available market-data coverage.
Use a new request ID for each distinct experiment. A request ID already associated with different inputs or parameters is rejected.
Timing and limits. Days are simulated in parallel; a single day takes about 15 minutes, a three-month run about 80 minutes of simulation and, with the hourly pickup, 2–3 hours from request to results. Runs are accepted in the order received and executed one at a time per client. Up to 6 runs per day are accepted; further requests are deferred and accepted automatically on the following day, with nothing to resubmit. A run may cover any period of up to 366 days inside the published coverage. Rejections happen at acceptance, within the hour, never after a run has started.
Monitoring
helyx runs # all runs and their state
helyx status esg-2026-09-24-floor20 # one run in detail
| State | Meaning |
|---|---|
unknown request (the CLI's literal output) |
the request has been written but not yet ingested by the backtest service; ingestion happens hourly |
queued |
accepted; the datasets as uploaded at that moment are frozen for this run |
deferred |
waiting for execution capacity; accepted automatically later |
rejected |
validation failed; reason identifies the check |
running |
the simulation is executing |
done |
results are available in the outbox |
failed |
execution failed inside the service; Helyx is notified |
Step 6 — Fetch and read the results¶
helyx fetch esg-2026-09-24-floor20 ./results
The run's outbox contains:
| File | Contents |
|---|---|
economics_<YYYY-MM-DD>.csv, one per delivery day |
96 rows, one per 15-minute period, with the full settlement chain |
economics_monthly.csv |
one row per day, a TOTAL row per month and a TOTAL row for the run |
run_parameters.json |
the resolved run: dataset names, generations and hashes, every parameter value, policy version |
validation.json, status.json |
the checks applied at acceptance, and the run state |
The primary metric for comparing intraday strategies is id_value_add_eur. It measures the difference between the realised economics with intraday trading and the counterfactual economics without intraday trading.
Define:
| Symbol | Meaning |
|---|---|
| A | day-ahead cash flow |
| B | day-ahead cash flow + imbalance settlement with no intraday trading |
| C | day-ahead cash flow + intraday cash flows + imbalance settlement of the remaining position |
Then id_value_add_eur = C − B.
Interpretation:
id_value_add_eur > 0: intraday trading improved the result relative to no intraday trading;id_value_add_eur < 0: intraday trading reduced the result relative to no intraday trading.
Cash-flow convention: cash received is positive; cash paid is negative.
id_value_add_eur is not total portfolio P&L. It excludes economics outside the simulated trading and settlement chain, including subsidies. B does not depend on execution, so it is identical across runs on the same datasets; id_value_add_eur is therefore the quantity to compare between runs.
Important output fields
| Field | Meaning |
|---|---|
dah_forecast_mw |
reporting view of the cleared day-ahead sold volume, represented as a positive quantity |
actual_mw |
realised production |
delta_mw |
realised production minus day-ahead sold volume, before intraday trading (positive = long, negative = short) |
id_sell_mw |
intraday sold volume |
id_sell_vwap |
volume-weighted average intraday sell price |
id_buy_mw |
intraday bought volume |
id_buy_vwap |
volume-weighted average intraday buy price |
dah_price |
day-ahead price |
imb_price_applied |
imbalance price applied to the final residual position |
id_value_add_eur |
incremental value of intraday trading versus no intraday trading |
Although the input field cleared_mwh uses the trading sign convention (a day-ahead sale is negative), some reporting fields use positive magnitudes for readability. This is a presentation transformation only and does not change the underlying economics.
All settlement calculations use UTC delivery periods. The CET field is provided for display only and does not affect the calculation.
In a notebook
import pandas as pd, glob
run = "results/esg-2026-09-24-floor20"
monthly = pd.read_csv(f"{run}/economics_monthly.csv")
print(monthly[monthly.delivery_date.str.contains("TOTAL")][["delivery_date", "id_sell_mwh", "id_buy_mwh", "id_value_add_eur"]])
days = pd.concat(pd.read_csv(f) for f in sorted(glob.glob(f"{run}/economics_2026-*.csv")))
days["delivery_start_utc"] = pd.to_datetime(days.delivery_start_utc)
days.groupby("delivery_date").id_value_add_eur.sum().plot(kind="bar", title="value add per day, EUR")
days.groupby(days.delivery_start_utc.dt.hour).id_value_add_eur.sum().plot(kind="bar", title="value add by hour of day, EUR")
Why did a day trade nothing?
helyx status <request-id> includes counts of decision reasons per run.
| Reason | Meaning |
|---|---|
BELOW_MIN_CORRECTION |
the position was below the configured deadband |
PRICE_FLOOR |
available market prices did not satisfy the minimum sell price |
DA_PRICE_GATED |
trading was blocked by the configured day-ahead price rule |
HELD_BACK |
the remaining volume was intentionally excluded by the hold-back rule |
A day with only these reasons indicates that the policy followed its configured rules; it does not by itself indicate an execution failure.
Step 7 — Choose a policy version¶
The trading logic is fixed for each policy version. helyx policies lists the published versions, their defaults and the parameters each exposes.
| Version | Deadband | Flush | Fee | Exposed parameters |
|---|---|---|---|---|
| v1.1.0 | 0.5 MW | lifted 60 min before gate | €0.10/MWh | deadband, flush, fee, per-tag window, urgency schedule, pacing curve |
| v1.2.0 | 1.0 MW | off: sub-threshold corrections settle on imbalance | €0.10/MWh | as v1.1.0 |
| v1.3.0 | 1.0 MW | off | €0.10/MWh | as v1.2.0, plus per-tag hold_back_mw and da_price_gate |
Guidance:
- v1.3.0 is the current policy version. With its default configuration, it preserves the v1.2.0 decision behaviour while additionally exposing the hold-back and day-ahead price-gating parameters.
- Earlier versions remain available for reproducing historical runs created under those versions.
- Compare versions only on identical datasets and period. A version change alters the trading logic and is not a parameter change.
Step 8 — Change a setting and run again¶
A defined set of parameters can be changed through a YAML configuration file, uploaded under a name and referenced with --config. Parameters not specified take the policy version's defaults; every value that applied is recorded in run_parameters.json.
schema_version: "1"
deadband_mw: 1.0
flush_minutes: 0
fee_eur_mwh: 0.10
tags:
schedule:
hold_back_mw: 1.0
da_price_gate:
sell_only_below: -5
helyx push config my_settings.yaml --name gate5-holdback1
helyx request --id esg-2026-09-24-gate5 --config gate5-holdback1 --policy v1.3.0 \
--pm-targets improved_floor20 --da-cleared actual_da --delivered realised --start 2026-03-01 --end 2026-05-31
Positions carry a tag derived from their cause: forecast_update (a forecast change), schedule (a day-ahead committed block), availability_update (a change in plant availability). Per-tag parameters apply to positions with that tag.
| Parameter | Meaning | Default (v1.3.0) |
|---|---|---|
deadband_mw |
On forecast_update positions, corrections smaller than this threshold are deferred rather than immediately worked. If the deadband is later lifted by flush_minutes, the remaining volume becomes eligible for execution; otherwise it may settle through imbalance. |
1.0 MW |
flush_minutes |
Minutes before gate closure at which the deadband is lifted so the remainder is worked. 0 disables the flush: a remainder below the deadband settles on imbalance. |
0 |
fee_eur_mwh |
External transaction fee applied per MWh traded. The execution test includes this fee when evaluating whether an order still satisfies its configured price constraint. | 0.10 |
tags.<tag>.hold_back_mw |
Never trade the last N MW of a position, as a buffer against forecast error. | 0 (off) |
tags.<tag>.da_price_gate |
sell_only_below: x works a sell position only on periods whose day-ahead price is below x €/MWh; buy_only_above: y likewise for buys. The day-ahead price is known before intraday trading starts. |
off |
tags.<tag>.default_window_minutes |
Trading window applied when a position carries no creation time (Position Manager records always do). | 1440 / 60 / 240 |
tags.<tag>.pacing_curve |
How fast a position is worked over its window, as points [fraction of time, fraction done]. [[0,0],[1,1]] is uniform; null is unpaced. |
per tag |
tags.<tag>.urgency_schedule |
How aggressiveness increases towards gate closure, as points [minutes to gate, urgency 0–1]. |
per tag |
Use one-parameter-at-a-time comparisons where possible:
- run the baseline;
- change one setting;
- rerun using the same datasets and policy version;
- compare
id_value_add_eurby day; - inspect traded volume and residual imbalance;
- verify the resolved parameters in
run_parameters.json.
For identical frozen inputs, policy version, engine version and resolved configuration, the service produces deterministic results, independent of how a run is split across compute tasks; this is part of the engine's release tests. Parameters outside the exposed set are rejected at upload.
Troubleshooting¶
| Message or state | Meaning | Action |
|---|---|---|
| market-data coverage error | requested dates fall outside available coverage or intersect a gap | run helyx coverage and adjust the period |
pm_targets '<name>' not found |
the referenced dataset does not exist under that name | upload the dataset or correct the name |
| day-ahead / delivered coverage error | one of the datasets does not cover all Position Manager periods | complete the dataset, validate it, and upload it under a new name |
Additional properties are not allowed |
the configuration contains an unsupported parameter | check helyx policies and remove or correct the parameter |
| PM created at/after delivery | a target revision introduces look-ahead | correct the timestamp |
REQUEST_ID_CONFLICT |
the request ID was already used for a different request | use a new request ID |
deferred |
the run is waiting for available execution capacity | no action required |
failed |
execution failed inside the service | inspect helyx status <request-id> and contact Helyx with the request ID if needed |
If local validation succeeds but the request is rejected, the rejection reason identifies the server-side validation that failed: market-data coverage, the policy's parameter set, or the pairing of the three datasets.