Financial machine learning developers often run into a core problem: a model that nails 24-hour returns might fall short when used for a 30-minute trade. This gap usually comes from a mismatch between what the model predicts and how it's actually used, rather than a flaw in the modeling itself. Knowing how to evaluate models across several horizons is key for building trading systems that work in practice.
Defining Forward Return Targets for Each Horizon
With log prices, forward return targets come from subtracting the current log price from the log price at a future point. For a 5-step horizon, the target is p_{t+5} - p_t. Each horizon—like 5, 15, 60, or 288 steps—needs its own target and separate evaluation. In Python, pandas can generate these targets:
import numpy as np
import pandas as pd
def build_forward_targets(prices, horizons):
log_prices = np.log(prices.astype(float))
return pd.DataFrame({ f"return_{h}": log_prices.shift(-h) - log_prices for h in horizons })
targets = build_forward_targets(prices, horizons=[5, 15, 60, 288])These horizons count observations, not fixed time spans. If your data isn't evenly spaced, you'll need to align targets by timestamps, since five observations might not mean five minutes or five hours. Keeping economic exposure consistent for each target matters, especially with irregular data. Overlapping targets show up when labels are generated often, so this overlap needs attention during validation.
Analyzing Signal Decay with Future Return Buckets
Predicting total returns doesn't show when the return happens. To see how long a signal stays useful, check it against a series of non-overlapping future return intervals. For five-minute steps, you might look at returns for 0-5 and 5-10 minutes. Here's a Python example:
import pandas as pd
import numpy as np
def future_return_buckets(log_prices, step=5, buckets=6):
result = {}
for i in range(buckets):
start = i * step
end = (i + 1) * step
result[f"{start}_{end}"] = (log_prices.shift(-end) - log_prices.shift(-start))
return pd.DataFrame(result)
log_prices = np.log(prices.astype(float))
future_returns = future_return_buckets(log_prices, step=5, buckets=6)
evaluation = future_returns.join(signal.rename("signal"))
decay = evaluation.corr(method="spearman").loc["signal"].drop("signal")
print(decay)This method reveals how the original signal lines up with returns in each future interval. These correlations give a basic sense of signal decay, but a deeper look should include conditional returns by signal strength, uncertainty, market regimes, and transaction costs. Claims about predictive power at intervals like 0–5 or 5–10 minutes depend on the dataset and method, and can't be assumed for other instruments or markets without more checks.
Managing Overlapping Labels in Model Validation
When forecasts come at short intervals, such as every five minutes for a 60-minute horizon, targets overlap. This overlap can make forecast errors serially dependent. Random train-test splits might put overlapping intervals in different sets, causing information leakage. To avoid this, always record both the prediction timestamp and the label-end timestamp for each observation. If training and testing labels share future data, remove the affected training points. For walk-forward validation, make sure training data only uses information from before the test period, and drop any training labels that reach into the test window. Statistical comparisons should factor in serial correlation, and tools like Diebold–Mariano help compare predictive accuracy when errors are dependent.
Separating Forecast Quality from Trading Performance
Prediction error doesn't tell the whole story about trading profits. For each horizon, report metrics like mean absolute error (MAE) or root mean squared error (RMSE), information coefficient (IC) or rank IC, calibration, prediction stability, signal persistence, turnover, transaction costs, and net returns under a set execution policy. The holding period should be chosen based on how the strategy is evaluated, not just matched to the forecast horizon. Research by Gârleanu and Pedersen shows how signal mean-reversion speed and transaction costs shape optimal portfolio moves.
Designing a Multi-Horizon Model Architecture
In real-world setups, different models can handle different horizons: short-term models for execution and intermediate-term models for trend management. Their outputs should be combined with a clear decision policy, rather than just voting on buy or sell signals. Each model in production needs a documented prediction target, information cutoff, update frequency, validation method, and intended use. This separation matters for quantitative research, since a model's value depends on both its predictive accuracy and its role in a trading and risk-management system.
Key Takeaways for Developers
Evaluating multi-horizon trading models in Python means more than just training on different target windows. Developers need to handle overlapping labels, signal decay, prediction uncertainty, execution costs, and the link between forecasts and trading decisions. Use models only for decisions that their validated predictive information can support. The goal is to build a system where predictive signals, portfolio changes, and risk controls work together smoothly.
No government agency, securities regulator, or exchange requires a specific forecast horizon, validation method, or overlap adjustment for multi-horizon trading-model evaluation. This topic remains a technical framework for developers and quantitative researchers. There are no current market developments, product launches, or regulatory actions tied to these methods. For more technical background, developers can review peer-reviewed research and official documentation from leading financial machine learning authors.
References: Gârleanu & Pedersen (2013), Grant, Mrazik & Satchell (2026), López de Prado (2018).
