API reference

Every public name is re-exported from overfitlab. Sharpe ratios are per period unless a name ends in _annual, and kurtosis is non-excess (normal = 3).

Sharpe statistics

Sharpe ratio statistics: moments, annualisation, PSR and minimum track record length.

"Per-period" Sharpe ratios are not annualised. Kurtosis is non-excess (normal = 3) everywhere, as in Bailey and Lopez de Prado.

sharpe_ratio(returns, *, axis=0, ddof=1)

Return the per-period Sharpe ratio mean / std.

A series with zero variance has no measurable Sharpe ratio and yields 0.0, so a flat strategy inside a trial matrix does not abort the evaluation.

Parameters:

Name	Type	Description	Default
returns	array_like	Per-period returns, one- or two-dimensional.	required
axis	int	Axis of observations.	0
ddof	int	Delta degrees of freedom of the standard deviation.	1

Returns:

Type	Description
float or ndarray	A float for one-dimensional input, otherwise one value per series.

Raises:

Type	Description
ValueError	If there are fewer than two observations or the input has NaN or inf.

Examples:

>>> round(sharpe_ratio([0.01, 0.02, 0.03]), 6)
2.0

return_moments(returns)

Return the Sharpe ratio, skewness and non-excess kurtosis of a series.

Skewness is :math:m_3 / m_2^{3/2} and kurtosis :math:m_4 / m_2^2, with central moments :math:m_k averaged over n_obs, without a small-sample correction.

Parameters:

Name	Type	Description	Default
returns	array_like	Per-period returns, one-dimensional.	required

Returns:

Type	Description
ReturnMoments	Per-period Sharpe ratio, skewness, kurtosis (normal = 3) and n_obs.

Raises:

Type	Description
ValueError	If there are fewer than two observations or the series is constant.

Examples:

>>> m = return_moments([0.01, -0.03, 0.01, 0.01])
>>> round(m.skew, 3), round(m.kurtosis, 3), m.n_obs
(-1.155, 2.333, 4)

annualize_sharpe(sr, periods_per_year, *, autocorr=None)

Annualise a per-period Sharpe ratio.

Without autocorr this multiplies by :math:\sqrt{q}. With autocorr it applies Lo (2002, eq. 22) for AR(1) returns, :math:\eta(q) = \sqrt{q}\,[1 + \frac{2\rho}{1-\rho}(1 - \frac{1-\rho^q}{q(1-\rho)})]^{-1/2}.

Parameters:

Name	Type	Description	Default
sr	float	Per-period Sharpe ratio.	required
periods_per_year	float	Periods per year q.	required
autocorr	float	First-order autocorrelation rho of per-period returns, in (-1, 1).	None

Returns:

Type	Description
float	Annual Sharpe ratio.

Raises:

Type	Description
ValueError	If periods_per_year is not positive or autocorr is outside (-1, 1).

Examples:

>>> round(annualize_sharpe(1.0, 12), 2)
3.46
>>> round(annualize_sharpe(1.0, 12, autocorr=0.2), 2)
2.88

sharpe_std(sr, n_obs, *, skew=0.0, kurtosis=3.0)

Return the standard error of a per-period Sharpe ratio estimate.

:math:\sigma(\widehat{SR}) = \sqrt{(1 - \gamma_3 SR + \frac{\gamma_4-1}{4} SR^2)/(T-1)}, from Bailey and Lopez de Prado (2012), with non-excess kurtosis.

Parameters:

Name	Type	Description	Default
sr	float	Per-period Sharpe ratio.	required
n_obs	int	Number of observations T.	required
skew	float	Skewness.	0.0
kurtosis	float	Non-excess kurtosis.	3.0

Returns:

Type	Description
float	Standard error, per period.

Raises:

Type	Description
ValueError	If n_obs < 2, kurtosis < 1 or the variance term is not positive.

Examples:

>>> round(sharpe_std(0.0, 101), 4)
0.1

probabilistic_sharpe_ratio(sr, n_obs, *, sr_benchmark=0.0, skew=0.0, kurtosis=3.0)

Return the Probabilistic Sharpe Ratio.

:math:PSR(SR^*) = \Phi\big((SR - SR^*)\sqrt{T-1} / \sqrt{1 - \gamma_3 SR + \frac{\gamma_4-1}{4} SR^2}\big), from Bailey and Lopez de Prado (2012). All Sharpe ratios are per period and kurtosis is non-excess.

Parameters:

Name	Type	Description	Default
sr	float	Per-period Sharpe ratio.	required
n_obs	int	Number of observations T.	required
sr_benchmark	float	Per-period benchmark Sharpe ratio SR*.	0.0
skew	float	Skewness.	0.0
kurtosis	float	Non-excess kurtosis.	3.0

Returns:

Type	Description
float	Probability that the true Sharpe ratio exceeds sr_benchmark.

Raises:

Type	Description
ValueError	If n_obs < 2, kurtosis < 1 or the variance term is not positive.

Examples:

>>> round(probabilistic_sharpe_ratio(0.458, 24), 3)
0.982

psr_from_returns(returns, *, sr_benchmark=0.0)

Return the PSR of a return series, using its own moments.

The statistic is that of Bailey and Lopez de Prado (2012).

Parameters:

Name	Type	Description	Default
returns	array_like	Per-period returns, one-dimensional.	required
sr_benchmark	float	Per-period benchmark Sharpe ratio.	0.0

Returns:

Type	Description
float	Probabilistic Sharpe Ratio.

Examples:

>>> r = [0.02, 0.01, 0.03, -0.01, 0.02, 0.01]
>>> 0.0 < psr_from_returns(r) < 1.0
True

min_track_record_length(sr, *, sr_benchmark=0.0, skew=0.0, kurtosis=3.0, alpha=0.05)

Return the Minimum Track Record Length, in observations.

:math:MinTRL = 1 + (1 - \gamma_3 SR + \frac{\gamma_4-1}{4} SR^2) (z_{1-\alpha} / (SR - SR^*))^2, from Bailey and Lopez de Prado (2012). All Sharpe ratios are per period and kurtosis is non-excess.

Parameters:

Name	Type	Description	Default
sr	float	Per-period Sharpe ratio.	required
sr_benchmark	float	Per-period benchmark Sharpe ratio SR*.	0.0
skew	float	Skewness.	0.0
kurtosis	float	Non-excess kurtosis.	3.0
alpha	float	Significance level.	0.05

Returns:

Type	Description
float	Number of observations, not years.

Raises:

Type	Description
ValueError	If sr <= sr_benchmark, alpha is outside (0, 1), kurtosis < 1 or the variance term is not positive.

Examples:

>>> round(min_track_record_length(2 / 12**0.5, sr_benchmark=1 / 12**0.5) / 12, 2)
3.24

Deflated Sharpe Ratio

Deflated Sharpe Ratio and minimum backtest length.

expected_max_sharpe(n_trials, var_trials, *, mean_trials=0.0)

Return the expected maximum Sharpe ratio over independent trials.

:math:E[\max] \approx \mu + \sqrt{V}\,((1-\gamma)\Phi^{-1}(1-1/N) + \gamma\Phi^{-1}(1-1/(Ne))), from Bailey and Lopez de Prado (2014), with :math:\gamma the Euler-Mascheroni constant. The result has the units of the trial Sharpe ratios: per-period inputs give a per-period result.

Parameters:

Name	Type	Description	Default
n_trials	float	Number of independent trials N, possibly non-integer. N = 1 gives mean_trials.	required
var_trials	float	Variance V of the trial Sharpe ratios.	required
mean_trials	float	Mean of the trial Sharpe ratios.	0.0

Returns:

Type	Description
float	Expected maximum Sharpe ratio.

Raises:

Type	Description
ValueError	If n_trials < 1 or var_trials < 0.

Examples:

>>> round(expected_max_sharpe(100, 0.5 / 250), 4)
0.1132

deflated_sharpe_ratio(sr, n_obs, *, n_trials, var_trials, skew=0.0, kurtosis=3.0)

Return the Deflated Sharpe Ratio.

:math:DSR = PSR(SR_0) with :math:SR_0 from :func:expected_max_sharpe, from Bailey and Lopez de Prado (2014). sr and var_trials are per period; kurtosis is non-excess.

Parameters:

Name	Type	Description	Default
sr	float	Per-period Sharpe ratio of the selected trial.	required
n_obs	int	Number of observations T.	required
n_trials	float	Number of independent trials N, possibly non-integer.	required
var_trials	float	Variance V of the per-period trial Sharpe ratios.	required
skew	float	Skewness of the selected trial.	0.0
kurtosis	float	Non-excess kurtosis of the selected trial.	3.0

Returns:

Type	Description
DSRResult	DSR, threshold sr0 and the score z.

Raises:

Type	Description
ValueError	If an argument is out of range or the variance term is not positive.

Examples:

>>> r = deflated_sharpe_ratio(2.5 / 250**0.5, 1250, n_trials=100, var_trials=0.5 / 250,
...                           skew=-3.0, kurtosis=10.0)
>>> round(r.dsr, 4)
0.9004

dsr_from_returns(returns, *, selected=None, n_trials=None)

Return the DSR of a T x N return matrix, from Bailey and Lopez de Prado (2014).

N defaults to the number of columns and V is the sample variance of the per-period column Sharpe ratios. With a single column, V is the null variance of the estimator, 1 / (T - 1), as in the trial-log contract.

Parameters:

Name	Type	Description	Default
returns	array_like	T x N matrix of per-period returns.	required
selected	int	Column of the selected trial. Defaults to the highest Sharpe ratio.	None
n_trials	float	Number of trials, for example an effective count. Defaults to N.	None

Returns:

Type	Description
DSRResult	DSR of the selected trial.

Raises:

Type	Description
ValueError	If the input has NaN or inf or selected is out of range.

Examples:

>>> rng = np.random.default_rng(0)
>>> r = rng.normal(0.0, 0.01, (500, 20))
>>> 0.0 <= dsr_from_returns(r).dsr <= 1.0
True

min_backtest_length(n_trials, *, max_sharpe_annual=1.0)

Return the Minimum Backtest Length, in years.

:math:MinBTL \approx (((1-\gamma)\Phi^{-1}(1-1/N) + \gamma\Phi^{-1}(1-1/(Ne))) / E[\max])^2, from Bailey et al. (2014, eq. 3.2): the years needed so that the expected maximum annual Sharpe ratio of N skill-less independent trials does not exceed max_sharpe_annual.

Parameters:

Name	Type	Description	Default
n_trials	float	Number of independent trials N.	required
max_sharpe_annual	float	Target expected maximum annual Sharpe ratio.	1.0

Returns:

Type	Description
float	Years of backtest.

Raises:

Type	Description
ValueError	If n_trials < 1 or max_sharpe_annual is not positive.

Examples:

>>> round(min_backtest_length(45), 1)
5.0

Haircut

Harvey-Liu Sharpe ratio haircut.

Follows Harvey and Liu (2015), "Backtesting", and the conventions of their reference code Haircut_SR.m: t-statistics on monthly observations with a Student t distribution for the strategy under test, Bonferroni over n_tests, and Holm and BHY over n_tests + 1 with the other tests simulated from the Harvey-Liu-Zhu model.

sharpe_to_tstat(sr_annual, n_obs, periods_per_year, *, autocorr=0.0)

Convert an annual Sharpe ratio to the t-statistic of Harvey and Liu.

:math:t = SR_{ann}/\sqrt{12}\cdot\sqrt{M}, with M the whole number of months in the sample and :math:SR_{ann} corrected for autocorrelation by Lo (2002).

Parameters:

Name	Type	Description	Default
sr_annual	float	Annual Sharpe ratio.	required
n_obs	int	Number of observations at periods_per_year.	required
periods_per_year	int	Observations per year.	required
autocorr	float	First-order autocorrelation at the sampling frequency.	0.0

Returns:

Type	Description
float	t-statistic.

Raises:

Type	Description
ValueError	If the sample spans fewer than two months or autocorr is out of range.

Examples:

>>> round(sharpe_to_tstat(1.0, 120, 12), 4)
3.1623

haircut_sharpe(sr_annual, n_obs, *, periods_per_year, n_tests, method='bhy', autocorr=0.0, rho_tests=0.2, n_sims=5000, rng=0)

Return the Sharpe ratio left after a multiple-testing haircut.

The annual Sharpe ratio is corrected for autocorrelation, converted to a t-statistic and a two-sided p-value on M - 1 degrees of freedom, adjusted for multiple testing and inverted back to an annual Sharpe ratio, following Harvey and Liu (2015). Bonferroni is exact, min(n_tests * p, 1). Holm and BHY take the median adjusted p-value over n_sims draws of the other tests from the Harvey-Liu-Zhu model with pairwise correlation rho_tests.

A non-positive sr_annual has no evidence to deflate: the p-values are 1, the haircut Sharpe ratio is 0 and the haircut is 100%.

Parameters:

Name	Type	Description	Default
sr_annual	float	Annual Sharpe ratio.	required
n_obs	int	Number of observations at periods_per_year.	required
periods_per_year	int	Observations per year.	required
n_tests	int	Number of tests.	required
method	('bonferroni', 'holm', 'bhy')	Multiple-testing adjustment.	"bonferroni"
autocorr	float	First-order autocorrelation at the sampling frequency.	0.0
rho_tests	float	Average correlation among strategy returns, in [0, 1).	0.2
n_sims	int	Number of simulated sets of other tests.	5000
rng	Generator or int or None	Source of randomness for Holm and BHY.	0

Returns:

Type	Description
HaircutResult	Corrected and haircut Sharpe ratios, t-statistic and p-values.

Raises:

Type	Description
ValueError	If an argument is out of range.

Examples:

>>> r = haircut_sharpe(1.0, 120, periods_per_year=12, n_tests=100, method="bonferroni",
...                    autocorr=0.1, rho_tests=0.4)
>>> round(r.haircut_sr_annual, 3), round(r.adj_pvalue, 3)
(0.232, 0.465)

Cross-validation

Purged k-fold and combinatorial purged cross-validation.

Implements the schemes of Lopez de Prado (2018, chapters 7 and 12). A label observed at row i spans the interval [t0[i], t1[i]], so two rows can share information even when they are not adjacent. The splitters follow the scikit-learn split protocol, so they can be passed as cv= without importing scikit-learn.

PurgedKFold(n_splits=5, *, t1, t0=None, embargo_pct=0.01)

K-fold over contiguous test folds, with purging and an embargo.

Parameters:

Name	Type	Description	Default
n_splits	int	Number of folds, at least 2.	5
t1	array_like	End of each label interval, integers or datetime64.	required
t0	array_like	Start of each label interval, ascending. Defaults to row positions, which needs integer t1.	None
embargo_pct	float	Embargo length as a share of the rows: ceil(embargo_pct * T) rows.	0.01

Raises:

Type	Description
ValueError	If an argument is out of range or the label times are inconsistent.

Examples:

>>> t1 = np.arange(10) + 2
>>> cv = PurgedKFold(2, t1=t1, embargo_pct=0.0)
>>> [train.tolist() for train, _ in cv.split(np.zeros(10))]
[[7, 8, 9], [0, 1, 2]]

get_n_splits(X=None, y=None, groups=None)

Return the number of folds.

Parameters:

Name	Type	Description	Default
X	object	Ignored, for scikit-learn compatibility.	None
y	object	Ignored, for scikit-learn compatibility.	None
groups	object	Ignored, for scikit-learn compatibility.	None

Returns:

Type	Description
int	Number of folds.

split(X, y=None, groups=None)

Yield train and test row numbers for each fold.

Parameters:

Name	Type	Description	Default
X	array_like	Data, used only for its length.	required
y	object	Ignored, for scikit-learn compatibility.	None
groups	object	Ignored, for scikit-learn compatibility.	None

Yields:

Type	Description
train, test : numpy.ndarray	Row numbers.

Raises:

Type	Description
ValueError	If X and t1 differ in length.

CombinatorialPurgedCV(n_groups=6, n_test_groups=2, *, t1, t0=None, embargo_pct=0.01)

Combinatorial purged cross-validation.

Rows are divided into N contiguous groups and every combination of k groups is a test set, so there are C(N, k) splits. Each group is tested k/N * C(N, k) times, which yields that many complete backtest paths.

Parameters:

Name	Type	Description	Default
n_groups	int	Number of groups N.	6
n_test_groups	int	Number of test groups k, below N.	2
t1	array_like	End of each label interval, integers or datetime64.	required
t0	array_like	Start of each label interval, ascending. Defaults to row positions.	None
embargo_pct	float	Embargo length as a share of the rows.	0.01

Raises:

Type	Description
ValueError	If an argument is out of range or the label times are inconsistent.

Examples:

>>> cv = CombinatorialPurgedCV(6, 2, t1=np.arange(60))
>>> cv.n_splits, cv.n_paths
(15, 5)

n_paths property

Number of complete backtest paths, k/N * C(N, k).

n_splits property

Number of splits, C(N, k).

get_n_splits(X=None, y=None, groups=None)

Return the number of splits.

Parameters:

Name	Type	Description	Default
X	object	Ignored, for scikit-learn compatibility.	None
y	object	Ignored, for scikit-learn compatibility.	None
groups	object	Ignored, for scikit-learn compatibility.	None

Returns:

Type	Description
int	Number of splits.

group_rows(group)

Return the row numbers of a group.

Parameters:

Name	Type	Description	Default
group	int	Group id.	required

Returns:

Type	Description
ndarray	Row numbers.

path_map()

Return which split supplies each group of each path.

Returns:

Type	Description
ndarray	Integer array of shape (n_paths, n_groups) holding split ids.

Examples:

>>> CombinatorialPurgedCV(3, 2, t1=np.arange(9)).path_map().tolist()
[[0, 0, 1], [1, 2, 2]]

split(X, y=None, groups=None)

Yield train and test row numbers for each split.

Parameters:

Name	Type	Description	Default
X	array_like	Data, used only for its length.	required
y	object	Ignored, for scikit-learn compatibility.	None
groups	object	Ignored, for scikit-learn compatibility.	None

Yields:

Type	Description
train, test : numpy.ndarray	Row numbers.

Raises:

Type	Description
ValueError	If X and t1 differ in length.

test_groups(split_id)

Return the test groups of a split.

Parameters:

Name	Type	Description	Default
split_id	int	Split id.	required

Returns:

Type	Description
tuple of int	Group ids.

assemble_paths(cv, oos_predictions)

Assemble per-split out-of-sample predictions into complete backtest paths.

The paths are those of combinatorial purged CV in Lopez de Prado (2018).

Parameters:

Name	Type	Description	Default
cv	CombinatorialPurgedCV	The splitter that produced the predictions.	required
oos_predictions	sequence of array_like	One array per split, aligned with that split's test rows.	required

Returns:

Type	Description
ndarray	Array of shape (n_paths, T).

Raises:

Type	Description
ValueError	If the number of arrays or the length of one is wrong.

Examples:

>>> cv = CombinatorialPurgedCV(3, 2, t1=np.arange(3), embargo_pct=0.0)
>>> preds = [test.astype(float) * 10 for _, test in cv.split(np.zeros(3))]
>>> assemble_paths(cv, preds).tolist()
[[0.0, 10.0, 20.0], [0.0, 10.0, 20.0]]

PBO

Probability of Backtest Overfitting by combinatorially symmetric cross-validation.

Implements CSCV of Bailey, Borwein, Lopez de Prado and Zhu (2017). Rows are cut into S contiguous blocks. Each way of choosing S/2 blocks is an in-sample set and its complement is the out-of-sample set. PBO is how often the in-sample winner lands in the lower half of the out-of-sample ranking.

PBOResult(pbo, logits, oos_rank, is_perf, oos_perf, degradation_slope, degradation_intercept, prob_oos_loss, n_combinations) dataclass

Outcome of CSCV.

Attributes:

Name	Type	Description
pbo	float	Share of combinations whose logit is at or below zero.
logits	ndarray	ln(w / (1 - w)) per combination.
oos_rank	ndarray	Relative out-of-sample rank w of the in-sample winner.
is_perf	ndarray	In-sample performance of the winner.
oos_perf	ndarray	Its out-of-sample performance.
degradation_slope	float	Slope of out-of-sample on in-sample performance.
degradation_intercept	float	Intercept of that regression.
prob_oos_loss	float	Share of combinations in which the winner loses out of sample.
n_combinations	int	C(S, S/2).

pbo_cscv(returns, *, n_partitions=16, metric='sharpe')

Return the Probability of Backtest Overfitting.

For each of the :math:C(S, S/2) combinations of blocks, the trial with the best in-sample metric has a relative out-of-sample rank :math:\omega = r / (N + 1) and a logit :math:\lambda = \ln(\omega / (1 - \omega)). PBO is the share of :math:\lambda \le 0, from Bailey et al. (2017). Rank ties take the average rank. Rows beyond the last whole block are dropped, as in the paper.

Parameters:

Name	Type	Description	Default
returns	array_like	T x N matrix of per-period returns.	required
n_partitions	int	Number of blocks S, even.	16
metric	('sharpe', 'mean')	A callable maps a T' x N matrix to N values, larger meaning better.	"sharpe"

Returns:

Type	Description
PBOResult	PBO, logits, performance degradation and probability of loss.

Raises:

Type	Description
ValueError	If n_partitions is odd or too small, there are fewer than two trials, there are too few rows, or metric is unknown.

Examples:

>>> r = np.random.default_rng(0).normal(0.0, 0.01, (320, 50))
>>> result = pbo_cscv(r, n_partitions=8)
>>> result.n_combinations
70
>>> 0.0 <= result.pbo <= 1.0
True

Bootstrap

Stationary bootstrap indices and automatic block length.

Indices are generated with a NumPy Generator, so results are the same on every OS, and are passed into the tests as plain arrays.

optimal_block_length(x)

Return the stationary bootstrap block length of Politis and White (2004).

Uses the flat-top lag window and the correction of Patton, Politis and White (2009): :math:b = (2\hat{G}^2/\hat{D})^{1/3} n^{1/3}, with :math:\hat{D} = 2\hat{\sigma}^4. The result is capped at the sample's upper bound :math:\min(3\sqrt{n}, n/3) and floored at 1.

Parameters:

Name	Type	Description	Default
x	array_like	One-dimensional series.	required

Returns:

Type	Description
float	Mean block length.

Raises:

Type	Description
ValueError	If there are fewer than nine observations.

Examples:

>>> rng = np.random.default_rng(0)
>>> round(optimal_block_length(rng.standard_normal(1000)), 1) < 3.0
True

stationary_bootstrap_indices(n_obs, n_boot, *, mean_block, rng=None)

Return indices of the stationary bootstrap of Politis and Romano (1994).

Each position starts a new block with probability 1 / mean_block and otherwise continues the previous block, wrapping around the end of the sample. Block lengths are geometric with mean mean_block.

Parameters:

Name	Type	Description	Default
n_obs	int	Observations in the sample and in each resample.	required
n_boot	int	Number of resamples.	required
mean_block	float	Mean block length, at least 1.	required
rng	Generator or int or None	Source of randomness.	None

Returns:

Type	Description
ndarray	Integer array of shape (n_boot, n_obs).

Raises:

Type	Description
ValueError	If an argument is out of range.

Examples:

>>> stationary_bootstrap_indices(5, 2, mean_block=2.0, rng=0).shape
(2, 5)

Data-snooping tests

Data-snooping tests: White's Reality Check, Hansen's SPA and the Romano-Wolf stepdown.

rel_perf is a T x N matrix of per-period performance relative to the benchmark, with larger meaning better. Bootstrap indices are drawn in NumPy and passed to a Numba kernel.

SnoopResult(stat, pvalue, n_boot, block_length) dataclass

Reality Check outcome.

Attributes:

Name	Type	Description
stat	float	max_k sqrt(T) * mean(d_k).
pvalue	float	Bootstrap p-value.
n_boot	int	Resamples.
block_length	float	Mean block length, NaN when indices were supplied without one.

SPAResult(stat, pvalue_lower, pvalue_consistent, pvalue_upper, n_boot, block_length) dataclass

Superior Predictive Ability outcome.

Attributes:

Name	Type	Description
stat	float	Studentised statistic, at least 0.
pvalue_lower	float	Lower-bound p-value.
pvalue_consistent	float	Consistent p-value.
pvalue_upper	float	Upper-bound p-value.
n_boot	int	Resamples.
block_length	float	Mean block length, NaN when indices were supplied without one.

StepMResult(rejected, adjusted_pvalues, n_steps, alpha) dataclass

Stepdown outcome.

Attributes:

Name	Type	Description
rejected	ndarray	N flags, True for strategies better than the benchmark.
adjusted_pvalues	ndarray	Adjusted p-value of each strategy.
n_steps	int	Steps taken, including the last one that rejected nothing.
alpha	float	Familywise error rate.

reality_check(rel_perf, *, n_boot=1000, block_length=None, indices=None, rng=None)

Return White's (2000) Reality Check.

The statistic is :math:\max_k \sqrt{T}\,\bar{d}_k, and the p-value is the share of resamples whose recentred maximum :math:\max_k \sqrt{T}(\bar{d}^*_k - \bar{d}_k) reaches it.

Parameters:

Name	Type	Description	Default
rel_perf	array_like	T x N per-period performance relative to the benchmark, larger meaning better.	required
n_boot	int	Resamples.	1000
block_length	float	Mean block length. Defaults to the median of Politis and White over the columns.	None
indices	ndarray	Bootstrap indices of shape (n_boot, T), which replace n_boot and rng.	None
rng	Generator or int or None	Source of randomness.	None

Returns:

Type	Description
SnoopResult	Statistic and p-value.

Raises:

Type	Description
ValueError	If an argument is out of range.

Examples:

>>> d = np.random.default_rng(0).normal(0.0, 0.01, (250, 5))
>>> 0.0 <= reality_check(d, n_boot=200, block_length=2.0, rng=0).pvalue <= 1.0
True

spa_test(rel_perf, *, n_boot=1000, block_length=None, indices=None, rng=None, studentize=True)

Return Hansen's (2005) test for Superior Predictive Ability.

The statistic is :math:\max(\max_k \sqrt{T}\,\bar{d}_k / \hat{\omega}_k, 0), with :math:\hat{\omega}_k^2 the long-run variance under the stationary bootstrap's kernel. Resamples are recentred by :math:\max(\bar{d}_k, 0) for the lower p-value, by :math:\bar{d}_k for the upper one, and for the consistent one by :math:\bar{d}_k unless it falls below :math:-\hat{\omega}_k\sqrt{2\ln\ln T / T}, when it is zero.

Parameters:

Name	Type	Description	Default
rel_perf	array_like	T x N per-period performance relative to the benchmark, larger meaning better.	required
n_boot	int	Resamples.	1000
block_length	float	Mean block length. Defaults to the median of Politis and White over the columns.	None
indices	ndarray	Bootstrap indices of shape (n_boot, T). Then block_length should be the mean block used for them, or the variance kernel assumes the default.	None
rng	Generator or int or None	Source of randomness.	None
studentize	bool	Hansen's statistic. False compares raw means with no floor at zero, which is the definition of the arch package, used by the cross-check.	True

Returns:

Type	Description
SPAResult	Statistic and the three p-values.

Raises:

Type	Description
ValueError	If an argument is out of range.

Examples:

>>> d = np.random.default_rng(0).normal(0.0, 0.01, (250, 5))
>>> result = spa_test(d, n_boot=200, block_length=2.0, rng=0)
>>> result.pvalue_lower <= result.pvalue_consistent <= result.pvalue_upper
True

romano_wolf_stepdown(rel_perf, *, alpha=0.05, n_boot=1000, block_length=None, indices=None, rng=None)

Return the studentised stepdown of Romano and Wolf (2005).

Strategies are sorted by :math:t_k = \sqrt{T}\,\bar{d}_k / \hat{\omega}_k. Each step rejects those whose :math:t_k beats the bootstrap distribution of the maximum over the strategies not yet rejected, and repeats until a step rejects nothing, which controls the familywise error rate at alpha. Adjusted p-values follow Romano and Wolf (2016).

Parameters:

Name	Type	Description	Default
rel_perf	array_like	T x N per-period performance relative to the benchmark, larger meaning better.	required
alpha	float	Familywise error rate.	0.05
n_boot	int	Resamples.	1000
block_length	float	Mean block length. Defaults to the median of Politis and White over the columns.	None
indices	ndarray	Bootstrap indices of shape (n_boot, T).	None
rng	Generator or int or None	Source of randomness.	None

Returns:

Type	Description
StepMResult	Rejected set and adjusted p-values.

Raises:

Type	Description
ValueError	If an argument is out of range.

Examples:

>>> d = np.random.default_rng(0).normal(0.0, 0.01, (500, 5))
>>> d[:, 0] += 0.01
>>> romano_wolf_stepdown(d, n_boot=200, block_length=2.0, rng=0).rejected.tolist()
[True, False, False, False, False]

Synthetic data

Synthetic data with known ground truth.

SynthResult(returns, is_true, true_sharpe_annual) dataclass

Return matrix with its ground truth.

Attributes:

Name	Type	Description
returns	ndarray	T x N per-period returns.
is_true	ndarray	N flags, True for strategies with a planted signal.
true_sharpe_annual	ndarray	True annual Sharpe ratio of each strategy.

gaussian_noise(n_obs, n_series=1, *, vol_annual=0.16, periods_per_year=252, rng=None)

Return zero-mean Gaussian returns.

Parameters:

Name	Type	Description	Default
n_obs	int	Observations T.	required
n_series	int	Series N.	1
vol_annual	float	Annual volatility.	0.16
periods_per_year	int	Periods per year.	252
rng	Generator or int or None	Source of randomness.	None

Returns:

Type	Description
ndarray	T x N per-period returns.

Raises:

Type	Description
ValueError	If a size is below 1.

Examples:

>>> gaussian_noise(100, 3, rng=0).shape
(100, 3)

student_t_noise(n_obs, n_series=1, *, df=4.0, vol_annual=0.16, periods_per_year=252, rng=None)

Return zero-mean Student t returns, scaled to the chosen volatility.

Parameters:

Name	Type	Description	Default
n_obs	int	Observations T.	required
n_series	int	Series N.	1
df	float	Degrees of freedom, above 2 so that the variance exists.	4.0
vol_annual	float	Annual volatility.	0.16
periods_per_year	int	Periods per year.	252
rng	Generator or int or None	Source of randomness.	None

Returns:

Type	Description
ndarray	T x N per-period returns.

Raises:

Type	Description
ValueError	If df <= 2 or a size is below 1.

Examples:

>>> student_t_noise(100, 2, rng=0).shape
(100, 2)

gbm_prices(n_obs, *, mu_annual=0.0, vol_annual=0.16, s0=100.0, periods_per_year=252, rng=None)

Return a geometric Brownian motion price path.

Parameters:

Name	Type	Description	Default
n_obs	int	Number of prices, starting with s0.	required
mu_annual	float	Annual drift.	0.0
vol_annual	float	Annual volatility.	0.16
s0	float	First price.	100.0
periods_per_year	int	Periods per year.	252
rng	Generator or int or None	Source of randomness.	None

Returns:

Type	Description
ndarray	n_obs prices.

Raises:

Type	Description
ValueError	If n_obs is below 1 or s0 is not positive.

Examples:

>>> float(gbm_prices(4, rng=0)[0])
100.0

planted_signal_matrix(n_obs, n_strategies, *, true_sharpe_annual, n_true=1, corr=0.0, vol_annual=0.16, periods_per_year=252, rng=None)

Return a strategy matrix in which the first n_true columns have an edge.

Parameters:

Name	Type	Description	Default
n_obs	int	Observations T.	required
n_strategies	int	Strategies N.	required
true_sharpe_annual	float	True annual Sharpe ratio of the planted strategies.	required
n_true	int	Planted strategies.	1
corr	float	Pairwise correlation of the shocks, in [0, 1).	0.0
vol_annual	float	Annual volatility.	0.16
periods_per_year	int	Periods per year.	252
rng	Generator or int or None	Source of randomness.	None

Returns:

Type	Description
SynthResult	Returns, truth flags and true annual Sharpe ratios.

Raises:

Type	Description
ValueError	If an argument is out of range.

Examples:

>>> synth = planted_signal_matrix(100, 5, true_sharpe_annual=1.0, n_true=2, rng=0)
>>> synth.is_true.tolist()
[True, True, False, False, False]

planted_signal_prices(n_obs, *, true_sharpe_annual, kind='trend', periods_per_year=252, rng=None)

Return prices with a signal that a named oracle strategy can harvest.

trend adds an AR(1) component with persistence 0.98 to the returns. mean_reversion adds a stationary AR(1) deviation with persistence 0.90 to the log price. The amplitude is calibrated on a long seeded path, so that the oracle's Sharpe ratio before costs equals true_sharpe_annual. Calibration runs once per target and is then cached.

Parameters:

Name	Type	Description	Default
n_obs	int	Number of prices.	required
true_sharpe_annual	float	True annual Sharpe ratio of the oracle, at least 0.	required
kind	('trend', 'mean_reversion')	Type of signal.	"trend"
periods_per_year	int	Periods per year.	252
rng	Generator or int or None	Source of randomness of the returned path.	None

Returns:

Name	Type	Description
prices	ndarray	n_obs prices.
oracle	str	Identifier of the oracle strategy, from StrategySpec.spec_id.

Raises:

Type	Description
ValueError	If an argument is out of range or the target cannot be reached.

Examples:

>>> prices, oracle = planted_signal_prices(50, true_sharpe_annual=0.0, rng=0)
>>> prices.size, oracle.split(":")[0]
(50, 'ma_cross')

block_bootstrap_returns(returns, n_obs, *, mean_block=None, demean=True, rng=None)

Return a stationary bootstrap of a real return series.

Parameters:

Name	Type	Description	Default
returns	array_like	One-dimensional returns.	required
n_obs	int	Length of the resampled series.	required
mean_block	float	Mean block length. Defaults to :func:~overfitlab.bootstrap.optimal_block_length.	None
demean	bool	Remove the sample mean first, which imposes the null of no edge.	True
rng	Generator or int or None	Source of randomness.	None

Returns:

Type	Description
ndarray	n_obs resampled returns.

Raises:

Type	Description
ValueError	If n_obs is below 1 or the input has NaN or inf.

Examples:

>>> block_bootstrap_returns([0.01, -0.02, 0.03, 0.0], 6, mean_block=2.0, rng=0).shape
(6,)

Strategy factory

Strategy factory: parameter variants of three rule families, on Numba kernels.

A position computed from data up to close t earns the return from close t to close t + 1. All positions are -1, 0 or +1. Long-only variants clip the long-short position at zero.

StrategySpec(family, params) dataclass

One parameter variant of a rule family.

Attributes:

Name	Type	Description
family	{'ma_cross', 'breakout', 'mean_reversion'}	Rule family.
params	mapping	Parameters of the family, including mode.

params_json()

Return the canonical JSON of the parameters, as in the trial log.

Returns:

Type	Description
str	Canonical JSON.

Examples:

>>> StrategySpec("ma_cross", {"slow": 60, "fast": 5}).params_json()
'{"fast":5,"slow":60}'

spec_id()

Return a readable identifier, family:params_json.

Returns:

Type	Description
str	Identifier.

strategy_grid(config=None)

Return the variants of a grid, in a stable order.

Parameters:

Name	Type	Description	Default
config	mapping	Family to parameter to values. Defaults to the 1,034-variant grid: 288 moving-average crossovers, 200 breakouts and 546 mean-reversion rules.	None

Returns:

Type	Description
list of StrategySpec	One spec per combination of parameter values.

Raises:

Type	Description
ValueError	If a family is unknown.

Examples:

>>> len(strategy_grid())
1034

run_strategies(close, specs, *, cost_bps=1.0)

Return the net returns of each variant on one price path.

The position after close t - 1 earns the simple return from close t - 1 to close t, less :math:c\,|\Delta position| for the trade made at close t - 1.

Parameters:

Name	Type	Description	Default
close	array_like	Positive closing prices, T + 1 values.	required
specs	sequence of StrategySpec	Variants to run.	required
cost_bps	float	Cost in basis points per unit of turnover.	1.0

Returns:

Type	Description
ndarray	T x N per-period net returns.

Raises:

Type	Description
ValueError	If prices are not positive, cost_bps is negative or there are under two prices.

Examples:

>>> spec = StrategySpec("ma_cross", {"fast": 1, "slow": 2, "mode": "long_short"})
>>> run_strategies([1.0, 2.0, 4.0], [spec], cost_bps=0.0)[:, 0].tolist()
[0.0, 1.0]

Verdict gate

Verdict gate: combine the evidence into PASS, FAIL or INCONCLUSIVE.

VerdictConfig(dsr_min=0.95, pbo_max=0.2, snoop_alpha=0.05, haircut_method='bhy', pbo_partitions=16, n_boot=1000, periods_per_year=252) dataclass

Thresholds of the verdict gate, with the defaults of decision D-06.

Attributes:

Name	Type	Description
dsr_min	float	Minimum Deflated Sharpe Ratio.
pbo_max	float	Maximum Probability of Backtest Overfitting.
snoop_alpha	float	Significance level of the SPA and stepdown tests.
haircut_method	{'bonferroni', 'holm', 'bhy'}	Multiple-testing adjustment of the haircut.
pbo_partitions	int	Number of CSCV partitions S.
n_boot	int	Number of bootstrap resamples.
periods_per_year	int	Periods per year of the returns.

VerdictReport(verdict, reasons, dsr, pbo, spa, stepm, haircut, selected) dataclass

Outcome of the gate.

Attributes:

Name	Type	Description
verdict	{'PASS', 'FAIL', 'INCONCLUSIVE'}	Overall result.
reasons	mapping	Criterion to "pass", "fail: ..." or "not computed: ...".
dsr	DSRResult	Deflated Sharpe Ratio.
pbo	PBOResult or None	PBO, if the sample allows it.
spa	SPAResult or None	SPA test, if the sample allows it.
stepm	StepMResult or None	Stepdown, if the sample allows it.
haircut	HaircutResult	Harvey-Liu haircut.
selected	int	Column of the selected trial.

to_json()

Return the report as one JSON object, with NaN as null.

Returns:

Type	Description
str	JSON text.

evaluate(returns, *, selected=None, n_trials=None, benchmark=None, config=VerdictConfig(), rng=0)

Judge the selected trial of a T x N return matrix.

PASS needs DSR at or above dsr_min, PBO at or below pbo_max, a consistent SPA p-value at or below snoop_alpha, the stepdown rejecting the selected trial and a positive haircut Sharpe ratio. FAIL means a computed criterion failed. INCONCLUSIVE means a criterion could not be computed and none failed.

Parameters:

Name	Type	Description	Default
returns	array_like	T x N per-period returns.	required
selected	int	Column of the selected trial. Defaults to the highest Sharpe ratio.	None
n_trials	float	Trials for the DSR and the haircut. Defaults to N.	None
benchmark	array_like	T benchmark returns. Defaults to zero.	None
config	VerdictConfig	Thresholds.	VerdictConfig()
rng	Generator or int or None	Source of randomness.	0

Returns:

Type	Description
VerdictReport	Verdict, reasons and evidence.

Raises:

Type	Description
ValueError	If an argument is out of range.

Examples:

>>> r = np.random.default_rng(0).normal(0.0, 0.01, (400, 20))
>>> evaluate(r, config=VerdictConfig(n_boot=100)).verdict
'FAIL'

Trial log

Cross-project trial-log contract v1.0: writer, reader, validator and evaluator.

The contract is published in docs/trial-log.md and described for machines in trial_log_schema.json. It is frozen: later changes only add optional x_ columns.

TRIAL_LOG_COLUMNS = ('run_id', 'timestamp', 'git_sha', 'strategy_family', 'params_json', 'start', 'end', 'n_obs', 'sharpe_annual', 'returns_path') module-attribute

TRIAL_LOG_VERSION = '1.0' module-attribute

TrialLogger(log_path='results/trial_log.csv', returns_dir='results/runs', *, repo_dir='.')

Append trials to a contract v1.0 log.

Call :meth:log inside the shared evaluation function, so that no evaluation can skip it. New logs get the columns x_periods_per_year and x_note plus any extra keys of the first row. An existing log keeps its header, because the file is append-only.

Parameters:

Name	Type	Description	Default
log_path	str or Path	Log file.	"results/trial_log.csv"
returns_dir	str or Path	Folder of the Parquet return series. Must be runs next to the log.	"results/runs"
repo_dir	str or Path	Git repository whose commit is recorded.	"."

Raises:

Type	Description
ValueError	If returns_dir is not the folder runs next to log_path.

Examples:

>>> import tempfile
>>> folder = Path(tempfile.mkdtemp())
>>> trials = TrialLogger(folder / "trial_log.csv", folder / "runs", repo_dir=folder)
>>> run_id = trials.log(strategy_family="demo", params={"n": 5},
...                     dates=["2024-01-02", "2024-01-03", "2024-01-04"],
...                     returns=[0.01, -0.02, 0.03])
>>> len(run_id)
36

log(*, strategy_family, params, dates, returns, periods_per_year=252, extra=None)

Write the return series and append one row.

Parameters:

Name	Type	Description	Default
strategy_family	str	Group of related trials.	required
params	mapping	All choices that define the trial.	required
dates	array_like	Return dates, strictly increasing.	required
returns	array_like	Net simple returns per period, in decimal.	required
periods_per_year	int	Periods per year, written as x_periods_per_year.	252
extra	mapping	More x_ columns. They must exist in the header of an existing log.	None

Returns:

Type	Description
str	The run_id.

Raises:

Type	Description
ValueError	If an argument breaks the contract.

log_failed(*, strategy_family, params, note)

Append a failed or aborted trial, which still counts in N.

Parameters:

Name	Type	Description	Default
strategy_family	str	Group of related trials.	required
params	mapping	All choices that define the trial.	required
note	str	Why it failed, written as x_note.	required

Returns:

Type	Description
str	The run_id.

Raises:

Type	Description
ValueError	If strategy_family breaks the contract.

TrialLog(rows, base_dir)

A loaded trial log, or a filtered part of one.

Parameters:

Name	Type	Description	Default
rows	sequence of mapping	Rows as strings, keyed by column name.	required
base_dir	Path	Folder of the CSV, which returns_path is relative to.	required

Attributes:

Name	Type	Description
rows	tuple of mapping	Rows as strings, keyed by column name.
base_dir	Path	Folder of the CSV.

Examples:

>>> log = TrialLog([], Path("."))
>>> log.n_trials()
0.0

filter(*, families=None, exclude_failed=False)

Return the rows of some families, optionally without failed rows.

Parameters:

Name	Type	Description	Default
families	sequence of str	Families to keep. Defaults to all.	None
exclude_failed	bool	Drop rows with n_obs = 0.	False

Returns:

Type	Description
TrialLog	The selected rows.

n_trials(*, mode='raw')

Return the number of trials in scope.

Parameters:

Name	Type	Description	Default
mode	('raw', 'dedup', 'clustered')	raw counts rows, including failed and manual ones. dedup counts unique family, params, window and commit. clustered estimates effective independent trials.	"raw"

Returns:

Type	Description
float	Number of trials.

Raises:

Type	Description
ValueError	If mode is unknown.

periods_per_year()

Return the q shared by the valid rows in scope.

Returns:

Type	Description
int	Periods per year.

Raises:

Type	Description
TrialLogError	With E_FREQ_MIX if the rows differ or q cannot be inferred.

returns_matrix()

Return the aligned return matrix: an inner join of all series on date.

Returns:

Name	Type	Description
dates	ndarray	Common dates.
matrix	ndarray	T x N returns.
run_ids	tuple of str	Trial of each column.

selected_index(run_id=None)

Return the position in rows of the selected trial.

The order is run_id, then a row with x_selected = true, then the highest sharpe_annual. Only rows with a track record can be selected by default.

Parameters:

Name	Type	Description	Default
run_id	str	Trial to select.	None

Returns:

Type	Description
int	Index into rows.

Raises:

Type	Description
ValueError	If run_id is not in scope or no row can be selected.

trial_sharpes(*, source='auto')

Return the per-period Sharpe ratio of every valid trial in scope.

Parameters:

Name	Type	Description	Default
source	('auto', 'returns', 'summary')	Use the Parquet series, sharpe_annual, or the series when any exist.	"auto"

Returns:

Type	Description
TrialSharpes	Run ids, Sharpe ratios, q and the source used.

Raises:

Type	Description
TrialLogError	If the rows in scope do not share one q.

variance_of_sharpes(*, source='auto')

Return the sample variance of the per-period trial Sharpe ratios.

With fewer than two valid trials it falls back to the null variance of the estimator, 1 / (T - 1), with T the mean n_obs.

Parameters:

Name	Type	Description	Default
source	('auto', 'returns', 'summary')	Source of the Sharpe ratios.	"auto"

Returns:

Type	Description
float	Variance, in per-period units.

TrialLogError(message, issues=())

Bases: ValueError

A trial log failed validation or cannot be evaluated.

Attributes:

Name	Type	Description
issues	list of TrialLogIssue	The error-level issues.

TrialLogIssue(code, severity, row, message) dataclass

One finding of the validator.

Attributes:

Name	Type	Description
code	str	E_* for errors and W_* for warnings.
severity	{'error', 'warning'}	Severity.
row	int or None	One-based data row, or None for the whole file.
message	str	Explanation.

TrialLogReport(families, n_trials, n_trials_mode, n_valid, variance, variance_annual, variance_source, variance_fallback, periods_per_year, selected_run_id, sr_annual, sr, sr0, sr0_annual, dsr, z, n_obs, skew, kurtosis, moments_assumed, haircut_sr_annual, haircut_pct, pbo, spa_pvalue, stepm_rejects_selected, verdict, reasons=dict()) dataclass

Evaluation of one scope of a trial log.

Attributes:

Name	Type	Description
families	tuple of str	Families in scope.
n_trials	float	N used in the DSR.
n_trials_mode	str	How N was counted.
n_valid	int	Trials with a track record, M.
variance	float	Variance of per-period trial Sharpe ratios.
variance_annual	float	variance * q.
variance_source	str	"returns" or "summary".
variance_fallback	bool	True when fewer than two valid trials forced the null variance.
periods_per_year	int	q of the scope.
selected_run_id	str	Selected trial.
sr_annual	float	Naive annual Sharpe ratio of the selected trial.
sr	float	Its per-period Sharpe ratio.
sr0	float	Per-period threshold.
sr0_annual	float	Annual threshold.
dsr	float	Deflated Sharpe Ratio.
z	float	Score of the DSR.
n_obs	int	Observations of the selected trial.
skew	float	Its skewness.
kurtosis	float	Its non-excess kurtosis.
moments_assumed	bool	True in summary-only mode, where skew 0 and kurtosis 3 are assumed.
haircut_sr_annual	float or None	Harvey-Liu haircut Sharpe ratio, if the sample spans two months.
haircut_pct	float or None	Haircut in percent.
pbo	float or None	Probability of Backtest Overfitting, if computed.
spa_pvalue	float or None	Consistent SPA p-value, if computed.
stepm_rejects_selected	bool or None	Whether the stepdown rejects the selected trial, if computed.
verdict	str	PASS, FAIL or INCONCLUSIVE.
reasons	mapping	Outcome per criterion.

to_dict()

Return the fields as a JSON-ready dictionary.

to_json()

Return the report as one JSON object.

TrialSharpes(run_ids, sr, periods_per_year, source) dataclass

Per-period Sharpe ratios of the valid trials in a scope.

Attributes:

Name	Type	Description
run_ids	tuple of str	Trials, in log order.
sr	ndarray	Per-period Sharpe ratios.
periods_per_year	int	Periods per year shared by the scope.
source	{'returns', 'summary'}	Whether they come from the Parquet series or from sharpe_annual.

add_manual_trial(log_path, *, strategy_family, params, note, start='', end='', sharpe_annual=None, repo_dir='.')

Append a hand iteration that left no saved return series.

The row counts in N and is excluded from the variance of Sharpe ratios.

Parameters:

Name	Type	Description	Default
log_path	str or Path	Log file.	required
strategy_family	str	Group of related trials.	required
params	mapping	All choices that define the trial.	required
note	str	What was tried, written as x_note.	required
start	str	First and last dates, YYYY-MM-DD, if known.	""
end	str	First and last dates, YYYY-MM-DD, if known.	""
sharpe_annual	float	Sharpe ratio seen at the time, if known.	None
repo_dir	str or Path	Git repository whose commit is recorded.	"."

Returns:

Type	Description
str	The run_id.

Raises:

Type	Description
ValueError	If an argument breaks the contract.

Examples:

>>> import tempfile
>>> log = Path(tempfile.mkdtemp()) / "trial_log.csv"
>>> run_id = add_manual_trial(log, strategy_family="demo", params={"idea": 1},
...                           note="tried by hand", repo_dir=log.parent)
>>> read_trial_log(log).n_trials()
1.0

canonical_params_json(params)

Return the canonical JSON text of a parameter mapping.

Parameters:

Name	Type	Description	Default
params	mapping	Choices that define the trial. JSON types only.	required

Returns:

Type	Description
str	Text with sorted keys, no spaces, ASCII only and no NaN.

Raises:

Type	Description
ValueError	If a value is NaN or infinite.
TypeError	If a value is not a JSON type.

Examples:

>>> canonical_params_json({"b": 1, "a": "x"})
'{"a":"x","b":1}'

evaluate_trial_log(path, *, families=None, selected=None, n_effective='raw', exclude_failed=False, config=VerdictConfig())

Evaluate a trial log: N, V, SR0, DSR, PBO, snooping tests, haircut and verdict.

Parameters:

Name	Type	Description	Default
path	str or Path	Log file.	required
families	sequence of str	Scope. Defaults to all families, which must share one q.	None
selected	str	run_id of the selected trial.	None
n_effective	('raw', 'clustered')	How to count N.	"raw"
exclude_failed	bool	Leave failed and manual rows out of N.	False
config	VerdictConfig	Haircut method and gate thresholds.	VerdictConfig()

Returns:

Type	Description
TrialLogReport	The evaluation of the scope.

Raises:

Type	Description
TrialLogError	If validation fails or the rows in scope do not share one q.
ValueError	If no trial in scope has a track record.

Examples:

>>> import tempfile
>>> folder = Path(tempfile.mkdtemp())
>>> trials = TrialLogger(folder / "trial_log.csv", folder / "runs", repo_dir=folder)
>>> rng = np.random.default_rng(0)
>>> days = np.arange("2024-01-01", "2024-07-01", dtype="datetime64[D]")
>>> for n in (1, 2):
...     _ = trials.log(strategy_family="demo", params={"n": n}, dates=days,
...                    returns=rng.normal(0.0, 0.01, days.size), periods_per_year=365)
>>> report = evaluate_trial_log(folder / "trial_log.csv")
>>> report.n_trials, report.periods_per_year
(2.0, 365)

read_trial_log(path, *, validate=True, strict=True)

Read a trial log.

Parameters:

Name	Type	Description	Default
path	str or Path	Log file.	required
validate	bool	Run :func:validate_trial_log first.	True
strict	bool	Raise on error-level issues. Otherwise they are logged and reading goes on.	True

Returns:

Type	Description
TrialLog	All rows.

Raises:

Type	Description
TrialLogError	If validate and strict are set and the log has errors.

Examples:

>>> import tempfile
>>> folder = Path(tempfile.mkdtemp())
>>> trials = TrialLogger(folder / "trial_log.csv", folder / "runs", repo_dir=folder)
>>> _ = trials.log_failed(strategy_family="demo", params={}, note="crashed")
>>> read_trial_log(folder / "trial_log.csv").n_trials()
1.0

validate_trial_log(path)

Check a trial log against contract v1.0.

A log with no Parquet file at all, such as a fresh clone, is in summary-only mode: the CSV is checked and W_SUMMARY_ONLY says the series were not. If some series exist, a missing one is the error E_RETURNS_MISSING.

Parameters:

Name	Type	Description	Default
path	str or Path	Log file.	required

Returns:

Type	Description
list of TrialLogIssue	Errors and warnings, empty for a clean log.

Examples:

>>> import tempfile
>>> log = Path(tempfile.mkdtemp()) / "trial_log.csv"
>>> _ = log.write_text("run_id\n", encoding="utf-8")
>>> [issue.code for issue in validate_trial_log(log)]
['E_COLUMNS']

Results

Result dataclasses returned by the public API.

ReturnMoments(sr, skew, kurtosis, n_obs) dataclass

Moments of a return series.

Attributes:

Name	Type	Description
sr	float	Per-period Sharpe ratio with ddof=1.
skew	float	Sample skewness (gamma_3).
kurtosis	float	Non-excess kurtosis (gamma_4; normal = 3).
n_obs	int	Number of observations.

DSRResult(dsr, sr, sr0, z, n_trials, var_trials, n_obs, skew, kurtosis) dataclass

Deflated Sharpe Ratio and its inputs. All Sharpe quantities are per period.

Attributes:

Name	Type	Description
dsr	float	Deflated Sharpe Ratio, a probability.
sr	float	Sharpe ratio of the selected trial.
sr0	float	Rejection threshold, the expected maximum Sharpe under the null.
z	float	Standard normal score whose CDF is dsr.
n_trials	float	Number of trials, possibly effective and non-integer.
var_trials	float	Variance of the trial Sharpe ratios.
n_obs	int	Number of observations of the selected trial.
skew	float	Skewness of the selected trial.
kurtosis	float	Non-excess kurtosis of the selected trial.

to_dict()

Return the fields as a plain dictionary.

HaircutResult(method, sr_annual, haircut_sr_annual, haircut_pct, tstat, pvalue, adj_pvalue) dataclass

Harvey-Liu haircut of an annual Sharpe ratio.

Attributes:

Name	Type	Description
method	str	"bonferroni", "holm" or "bhy".
sr_annual	float	Annual Sharpe ratio after the autocorrelation correction.
haircut_sr_annual	float	Annual Sharpe ratio implied by the adjusted p-value.
haircut_pct	float	Haircut as a percentage of sr_annual.
tstat	float	t-statistic of sr_annual.
pvalue	float	Two-sided single-test p-value.
adj_pvalue	float	Multiple-testing adjusted p-value.

Constants

overfitlab.EULER_GAMMA is the Euler–Mascheroni constant, used by the expected maximum. overfitlab.__version__ is the installed version.