run_ptl_job

Overview

run_ptl_job scrambles the price history many times, replays the strategy on each scrambled version, and reports where the real result falls in that distribution. A high p-value means the observed performance is not distinguishable from chance. Two modes, chosen by mode (default "auto"):

Mode	What it needs	What you get
`curve`	An equity-curve chart from `run_plot_job`	Bootstrap return and drawdown bounds
`permutation`	A weight-dynamics chart from `run_plot_job` or `run_risk_job`	The full scrambler ladder with p-values
`auto`	—	Permutation when weights are available, otherwise curve

In permutation mode the service downloads the prices it needs itself; the tickers come from the weights chart. Runs scale with n_permutations and can take tens of minutes.

Parameters

object

required

Test configuration. Pass {} for the defaults: 500 permutations, seed 42, all six metrics, all six scramblers, and a pass/fail gate at Sharpe p-value < 0.05.

Show config fields

object

required

Show PTL params fields

string

"auto", "curve" or "permutation". Default: "auto".

integer

Permutations per scrambler. Default: 500. Runtime scales with it — lower it for a first look, raise it for a result you intend to report.

integer

Random seed, so a run can be repeated. Default: 42.

integer

Parallel workers. Default: 1.

array

SHARPE, CAGR, MAX_DRAWDOWN, CALMAR, SORTINO, TOTAL_RETURN.

array

IID, BLOCK, STATIONARY, PHASE, PHASE_MULTI_ASSET, IAAFT. Each destroys a different structure in the series, so a strategy can survive one and fail another.

array

EARLY_OVERFIT, RETURN_BOUNDS, DRAWDOWN_BOUNDS, WF_STATELESS. LUCKINESS and SYSTEM_FACTORY are aliases of EARLY_OVERFIT.

object

The pass/fail gate: a metric and a p-value threshold.

object

Price source for permutation mode. The default downloads from Massive; use {"data_source": "yahoo"} only when you specifically want Yahoo.

string

Equity-curve chart from run_plot_job. Drives curve mode.

string

Weight-dynamics chart from run_plot_job or run_risk_job. Drives permutation mode.

string

Optional OHLCV chart from run_plot_job, as an alternative price source.

string

Optional trading_report_*.json from run_trading_job. Used only to reconcile this tool’s metrics against the backtest; it never replaces the equity series.

Passing a data_extractor_*.json link is temporarily unsupported — it is ignored, and the prices are downloaded anyway. Supply the charts listed above instead.

Returns

{
  "status": "Succeeded",
  "output_url": "https://stqsnpprod.blob.core.windows.net/data/ptl_engine_7c8d9e0f1a2b.json",
  "output_name": "ptl_engine_7c8d9e0f1a2b.json",
  "execution_name": "ptl-worker-abc123xyz"
}

The artifact holds observed_metrics, the per-scrambler ladder with p-values and confidence intervals, return and drawdown bounds, a skipped list explaining which requested tests did not run and why, and metric_reconciliation when vectorbt_url was supplied. Read the skipped list before reporting a verdict — a test that never ran is not a test that passed.

Example — quick curve check

{
  "equity_url": "https://stqsnpprod.blob.core.windows.net/data/plot_profit_and_loss_graphs_plot_equity_curve_3f2e1d0c9b8a.html",
  "config": {
    "PTL params": {
      "mode": "curve",
      "n_permutations": 100
    }
  }
}

Example — permutation test

{
  "weights_url": "https://stqsnpprod.blob.core.windows.net/data/risk_dynamic_weights_f1e2d3c4b5a6.html",
  "config": {
    "PTL params": {
      "mode": "permutation",
      "n_permutations": 500,
      "seed": 42,
      "metrics": ["SHARPE", "CAGR", "MAX_DRAWDOWN"],
      "scramblers": ["IID", "BLOCK", "STATIONARY", "PHASE"],
      "tests_to_run": ["EARLY_OVERFIT", "RETURN_BOUNDS", "DRAWDOWN_BOUNDS"],
      "conjunction": { "metric": "SHARPE", "threshold": 0.05 }
    }
  }
}

Next Step

The result is a verdict, not an input: nothing downstream reads a ptl_engine_ file. A weak p-value is a reason to go back to run_po_job and run_trading_job and test a changed strategy — not to rerun the same one until it passes.

Overview

How To

Tools description

Overview

Parameters

Returns

Example — quick curve check

Example — permutation test

Next Step

​Overview

​Parameters

​Returns

​Example — quick curve check

​Example — permutation test

​Next Step

Overview

Parameters

Returns

Example — quick curve check

Example — permutation test

Next Step