Optimize a Strategy
A backtest gives you one result from one setup on one window of history. Optimization asks whether that number is a real property of your logic, or whether you tuned your way into a lucky corner of the past that will not repeat live.
Optimization is a separate, shipped step from backtesting. It sweeps the knobs in your strategy across many values, scores each version against an objective you pick, and then stress-tests the winner to tell you how much of the result to trust. You run it from the canvas or hand it to Signac, and the answer lands as an Optimize Result node next to your strategy.
Backtests and optimization runs are simulations on historical market data, never live orders. The mechanics are identical to what a deployed strategy runs. Check for overfitting now, before real funds are on the line.
How optimization differs from a backtest
A backtest runs your current setup once. It answers “how would this exact strategy have behaved.” That is useful, and the right first move, but a single run cannot tell you whether a slightly different period or threshold would have done far better, or whether your good result survives on data it never trained on.
Optimization runs the setup many times over. It changes the parameters between runs, scores each one, and adds two things a lone backtest cannot give you: a test on out-of-sample data, and a probability that your best result is an illusion.
| Backtest | Optimize | |
|---|---|---|
| Runs | One setup, once | Many setups across a parameter grid |
| Answers | How did this version do | Which version scores best, and is that score real |
| Validation | The window you chose | Out-of-sample windows and an overfitting probability |
| Output | Backtest Result node | Optimize Result node |
The three modes
The Optimize panel offers three modes. Each answers a different question, and each builds on the one before it.
Optimize
Search the selected parameter grid on one backtest window and pick the best-scoring setup. This is the fast pass: define which knobs to sweep and over what values, choose an objective, and the optimizer scores every combination and hands back the winner and the full candidate table. It is good for finding a strong setup quickly, but on its own it does not prove that setup will hold out of sample.
Walk-forward
Train on rolling historical windows, then test each chosen setup on the next out-of-sample window. Walk-forward picks parameters on one stretch of history and then grades them on the stretch that follows, the data they never saw. A setup that only shines in-sample and collapses out-of-sample is exactly the trap walk-forward is built to catch.
CSCV/PBO overfitting check
Estimate overfitting risk by trying many train/test block splits and reporting PBO, the probability of backtest overfitting. CSCV (combinatorially symmetric cross-validation) reshuffles your data into many train-and-test splits and measures how often the setup that looked best in training fails to stay best in testing. The headline number, PBO, is a plain-language reading of overfitting risk: a high PBO means the winning setup probably will not hold up live.
Use the three modes in sequence: Optimize to find candidates, walk-forward to test them on unseen data, then a CSCV/PBO run to put a number on the overfitting risk. A setup that survives all three has earned a small live allocation.
Choose an objective
An objective is the single metric the optimizer tries to make better, plus a direction, either Maximize or Minimize. Pick Maximize for a metric where more is better (return, Sharpe) and Minimize where less is (drawdown).
The objectives:
| Objective | What it rewards |
|---|---|
| Total return % | The largest total percentage gain over the window |
| Sharpe | The best return per unit of total volatility |
| Sortino | The best return per unit of downside volatility only |
| Max drawdown % | The shallowest worst peak-to-trough decline (minimize this) |
| Profit factor | The highest gross profit divided by gross loss |
| Win rate | The highest share of profitable trades |
| Trade count | The most trades taken |
| Closed trade count | The most positions actually closed |
Profit factor above 1 means the winners outweigh the losers in dollars. It is a favorite because it resists the trap of a high win rate hiding a few catastrophic losses.
The CSCV/PBO overfitting check works on return-path objectives only: Total return %, Sharpe, Sortino, and Max drawdown %. Pick one of those before you run a CSCV pass, or the panel will ask you to switch.
Choose what to optimize
The optimizer works on the numeric knobs already in your strategy: indicator periods, comparison thresholds, order size, take-profit and stop-loss levels. It detects those locations for you. You then decide which ones to sweep and over what values, either as a range (min, max, step) or an explicit list.
Locations that look like the same knob, an EMA period used in two places at the same value, merge into a single parameter so they move together. Split them if you want them tuned independently. The count of combinations is the grid: three parameters with ten values each is a thousand backtests, so sweep the knobs that matter and keep the grid manageable.
Run it from the canvas
Load your strategy
Build or load your strategy on the canvas, the same one you would deploy.
Open the Optimize panel
Click Tune parameters in the floating toolbar.
Pick a mode
Optimize, Walk-Forward, or CSCV/PBO.
Choose the parameters
Select the parameters to sweep and set each one’s range or list of values.
Set the objective
Pick the metric and its direction, Maximize or Minimize.
Run it
An Optimize Result node appears on the canvas with the outcome.
If the strategy has validation errors, or the grid exceeds the run caps, the panel flags it before anything starts. Fix the flagged issue and run again.
Run it with Signac
Signac can drive the whole thing from a sentence. Describe the goal in plain words: “tune the fast and slow EMA periods for the best Sharpe,” “run a walk-forward on this,” or “check whether this is overfit.”
- If you name a mode, Signac uses it. If you just say “optimize,” Signac asks which mode you want before running anything.
- By default, Signac opens the Optimize panel prefilled with an editable plan and a short rationale, so you review the parameters, objective, and mode and adjust before you run. Nothing runs until you say so.
- Ask Signac to run it now and it runs directly. Either way the result lands as an Optimize Result node you can inspect.
Signac reads the same result you do and will not claim performance until the node returns.
Reading the results
The Optimize Result node reports what its mode measured. Across all three, it also carries an overfitting and evidence card that reads how much weight the result can bear, so a thin or suspect run is flagged instead of standing in as proof. This is the same Evidence Card Signac places when it reviews a strategy, a canvas node that records exactly what was computed so every claim can be checked against it.
- Optimize. The best-scoring setup and its objective value, its profit factor, and a summary of how many candidates ran, plus the full candidate table so you can compare the runners-up.
- Walk-forward. The recommended parameter set and its out-of-sample metrics, the return and profit factor earned on data the optimizer did not train on. This out-of-sample number matters more than any in-sample high.
- CSCV/PBO. The probability of backtest overfitting, with the split counts behind it (how many splits were usable, skipped, or came back overfit). Read PBO first. A low PBO on a return-path objective is the strongest single signal that a setup is real.
Treat a clean walk-forward and a low PBO as permission to test further with a small live allocation, never as a promise. The market regime you optimized on is one of many.
Risk profiles
Optimization does not just hand you a winner. When a run produces a linked set of results, an Optimize pass, a walk-forward, and a CSCV/PBO check from the same bundle, the panel can synthesize three ready-to-apply versions of your strategy, each tuned to a different appetite:
- Conservative ranks candidates by risk-adjusted score first (a deflated or probabilistic Sharpe), then by the smallest drawdown, then by the lowest overfitting probability.
- Balanced ranks by the objective you optimized for, then by risk-adjusted score, then by mean out-of-sample Sharpe.
- Aggressive ranks by out-of-sample total return, and is only offered when the overfitting probability sits below a set boundary.
All three come from the same explicit linked run, so they are three honest readings of one body of evidence rather than three separate optimizations you have to reconcile. If the linked results are missing or do not match, the panel says so instead of inventing a profile.
Applying a profile
Each profile carries an Apply profile button. Applying it writes that profile’s parameter values into your strategy’s target nodes on the canvas: the periods, thresholds, and stop-loss or take-profit levels the profile chose. Nothing else moves.
A confirmation step spells out the boundary in plain words: “This edits canvas parameters only. It is not a trade approval.” Applying a profile changes the numbers on your canvas. It does not deploy anything, place an order, or take a position. You still review the updated strategy and deploy it yourself, the same as any other change.
The Apply button is active only when the profile has writable targets on the current canvas. If you have edited the strategy since the run, apply from a fresh run rather than onto a canvas the evidence no longer describes.
Evidence readiness
Three profiles are only as trustworthy as the run behind them, and the Evidence readiness panel is where you check that before you act. It reads the whole run and reports how much weight it can bear.
- Readiness status. A single badge summarizes the run: Evidence ready, Evidence caution, Evidence failed, or Evidence unknown. Read it first. A caution or a failure means the run cannot support a decision yet.
- Decision gate. A dedicated gate carries its own status and a count of how many risk profiles are actually applicable, so you know whether the run has earned a decision at all.
- Coverage. The panel lists each optimize mode with a coverage badge: available, missing, loading, error, or unavailable, so any hole in the evidence is visible.
- Falsification tests. For each profile, the panel can suggest tests designed to break it rather than flatter it: a 2x fee and slippage stress, a look at the weakest out-of-sample fold, and a CSCV/PBO split-sensitivity recheck. They are suggestions; run them to find the weaknesses before you deploy.
The sequence stays the same throughout: find a candidate, test it on data it never saw, put a number on the overfitting risk, then read the readiness before you apply a profile and deploy.
Limits
Optimization inherits every limit of the backtests it runs. Fills are simulated at historical prices, so it cannot reproduce live order-book depth, latency, or market impact. See the backtesting limitations for the full list, and weight the results the same way.
There is also a deeper trap: the more combinations you try, the more likely one looks good by chance alone, which is why walk-forward and CSCV/PBO exist. A backtest gives you a number, and optimization tells you how much to trust it.