diff --git a/docs/lookahead-analysis.md b/docs/lookahead-analysis.md index 878589fca..9ff3ed792 100644 --- a/docs/lookahead-analysis.md +++ b/docs/lookahead-analysis.md @@ -35,24 +35,47 @@ These are set to avoid users accidentally generating false positives. ### Introduction Many strategies - without the programmer knowing - have fallen prey to lookahead bias. -This typically make the strategy backtest look profitable, sometimes to extremes, but this is not realistic as the strategy is "cheating" by looking at data it would not have in dry or live modes. +This typically make the strategy backtest look profitable, sometimes to extremes, +but this is not realistic as the strategy is "cheating" by looking at data it would not have in dry or live modes. -The reason why strategies can "cheat" is because the freqtrade backtesting process populates the full dataframe including all candle timestamps at the outset. -If the programmer is not careful or oblivious how things work internally (which sometimes can be really hard to find out) then the strategy will look into the future. +The reason why strategies can "cheat" is, +because the freqtrade backtesting process populates the full dataframe including all candle timestamps at the outset. +If the programmer is not careful or oblivious how things work internally +(which sometimes can be really hard to find out) then the strategy will look into the future. This command is made to try to verify the validity in the form of the aforementioned lookahead bias. ### How does the command work? It will start with a backtest of all pairs to generate a baseline for indicators and entries/exits. -After this initial backtest runs, it will look if the `minimum-trade-amount` is met and if not cancel the lookahead-analysis for this strategy. - +After this initial backtest runs, +it will look if the `minimum-trade-amount` is met and if not cancel the lookahead-analysis for this strategy. If this happens, use a wider timerange to get more trades for the analysis, or use a timerange where more trades occur. -After setting the baseline it will then do additional backtest runs for every entry and exit separately. -When these verification backtests complete, it will compare the indicators at the signal candles (both entry or exit) and report the bias. +After setting the baseline it will then do additional backtest runs for every entry and exit separately. +When these verification backtests complete, it will compare the indicators at the signal candles (both entry or exit) +and report the bias. After all signals have been verified or falsified a result table will be generated for the user to see. +### How to find and remove bias? How can I salvage the strategy? +If you found a biased strategy online and want to have the same results - just without bias - +then you will be out of luck most of the time. +Usually the bias in the strategy is THE driving factor for those outrageous profits. +In the end it is pretty easy to produce amazing results if you can look into the future. +Removing that central part that pushed the profits up so hich will make it significantly worse. Right? +You might be able to salvage it partially if the biased indicators or parts are not the core of the strategy. + +### Examples of lookahead-bias +- shift(-10) looks 10 candles into the future. +- iloc[] accesses a specific row in the dataframe, be very careful with this. +- For-loops are prone to introduce lookahead-bias if you don't tightly control which numbers are looped through. +- Aggregation funktions like .mean() .min(), .max(), without a rolling window, + will show you the respective pointers of the **whole** dataframe. + A non-biased example of .mean() would be the following line. + It just looks back 12 candles and takes its mean instead of the mean of the whole dataframe. + dataframe['volume_mean_12'] = dataframe['volume'].rolling(12).mean() +- ta.MACD(dataframe,12,26,1) will introduce bias with a signalperiod of 1. + ### What do the columns in the results table mean? - `filename`: name of the checked strategy file @@ -64,17 +87,22 @@ After all signals have been verified or falsified a result table will be generat - `biased_indicators`: shows you the indicators themselves that are defined in populate_indicators You might get false positives in the `biased_exit_signals` if you have biased entry signals paired with those exits. -However, a biased entry will usually result in a biased exit too, even if the exit itself does not produce the bias - especially if your entry and exit conditions use the same biased indicator. +However, a biased entry will usually result in a biased exit too, +even if the exit itself does not produce the bias - +especially if your entry and exit conditions use the same biased indicator. -**So, address the bias in the entries first, then address the exits.** +**Address the bias in the entries first, then address the exits.** ### Caveats - `lookahead-analysis` can only verify / falsify the trades it calculated and verified. -If the strategy has many different signals / signal types, it's up to you to select appropriate parameters to ensure that all signals have triggered at least once. Signals that are not triggered will not have been verified. +If the strategy has many different signals / signal types, it's up to you to select appropriate parameters + to ensure that all signals have triggered at least once. Signals that are not triggered will not have been verified. This would lead to a false-negative, i.e. the strategy will be reported as non-biased. - `lookahead-analysis` has access to the same backtesting options and this can introduce problems. Please don't use any options like enabling position stacking as this will distort the number of checked signals. -If you decide to do so, then make doubly sure that you won't ever run out of `max_open_trades` slots, and that you have enough capital in the backtest wallet configuration. -- In the results table, the `biased_indicators` column will falsely flag FreqAI target indicators defined in `set_freqai_targets()` as biased. +If you decide to do so, then make doubly sure that you won't ever run out of `max_open_trades` slots, +and that you have enough capital in the backtest wallet configuration. +- In the results table, the `biased_indicators` column +will falsely flag FreqAI target indicators defined in `set_freqai_targets()` as biased. **These are not biased and can safely be ignored.**