No default value is specified in the docs for the processing_mode, making it unclear that the default behaviour is to filter out pairs, rather than append.
Fix logical error in the conditional checks for model classes. The `elif` statement that looks for "lightgbm.sklearn" or "xgb" in the model class string is now broken into two separate conditions because the old condition would always evaluate to `True` due to the non-empty string "xgb".
In a combination with a wallet size of 1 billion it should never be able to run out of money avoiding false-positives of some users who just wanted to test a strategy without actually checking how the stake_amount-variable should be used in combination with the strategy-function custom_stake_amount.
reason: some strategies demand a custom_stake_amount of 1$ demanding a very large wallet-size (which already was set previously)
Others start with 100% of a slot size and subdivide the base-orders and safety-orders down to finish at 100% of a slot-size and use unlimited stake_amount.
Edited docs to reflect that change.
In a combination with a wallet size of 1 billion it should never be able to run out of money avoiding false-positives of some users who just wanted to test a strategy without actually checking how the stake_amount-variable should be used in combination with the strategy-function custom_stake_amount
reason: some strategies demand a custom_stake_amount of 1$ demanding a very large wallet-size (which already was set previously)
Others start with 100% of a slot size and subdivide the base-orders and safety-orders down to finish at 100% of a slot-size and use unlimited stake_amount.
Edited docs to reflect that change too
There are some trade- and candle-related fields that are always available to output on the indicator-list so have updated the docs to include the most commonly used ones.
- moved doc from utils.md to lookahead-analysis.md and modified it (unfinished)
- added a check to automatically edit the config['backtest_cache'] to be 'none'
- adjusted test_lookahead_helper_export_to_csv to catch the new catching of errors
- adjusted test_lookahead_helper_text_table_lookahead_analysis_instances to catch the new catching of errors
- changed lookahead_analysis.start result-reporting to show that not enough trades were caught including x of y
moved doc from utils.md to lookahead-analysis.md and modified it (unfinished)
added a check to automatically edit the config['backtest_cache'] to be 'none'
Looking at has_bias should be enough to statisfy the test.
The tests could be extended with thecking the buy/sell signals and the dataframe itself -
but this should be sufficient for now.
switched from args to config (args still work)
renamed exportfilename to lookahead_analysis_exportfilename so if users decide to put something into it then it won't compete with other configurations
- optimized pairs for entry_varholder and exit_varholder to only check a single pair instead of all pairs.
- bias-check of freqai strategies now possible
- added condition to not crash when compared_df is empty (meaning no differences have been found)
open ended timeranges now work
if a file fails then it will not report as non-bias, but report in the table as error and the csv file will not have it listed.
removed args_common_optimize for strategy-updater
backtest_lookahead_bias_checker:
added args and cli-options for minimum and target trade amounts
fixed code according to best-practice coding requests of matthias (CamelCase etc)
Contributors may be given commit privileges. Preference will be given to those with:
1. Past contributions to Freqtrade and other related open-source projects. Contributions to Freqtrade include both code (both accepted and pending) and friendly participation in the issue tracker and Pull request reviews. Quantity and quality are considered.
1. Past contributions to Freqtrade and other related open-source projects. Contributions to Freqtrade include both code (both accepted and pending) and friendly participation in the issue tracker and Pull request reviews. Both quantity and quality are considered.
1. A coding style that the other core committers find simple, minimal, and clean.
1. Access to resources for cross-platform development and testing.
@@ -136,7 +136,7 @@ class MyAwesomeStrategy(IStrategy):
### Dynamic parameters
Parameters can also be defined dynamically, but must be available to the instance once the * [`bot_start()` callback](strategy-callbacks.md#bot-start) has been called.
Parameters can also be defined dynamically, but must be available to the instance once the [`bot_start()` callback](strategy-callbacks.md#bot-start) has been called.
@@ -305,7 +305,7 @@ A backtesting result will look like that:
| Sharpe | 2.97 |
| Calmar | 6.29 |
| Profit factor | 1.11 |
| Expectancy | -0.15 |
| Expectancy (Ratio) | -0.15 (-0.05) |
| Avg. stake amount | 0.001 BTC |
| Total trade volume | 0.429 BTC |
| | |
@@ -324,6 +324,7 @@ A backtesting result will look like that:
| Days win/draw/lose | 12 / 82 / 25 |
| Avg. Duration Winners | 4:23:00 |
| Avg. Duration Loser | 6:55:00 |
| Max Consecutive Wins / Loss | 3 / 4 |
| Rejected Entry signals | 3089 |
| Entry/Exit Timeouts | 0 / 0 |
| Canceled Trade Entries | 34 |
@@ -409,7 +410,7 @@ It contains some useful key metrics about performance of your strategy on backte
| Sharpe | 2.97 |
| Calmar | 6.29 |
| Profit factor | 1.11 |
| Expectancy | -0.15 |
| Expectancy (Ratio) | -0.15 (-0.05) |
| Avg. stake amount | 0.001 BTC |
| Total trade volume | 0.429 BTC |
| | |
@@ -428,6 +429,7 @@ It contains some useful key metrics about performance of your strategy on backte
| Days win/draw/lose | 12 / 82 / 25 |
| Avg. Duration Winners | 4:23:00 |
| Avg. Duration Loser | 6:55:00 |
| Max Consecutive Wins / Loss | 3 / 4 |
| Rejected Entry signals | 3089 |
| Entry/Exit Timeouts | 0 / 0 |
| Canceled Trade Entries | 34 |
@@ -467,6 +469,7 @@ It contains some useful key metrics about performance of your strategy on backte
-`Best day` / `Worst day`: Best and worst day based on daily profit.
-`Days win/draw/lose`: Winning / Losing days (draws are usually days without closed trade).
-`Avg. Duration Winners` / `Avg. Duration Loser`: Average durations for winning and losing trades.
-`Max Consecutive Wins / Loss`: Maximum consecutive wins/losses in a row.
-`Rejected Entry signals`: Trade entry signals that could not be acted upon due to `max_open_trades` being reached.
-`Entry/Exit Timeouts`: Entry/exit orders which did not fill (only applicable if custom pricing is used).
-`Canceled Trade Entries`: Number of trades that have been canceled by user request via `adjust_entry_price`.
@@ -534,6 +537,7 @@ Since backtesting lacks some detailed information about what happens within a ca
- ROI
- exits are compared to high - but the ROI value is used (e.g. ROI = 2%, high=5% - so the exit will be at 2%)
- exits are never "below the candle", so a ROI of 2% may result in a exit at 2.4% if low was at 2.4% profit
- ROI entries which came into effect on the triggering candle (e.g. `120: 0.02` for 1h candles, from `60: 0.05`) will use the candle's open as exit rate
- Force-exits caused by `<N>=-1` ROI entries use low as exit value, unless N falls on the candle open (e.g. `120: -1` for 1h candles)
- Stoploss exits happen exactly at stoploss price, even if low was lower, but the loss will be `2 * fees` higher than the stoploss price
- Stoploss is evaluated before ROI within one candle. So you can often see more trades with the `stoploss` exit reason comparing to the results obtained with the same strategy in the Dry Run/Live Trade modes
@@ -614,13 +618,13 @@ To compare multiple strategies, a list of Strategies can be provided to backtest
This is limited to 1 timeframe value per run. However, data is only loaded once from disk so if you have multiple
strategies you'd like to compare, this will give a nice runtime boost.
All listed Strategies need to be in the same directory.
All listed Strategies need to be in the same directory, unless also `--recursive-strategy-search` is specified, where sub-directories within the strategy directory are also considered.
This will save the results to `user_data/backtest_results/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename.
This will save the results to `user_data/backtest_results/backtest-result-<datetime>.json`, including results for both `Strategy001` and `Strategy002`.
There will be an additional table comparing win/losses of the different strategies (identical to the "Total" row in the first table).
Detailed output for all strategies one after the other will be available, so make sure to scroll up to see the details per strategy.
@@ -7,7 +7,7 @@ This page provides you some basic concepts on how Freqtrade works and operates.
* **Strategy**: Your trading strategy, telling the bot what to do.
* **Trade**: Open position.
* **Open Order**: Order which is currently placed on the exchange, and is not yet complete.
* **Pair**: Tradable pair, usually in the format of Base/Quote (e.g. XRP/USDT).
* **Pair**: Tradable pair, usually in the format of Base/Quote (e.g. `XRP/USDT` for spot, `XRP/USDT:USDT` for futures).
* **Timeframe**: Candle length to use (e.g. `"5m"`, `"1h"`, ...).
* **Indicators**: Technical indicators (SMA, EMA, RSI, ...).
* **Limit order**: Limit orders which execute at the defined limit price or better.
@@ -20,6 +20,20 @@ This page provides you some basic concepts on how Freqtrade works and operates.
All profit calculations of Freqtrade include fees. For Backtesting / Hyperopt / Dry-run modes, the exchange default fee is used (lowest tier on the exchange). For live operations, fees are used as applied by the exchange (this includes BNB rebates etc.).
## Pair naming
Freqtrade follows the [ccxt naming convention](https://docs.ccxt.com/#/README?id=consistency-of-base-and-quote-currencies) for currencies.
Using the wrong naming convention in the wrong market will usually result in the bot not recognizing the pair, usually resulting in errors like "this pair is not available".
### Spot pair naming
For spot pairs, naming will be `base/quote` (e.g. `ETH/USDT`).
### Futures pair naming
For futures pairs, naming will be `base/quote:settle` (e.g. `ETH/USDT:USDT`).
## Bot execution logic
Starting freqtrade in dry-run or live mode (using `freqtrade trade`) will start the bot and start the bot iteration loop.
This page explains the different parameters of the bot and how to run it.
!!! Note
If you've used `setup.sh`, don't forget to activate your virtual environment (`source .env/bin/activate`) before running freqtrade commands.
If you've used `setup.sh`, don't forget to activate your virtual environment (`source .venv/bin/activate`) before running freqtrade commands.
!!! Warning "Up-to-date clock"
The clock on the system running the bot must be accurate, synchronized to a NTP server frequently enough to avoid problems with communication to the exchanges.
@@ -14,7 +14,7 @@ You can specify a different configuration file used by the bot with the `-c/--co
If you used the [Quick start](docker_quickstart.md#docker-quick-start) method for installing
the bot, the installation script should have already created the default configuration file (`config.json`) for you.
If the default configuration file is not created we recommend to use `freqtrade new-config --config config.json` to generate a basic configuration file.
If the default configuration file is not created we recommend to use `freqtrade new-config --config user_data/config.json` to generate a basic configuration file.
The Freqtrade configuration file is to be written in JSON format.
@@ -177,7 +177,7 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `exit_pricing.order_book_top` | Bot will use the top N rate in Order Book "price_side" to exit. I.e. a value of 2 will allow the bot to pick the 2nd ask rate in [Order Book Exit](#exit-price-with-orderbook-enabled)<br>*Defaults to `1`.* <br> **Datatype:** Positive Integer
| `custom_price_max_distance_ratio` | Configure maximum distance ratio between current and custom entry or exit price. <br>*Defaults to `0.02` 2%).*<br> **Datatype:** Positive float
| | **TODO**
| `use_exit_signal` | Use exit signals produced by the strategy in addition to the `minimal_roi`. [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `true`.* <br> **Datatype:** Boolean
| `use_exit_signal` | Use exit signals produced by the strategy in addition to the `minimal_roi`. <br>Setting this to false disables the usage of `"exit_long"` and `"exit_short"` columns. Has no influence on other exit methods (Stoploss, ROI, callbacks). [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `true`.* <br> **Datatype:** Boolean
| `exit_profit_only` | Wait until the bot reaches `exit_profit_offset` before taking an exit decision. [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `false`.* <br> **Datatype:** Boolean
| `exit_profit_offset` | Exit-signal is only active above this value. Only active in combination with `exit_profit_only=True`. [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `0.0`.* <br> **Datatype:** Float (as ratio)
| `ignore_roi_if_entry_signal` | Do not exit if the entry signal is still active. This setting takes preference over `minimal_roi` and `use_exit_signal`. [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `false`.* <br> **Datatype:** Boolean
@@ -188,7 +188,6 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `max_entry_position_adjustment` | Maximum additional order(s) for each open trade on top of the first entry Order. Set it to `-1` for unlimited additional orders. [More information here](strategy-callbacks.md#adjust-trade-position). <br> [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `-1`.*<br> **Datatype:** Positive Integer or -1
| | **Exchange**
| `exchange.name` | **Required.** Name of the exchange class to use. [List below](#user-content-what-values-for-exchangename). <br> **Datatype:** String
| `exchange.sandbox` | Use the 'sandbox' version of the exchange, where the exchange provides a sandbox for risk-free integration. See [here](sandbox-testing.md) in more details.<br> **Datatype:** Boolean
| `exchange.key` | API key to use for the exchange. Only required when you are in production mode.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
| `exchange.secret` | API secret to use for the exchange. Only required when you are in production mode.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
| `exchange.password` | API password to use for the exchange. Only required when you are in production mode and for exchanges that use password for API requests.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
@@ -251,8 +250,8 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `db_url` | Declares database URL to use. NOTE: This defaults to `sqlite:///tradesv3.dryrun.sqlite` if `dry_run` is `true`, and to `sqlite:///tradesv3.sqlite` for production instances. <br> **Datatype:** String, SQLAlchemy connect string
| `logfile` | Specifies logfile name. Uses a rolling strategy for log file rotation for 10 files with the 1MB limit per file. <br> **Datatype:** String
| `add_config_files` | Additional config files. These files will be loaded and merged with the current config file. The files are resolved relative to the initial file.<br> *Defaults to `[]`*. <br> **Datatype:** List of strings
| `dataformat_ohlcv` | Data format to use to store historical candle (OHLCV) data. <br> *Defaults to `json`*. <br> **Datatype:** String
| `dataformat_trades` | Data format to use to store historical trades data. <br> *Defaults to `jsongz`*. <br> **Datatype:** String
| `dataformat_ohlcv` | Data format to use to store historical candle (OHLCV) data. <br> *Defaults to `feather`*. <br> **Datatype:** String
| `dataformat_trades` | Data format to use to store historical trades data. <br> *Defaults to `feather`*. <br> **Datatype:** String
| `reduce_df_footprint` | Recast all numeric columns to float32/int32, with the objective of reducing ram/disk usage (and decreasing train/inference timing in FreqAI). (Currently only affects FreqAI use-cases) <br> **Datatype:** Boolean. <br> Default: `False`.
### Parameters in the strategy
@@ -322,7 +321,7 @@ For example, if you have 10 ETH available in your wallet on the exchange and `tr
To fully utilize compounding profits when using multiple bots on the same exchange account, you'll want to limit each bot to a certain starting balance.
This can be accomplished by setting `available_capital` to the desired starting balance.
Assuming your account has 10.000 USDT and you want to run 2 different strategies on this exchange.
Assuming your account has 10000 USDT and you want to run 2 different strategies on this exchange.
You'd set `available_capital=5000` - granting each bot an initial capital of 5000 USDT.
The bot will then split this starting balance equally into `max_open_trades` buckets.
Profitable trades will result in increased stake-sizes for this bot - without affecting the stake-sizes of the other bot.
@@ -573,9 +572,11 @@ In addition to fiat currencies, a range of crypto currencies is supported.
The valid values are:
```json
"BTC", "ETH", "XRP", "LTC", "BCH", "USDT"
"BTC", "ETH", "XRP", "LTC", "BCH", "BNB"
```
Removing `fiat_display_currency` completely from the configuration will skip initializing coingecko, and will not show any FIAT currency conversion. This has no importance for the correct functioning of the bot.
## Using Dry-run mode
We recommend starting the bot in the Dry-run mode to see how your bot will
@@ -595,7 +596,7 @@ creating trades on the exchange.
```json
"exchange": {
"name": "bittrex",
"name": "binance",
"key": "key",
"secret": "secret",
...
@@ -614,6 +615,7 @@ Once you will be happy with your bot performance running in the Dry-run mode, yo
* Orders are simulated, and will not be posted to the exchange.
* Market orders fill based on orderbook volume the moment the order is placed.
* Limit orders fill once the price reaches the defined level - or time out based on `unfilledtimeout` settings.
* Limit orders will be converted to market orders if they cross the price by more than 1%.
* In combination with `stoploss_on_exchange`, the stop_loss price is assumed to be filled.
* Open orders (not trades, which are stored in the database) are kept open after bot restarts, with the assumption that they were not filled while being offline.
@@ -644,7 +646,7 @@ API Keys are usually only required for live trading (trading for real money, bot
@@ -6,7 +6,7 @@ To download data (candles / OHLCV) needed for backtesting and hyperoptimization
If no additional parameter is specified, freqtrade will download data for `"1m"` and `"5m"` timeframes for the last 30 days.
Exchange and pairs will come from `config.json` (if specified using `-c/--config`).
Otherwise`--exchange` becomes mandatory.
Without provided configuration,`--exchange` becomes mandatory.
You can use a relative timerange (`--days 20`) or an absolute starting point (`--timerange 20200101-`). For incremental downloads, the relative approach should be used.
--prepend Allow data prepending. (Data-appending is disabled)
Common arguments:
-v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
--logfile FILE Log to the file specified. Special values are:
--logfile FILE, --log-file FILE
Log to the file specified. Special values are:
'syslog', 'journald'. See the documentation for more
details.
-V, --version show program's version number and exit
@@ -83,40 +83,47 @@ Common arguments:
```
!!! Tip "Downloading all data for one quote currency"
Often, you'll want to download data for all pairs of a specific quote-currency. In such cases, you can use the following shorthand:
`freqtrade download-data --exchange binance --pairs .*/USDT <...>`. The provided "pairs" string will be expanded to contain all active pairs on the exchange.
To also download data for inactive (delisted) pairs, add `--include-inactive-pairs` to the command.
!!! Note "Startup period"
`download-data` is a strategy-independent command. The idea is to download a big chunk of data once, and then iteratively increase the amount of data stored.
For that reason, `download-data` does not care about the "startup-period" defined in a strategy. It's up to the user to download additional days if the backtest should start at a specific point in time (while respecting startup period).
### Pairs file
### Start download
In alternative to the whitelist from `config.json`, a `pairs.json` file can be used.
If you are using Binance for example:
- create a directory `user_data/data/binance` and copy or create the `pairs.json` file in that directory.
- update the `pairs.json` file to contain the currency pairs you are interested in.
A very simple command (assuming an available `config.json` file) can look as follows.
```bash
mkdir -p user_data/data/binance
touch user_data/data/binance/pairs.json
freqtrade download-data --exchange binance
```
The format of the `pairs.json` file is a simple json list.
Mixing different stake-currencies is allowed for this file, since it's only used for downloading.
This will download historical candle (OHLCV) data for all the currency pairs defined in the configuration.
!!! Tip "Downloading all data for one quote currency"
Often, you'll want to download data for all pairs of a specific quote-currency. In such cases, you can use the following shorthand:
`freqtrade download-data --exchange binance --pairs .*/USDT <...>`. The provided "pairs" string will be expanded to contain all active pairs on the exchange.
To also downloaddata for inactive (delisted) pairs, add `--include-inactive-pairs` to the command.
or as regex (in this case, to download all active USDT pairs)
* To use a different directory than the exchange specific default, use `--datadir user_data/data/some_directory`.
* To change the exchange used to download the historical data from, please use a different configuration file (you'll probably need to adjust rate limits etc.)
* To use `pairs.json` from some other directory, use `--pairs-file some_other_dir/pairs.json`.
* To download historical candle (OHLCV) data for only 10 days, use `--days 10` (defaults to 30 days).
* To download historical candle (OHLCV) data from a fixed starting point, use `--timerange 20200101-` - which will download all data from January 1st, 2020.
* Use `--timeframes` to specify what timeframe download the historical candle (OHLCV) data for. Default is `--timeframes 1m 5m` which will download 1-minute and 5-minute data.
* To use exchange, timeframe and list of pairs as defined in your configuration file, use the `-c/--config` option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine `-c/--config` with most other options.
??? Note "Permission denied errors"
If your configuration directory `user_data` was made by docker, you may get the following error:
@@ -131,39 +138,7 @@ Mixing different stake-currencies is allowed for this file, since it's only used
sudo chown -R $UID:$GID user_data
```
### Start download
Then run:
```bash
freqtrade download-data --exchange binance
```
This will download historical candle (OHLCV) data for all the currency pairs you defined in `pairs.json`.
- To use a different directory than the exchange specific default, use `--datadir user_data/data/some_directory`.
- To change the exchange used to download the historical data from, please use a different configuration file (you'll probably need to adjust rate limits etc.)
- To use `pairs.json` from some other directory, use `--pairs-file some_other_dir/pairs.json`.
- To download historical candle (OHLCV) data for only 10 days, use `--days 10` (defaults to 30 days).
- To download historical candle (OHLCV) data from a fixed starting point, use `--timerange 20200101-` - which will download all data from January 1st, 2020.
- Use `--timeframes` to specify what timeframe download the historical candle (OHLCV) data for. Default is `--timeframes 1m 5m` which will download 1-minute and 5-minute data.
- To use exchange, timeframe and list of pairs as defined in your configuration file, use the `-c/--config` option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine `-c/--config` with most other options.
#### Download additional data before the current timerange
### Download additional data before the current timerange
Assuming you downloaded all data from 2022 (`--timerange 20220101-`) - but you'd now like to also backtest with earlier data.
You can do so by using the `--prepend` flag, combined with `--timerange` - specifying an end-date.
Freqtrade currently supports the following data-formats:
* `feather` - a dataformat based on Apache Arrow
* `json` - plain "text" json files
* `jsongz` - a gzip-zipped version of json files
* `hdf5` - a high performance datastore
* `feather` - a dataformat based on Apache Arrow (OHLCV only)
* `parquet` - columnar datastore (OHLCV only)
By default, OHLCV data is stored as `json` data, while trades data is stored as `jsongz` data.
By default, both OHLCV data and trades data are stored in the `feather` format.
This can be changed via the `--data-format-ohlcv` and `--data-format-trades` command line arguments respectively.
To persist this change, you should also add the following snippet to your configuration, so you don't have to insert the above arguments each time:
@@ -228,17 +203,46 @@ time freqtrade list-data --show-timerange --data-format-ohlcv <dataformat>
| Format | Size | timing |
|------------|-------------|-------------|
| `feather` | 72Mb | 3.5s |
| `json` | 149Mb | 25.6s |
| `jsongz` | 39Mb | 27s |
| `hdf5` | 145Mb | 3.9s |
| `feather` | 72Mb | 3.5s |
| `parquet` | 83Mb | 3.8s |
Size has been taken from the BTC/USDT 1m spot combination for the timerange specified above.
To have a best performance/size mix, we recommend the use of either feather or parquet.
To have a best performance/size mix, we recommend using the default feather format, or parquet.
#### Sub-command convert data
### Pairs file
In alternative to the whitelist from `config.json`, a `pairs.json` file can be used.
If you are using Binance for example:
* create a directory `user_data/data/binance` and copy or create the `pairs.json` file in that directory.
* update the `pairs.json` file to contain the currency pairs you are interested in.
```bash
mkdir -p user_data/data/binance
touch user_data/data/binance/pairs.json
```
The format of the `pairs.json` file is a simple json list.
Mixing different stake-currencies is allowed for this file, since it's only used for downloading.
``` json
[
"ETH/BTC",
"ETH/USDT",
"BTC/USDT",
"XRP/ETH"
]
```
!!! Note
The `pairs.json` file is only used when no configuration is loaded (implicitly by naming, or via `--config` flag).
You can force the usage of this file via `--pairs-file pairs.json` - however we recommend to use the pairlist from within the configuration, either via `exchange.pair_whitelist` or `pairs` setting in the configuration.
-v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
--logfile FILE Log to the file specified. Special values are:
--logfile FILE, --log-file FILE
Log to the file specified. Special values are:
'syslog', 'journald'. See the documentation for more
details.
-V, --version show program's version number and exit
@@ -287,10 +292,9 @@ Common arguments:
Path to directory with historical backtesting data.
--userdir PATH, --user-data-dir PATH
Path to userdata directory.
```
##### Example converting data
### Example converting data
The following command will convert all candle (OHLCV) data available in `~/.freqtrade/data/binance` from json to jsongz, saving diskspace in the process.
It'll also remove original json data files (`--erase` parameter).
@@ -299,7 +303,7 @@ It'll also remove original json data files (`--erase` parameter).
By default, `download-data` sub-command downloads Candles (OHLCV) data. Some exchanges also provide historic trade-data via their API.
This data can be useful if you need many different timeframes, since it is only downloaded once, and then resampled locally to the desired timeframes.
Since this data is large by default, the files use gzip by default. They are stored in your data-directory with the naming convention of `<pair>-trades.json.gz` (`ETH_BTC-trades.json.gz`). Incremental mode is also supported, as for historic OHLCV data, so downloading the data once per week with `--days 8` will create an incremental data-repository.
Since this data is large by default, the files use the feather fileformat by default. They are stored in your data-directory with the naming convention of `<pair>-trades.feather` (`ETH_BTC-trades.feather`). Incremental mode is also supported, as for historic OHLCV data, so downloading the data once per week with `--days 8` will create an incremental data-repository.
To use this mode, simply add `--dl-trades` to your call. This will swap the download method to download trades, and resamples the data locally.
To debug freqtrade, we recommend VSCode with the following launch configuration (located in `.vscode/launch.json`).
To debug freqtrade, we recommend VSCode (with the Python extension) with the following launch configuration (located in `.vscode/launch.json`).
Details will obviously vary between setups - but this should work to get you started.
``` json
@@ -102,6 +102,19 @@ This method can also be used to debug a strategy, by setting the breakpoints wit
A similar setup can also be taken for Pycharm - using `freqtrade` as module name, and setting the command line arguments as "parameters".
??? Tip "Correct venv usage"
When using a virtual environment (which you should), make sure that your Editor is using the correct virtual environment to avoid problems or "unknown import" errors.
#### Vscode
You can select the correct environment in VSCode with the command "Python: Select Interpreter" - which will show you environments the extension detected.
If your environment has not been detected, you can also pick a path manually.
#### Pycharm
In pycharm, you can select the appropriate Environment in the "Run/Debug Configurations" window.
This assumes that you have the repository checked out, and the editor is started at the repository root level (so setup.py is at the top level of your repository).
@@ -305,6 +318,7 @@ Additional tests / steps to complete:
* Check if balance shows correctly (*)
* Create market order (*)
* Create limit order (*)
* Cancel order (*)
* Complete trade (enter + exit) (*)
* Compare result calculation between exchange and bot
* Ensure fees are applied correctly (check the database against the exchange)
@@ -405,6 +419,9 @@ This part of the documentation is aimed at maintainers, and shows how to create
### Create release branch
!!! Note
Make sure that the `stable` branch is up-to-date!
First, pick a commit that's about one week old (to not include latest additions to releases).
``` bash
@@ -417,14 +434,11 @@ Determine if crucial bugfixes have been made between this commit and the current
* Merge the release branch (stable) into this branch.
* Edit `freqtrade/__init__.py` and add the version matching the current date (for example `2019.7` for July 2019). Minor versions can be `2019.7.1` should we need to do a second release that month. Version numbers must follow allowed versions from PEP0440 to avoid failures pushing to pypi.
* Commit this part.
* push that branch to the remote and create a PR against the stable branch.
* Push that branch to the remote and create a PR against the **stable branch**.
* Update develop version to next version following the pattern `2019.8-dev`.
### Create changelog from git commits
!!! Note
Make sure that the `stable` branch is up-to-date!
``` bash
# Needs to be done before merging / pulling that branch.
@@ -14,6 +14,9 @@ Start by downloading and installing Docker / Docker Desktop for your platform:
Freqtrade documentation assumes the use of Docker desktop (or the docker compose plugin).
While the docker-compose standalone installation still works, it will require changing all `docker compose` commands from `docker compose` to `docker-compose` to work (e.g. `docker compose up -d` will become `docker-compose up -d`).
??? Warning "Docker on windows"
If you just installed docker on a windows system, make sure to reboot your system, otherwise you might encounter unexplainable Problems related to network connectivity to docker containers.
## Freqtrade with docker
Freqtrade provides an official Docker image on [Dockerhub](https://hub.docker.com/r/freqtradeorg/freqtrade/), as well as a [docker compose file](https://github.com/freqtrade/freqtrade/blob/stable/docker-compose.yml) ready for usage.
@@ -78,7 +81,7 @@ If you've selected to enable FreqUI in the `new-config` step, you will have freq
You can now access the UI by typing localhost:8080 in your browser.
??? Note "UI Access on a remote servers"
??? Note "UI Access on a remote server"
If you're running on a VPS, you should consider using either a ssh tunnel, or setup a VPN (openVPN, wireguard) to connect to your bot.
This will ensure that freqUI is not directly exposed to the internet, which is not recommended for security reasons (freqUI does not support https out of the box).
Setup of these tools is not part of this tutorial, however many good tutorials can be found on the internet.
@@ -128,7 +131,7 @@ All freqtrade arguments will be available by running `docker compose run --rm fr
!!! Note "`docker compose run --rm`"
Including `--rm` will remove the container after completion, and is highly recommended for all modes except trading mode (running with `freqtrade trade` command).
??? Note "Using docker without docker"
??? Note "Using docker without docker compose"
"`docker compose run --rm`" will require a compose file to be provided.
Some freqtrade commands that don't require authentication such as `list-pairs` can be run with "`docker run --rm`" instead.
For example `docker run --rm freqtradeorg/freqtrade:stable list-pairs --exchange binance --quote BTC --print-json`.
@@ -172,7 +175,7 @@ You can then run `docker compose build --pull` to build the docker image, and ru
### Plotting with docker
Commands `freqtrade plot-profit` and `freqtrade plot-dataframe` ([Documentation](plotting.md)) are available by changing the image to `*_plot` in your docker-compose.yml file.
Commands `freqtrade plot-profit` and `freqtrade plot-dataframe` ([Documentation](plotting.md)) are available by changing the image to `*_plot` in your `docker-compose.yml` file.
If you're on windows and just installed Docker (desktop), make sure to reboot your System. Docker can have problems with network connectivity without a restart.
You should obviously also make sure to have your [settings](#accessing-the-ui) accordingly.
!!! Warning
Due to the above, we do not recommend the usage of docker on windows for production setups, but only for experimentation, datadownload and backtesting.
The `Edge Positioning` module uses probability to calculate your win rate and risk reward ratio. It will use these statistics to control your strategy trade entry points, position size and, stoploss.
!!! Danger "Deprecated functionality"
`Edge positioning` (or short Edge) is currently in maintenance mode only (we keep existing functionality alive) and should be considered as deprecated.
It will currently not receive new features until either someone stepped forward to take up ownership of that module - or we'll decide to remove edge from freqtrade.
!!! Warning
When using `Edge positioning` with a dynamic whitelist (VolumePairList), make sure to also use `AgeFilter` and set it to at least `calculate_since_number_of_days` to avoid problems with missing data.
@@ -55,7 +55,7 @@ This configuration enables kraken, as well as rate-limiting to avoid bans from t
## Binance
!!! Warning "Server location and geo-ip restrictions"
Please be aware that binance restrict api access regarding the server country. The currents and nonexhaustive countries blocked are United States, Malaysia (Singapour), Ontario (Canada). Please go to [binance terms > b. Eligibility](https://www.binance.com/en/terms) to find up to date list.
Please be aware that Binance restricts API access regarding the server country. The current and non-exhaustive countries blocked are Canada, Malaysia, Netherlands and United States. Please go to [binance terms > b. Eligibility](https://www.binance.com/en/terms) to find up to date list.
@@ -127,6 +127,8 @@ Freqtrade will not attempt to change these settings.
## Kraken
Kraken supports [time_in_force](configuration.md#understand-order_time_in_force) with settings "GTC" (good till cancelled), "IOC" (immediate-or-cancel) and "PO" (Post only) settings.
!!! Tip "Stoploss on Exchange"
Kraken supports `stoploss_on_exchange` and can use both stop-loss-market and stop-loss-limit orders. It provides great advantages, so we recommend to benefit from it.
You can use either `"limit"` or `"market"` in the `order_types.stoploss` configuration setting to decide which type to use.
@@ -136,13 +138,41 @@ Freqtrade will not attempt to change these settings.
The Kraken API does only provide 720 historic candles, which is sufficient for Freqtrade dry-run and live trade modes, but is a problem for backtesting.
To download data for the Kraken exchange, using `--dl-trades` is mandatory, otherwise the bot will download the same 720 candles over and over, and you'll not have enough backtest data.
Due to the heavy rate-limiting applied by Kraken, the following configuration section should be used to download data:
To speed up downloading, you can download the [trades zip files](https://support.kraken.com/hc/en-us/articles/360047543791-Downloadable-historical-market-data-time-and-sales-) kraken provides.
These are usually updated once per quarter. Freqtrade expects these files to be placed in `user_data/data/kraken/trades_csv`.
``` json
"ccxt_async_config": {
"enableRateLimit": true,
"rateLimit": 3100
},
A structure as follows can make sense if using incremental files, with the "full" history in one directory, and incremental files in different directories.
The assumption for this mode is that the data is downloaded and unzipped keeping filenames as they are.
Duplicate content will be ignored (based on timestamp) - though the assumption is that there is no gap in the data.
This means, if your "full" history ends in Q4 2022 - then both incremental updates Q1 2023 and Q2 2023 are available.
Not having this will lead to incomplete data, and therefore invalid results while using the data.
@@ -153,48 +183,6 @@ Due to the heavy rate-limiting applied by Kraken, the following configuration se
Please pay attention that rateLimit configuration entry holds delay in milliseconds between requests, NOT requests\sec rate.
So, in order to mitigate Kraken API "Rate limit exceeded" exception, this configuration should be increased, NOT decreased.
## Bittrex
### Order types
Bittrex does not support market orders. If you have a message at the bot startup about this, you should change order type values set in your configuration and/or in the strategy from `"market"` to `"limit"`. See some more details on this [here in the FAQ](faq.md#im-getting-the-exchange-bittrex-does-not-support-market-orders-message-and-cannot-run-my-strategy).
Bittrex also does not support `VolumePairlist` due to limited / split API constellation at the moment.
Please use `StaticPairlist`. Other pairlists (other than `VolumePairlist`) should not be affected.
### Volume pairlist
Bittrex does not support the direct usage of VolumePairList. This can however be worked around by using the advanced mode with `lookback_days: 1` (or more), which will emulate 24h volume.
Read more in the [pairlist documentation](plugins.md#volumepairlist-advanced-mode).
### Restricted markets
Bittrex split its exchange into US and International versions.
The International version has more pairs available, however the API always returns all pairs, so there is currently no automated way to detect if you're affected by the restriction.
If you have restricted pairs in your whitelist, you'll get a warning message in the log on Freqtrade startup for each restricted pair.
The warning message will look similar to the following:
If you're an "International" customer on the Bittrex exchange, then this warning will probably not impact you.
If you're a US customer, the bot will fail to create orders for these pairs, and you should remove them from your whitelist.
You can get a list of restricted markets by using the following snippet:
``` python
import ccxt
ct = ccxt.bittrex()
lm = ct.load_markets()
res = [p for p, x in lm.items() if 'US' in x['info']['prohibitedIn']]
print(res)
```
## Kucoin
Kucoin requires a passphrase for each api key, you will therefore need to add this key into the configuration so your exchange section looks as follows:
For Kucoin, it is suggested to add `"KCS/<STAKE>"` to your blacklist to avoid issues, unless you are willing to maintain enough extra `KCS` on the account or unless you're willing to disable using `KCS` for fees.
Kucoin accounts may use `KCS` for fees, and if a trade happens to be on `KCS`, further trades may consume this position and make the initial `KCS` trade unsellable as the expected amount is not there anymore.
## Huobi
## HTX (formerly Huobi)
!!! Tip "Stoploss on Exchange"
Huobi supports `stoploss_on_exchange` and uses `stop-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
HTX supports `stoploss_on_exchange` and uses `stop-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
## OKX (former OKEX)
@@ -259,14 +247,39 @@ The configuration parameter `exchange.unknown_fee_rate` can be used to specify t
Futures trading on bybit is currently supported for USDT markets, and will use isolated futures mode.
Users with unified accounts (there's no way back) can create a Sub-account which will start as "non-unified", and can therefore use isolated futures.
On startup, freqtrade will set the position mode to "One-way Mode" for the whole (sub)account. This avoids making this call over and over again (slowing down bot operations), but means that changes to this setting may result in exceptions and errors.
On startup, freqtrade will set the position mode to "One-way Mode" for the whole (sub)account. This avoids making this call over and over again (slowing down bot operations), but means that changes to this setting may result in exceptions and errors
As bybit doesn't provide funding rate history, the dry-run calculation is used for live trades as well.
API Keys for live futures trading (Subaccount on non-unified) must have the following permissions:
* Read-write
* Contract - Orders
* Contract - Positions
We do strongly recommend to limit all API keys to the IP you're going to use it from.
!!! Tip "Stoploss on Exchange"
Bybit (futures only) supports `stoploss_on_exchange` and uses `stop-loss-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
On futures, Bybit supports both `stop-limit` as well as `stop-market` orders. You can use either `"limit"` or `"market"` in the `order_types.stoploss` configuration setting to decide which type to use.
## Bitmart
Bitmart requires the API key Memo (the name you give the API key) to go along with the exchange key and secret.
It's therefore required to pass the UID as well.
```json
"exchange": {
"name": "bitmart",
"uid": "your_bitmart_api_key_memo",
"secret": "your_exchange_secret",
"password": "your_exchange_api_key_password",
// ...
}
```
!!! Warning "Necessary Verification"
Bitmart requires Verification Lvl2 to successfully trade on the spot market through the API - even though trading via UI works just fine with just Lvl1 verification.
## All exchanges
Should you experience constant errors with Nonce (like `InvalidNonce`), it is best to regenerate the API keys. Resetting Nonce is difficult and it's usually easier to regenerate the API keys.
@@ -20,7 +20,7 @@ Futures trading is supported for selected exchanges. Please refer to the [docume
* When you work with your strategy & hyperopt file you should use a proper code editor like VSCode or PyCharm. A good code editor will provide syntax highlighting as well as line numbers, making it easy to find syntax errors (most likely pointed out by Freqtrade during startup).
## Freqtrade common issues
## Freqtrade common questions
### Can freqtrade open multiple positions on the same pair in parallel?
@@ -36,7 +36,7 @@ Running the bot with `freqtrade trade --config config.json` shows the output `fr
This could be caused by the following reasons:
* The virtual environment is not active.
* Run `source .env/bin/activate` to activate the virtual environment.
* Run `source .venv/bin/activate` to activate the virtual environment.
* The installation did not complete successfully.
* Please check the [Installation documentation](installation.md).
@@ -78,6 +78,14 @@ Where possible (e.g. on binance), the use of the exchange's dedicated fee curren
On binance, it's sufficient to have BNB in your account, and have "Pay fees in BNB" enabled in your profile. Your BNB balance will slowly decline (as it's used to pay fees) - but you'll no longer encounter dust (Freqtrade will include the fees in the profit calculations).
Other exchanges don't offer such possibilities, where it's simply something you'll have to accept or move to a different exchange.
### I deposited more funds to the exchange, but my bot doesn't recognize this
Freqtrade will update the exchange balance when necessary (Before placing an order).
RPC calls (Telegram's `/balance`, API calls to `/balance`) can trigger an update at max. once per hour.
If `adjust_trade_position` is enabled (and the bot has open trades eligible for position adjustments) - then the wallets will be refreshed once per hour.
To force an immediate update, you can use `/reload_config` - which will restart the bot.
### I want to use incomplete candles
Freqtrade will not provide incomplete candles to strategies. Using incomplete candles will lead to repainting and consequently to strategies with "ghost" buys, which are impossible to both backtest, and verify after they happened.
@@ -120,15 +128,9 @@ This warning can point to one of the below problems:
* Barely traded pair -> Check the pair on the exchange webpage, look at the timeframe your strategy uses. If the pair does not have any volume in some candles (usually visualized with a "volume 0" bar, and a "_" as candle), this pair did not have any trades in this timeframe. These pairs should ideally be avoided, as they can cause problems with order-filling.
* API problem -> API returns wrong data (this only here for completeness, and should not happen with supported exchanges).
### I'm getting the "RESTRICTED_MARKET" message in the log
Currently known to happen for US Bittrex users.
Read [the Bittrex section about restricted markets](exchanges.md#restricted-markets) for more information.
### I'm getting the "Exchange XXX does not support market orders." message and cannot run my strategy
As the message says, your exchange does not support market orders and you have one of the [order types](configuration.md/#understand-order_types) set to "market". Your strategy was probably written with other exchanges in mind and sets "market" orders for "stoploss" orders, which is correct and preferable for most of the exchanges supporting market orders (but not for Bittrex and Gate.io).
As the message says, your exchange does not support market orders and you have one of the [order types](configuration.md/#understand-order_types) set to "market". Your strategy was probably written with other exchanges in mind and sets "market" orders for "stoploss" orders, which is correct and preferable for most of the exchanges supporting market orders (but not for Gate.io).
To fix this, redefine order types in the strategy to use "limit" instead of "market":
@@ -160,9 +160,10 @@ Below are the values you can expect to include/use inside a typical strategy dat
|------------|-------------|
| `df['&*']` | Any dataframe column prepended with `&` in `set_freqai_targets()` is treated as a training target (label) inside FreqAI (typically following the naming convention `&-s*`). For example, to predict the close price 40 candles into the future, you would set `df['&-s_close'] = df['close'].shift(-self.freqai_info["feature_parameters"]["label_period_candles"])` with `"label_period_candles": 40` in the config. FreqAI makes the predictions and gives them back under the same key (`df['&-s_close']`) to be used in `populate_entry/exit_trend()`. <br>**Datatype:** Depends on the output of the model.
| `df['&*_std/mean']` | Standard deviation and mean values of the defined labels during training (or live tracking with `fit_live_predictions_candles`). Commonly used to understand the rarity of a prediction (use the z-score as shown in `templates/FreqaiExampleStrategy.py` and explained [here](#creating-a-dynamic-target-threshold) to evaluate how often a particular prediction was observed during training or historically with `fit_live_predictions_candles`). <br>**Datatype:** Float.
| `df['do_predict']` | Indication of an outlier data point. The return value is integer between -2 and 2, which lets you know if the prediction is trustworthy or not. `do_predict==1` means that the prediction is trustworthy. If the Dissimilarity Index (DI, see details [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di)) of the input data point is above the threshold defined in the config, FreqAI will subtract 1 from `do_predict`, resulting in `do_predict==0`. If `use_SVM_to_remove_outliers()` is active, the Support Vector Machine (SVM, see details [here](freqai-feature-engineering.md#identifying-outliers-using-a-support-vector-machine-svm)) may also detect outliers in training and prediction data. In this case, the SVM will also subtract 1 from `do_predict`. If the input data point was considered an outlier by the SVM but not by the DI, or vice versa, the result will be `do_predict==0`. If both the DI and the SVM considers the input data point to be an outlier, the result will be `do_predict==-1`. As with the SVM, if `use_DBSCAN_to_remove_outliers` is active, DBSCAN (see details [here](freqai-feature-engineering.md#identifying-outliers-with-dbscan)) may also detect outliers and subtract 1 from `do_predict`. Hence, if both the SVM and DBSCAN are active and identify a datapoint that was above the DI threshold as an outlier, the result will be `do_predict==-2`. A particular case is when `do_predict == 2`, which means that the model has expired due to exceeding `expired_hours`. <br>**Datatype:** Integer between -2 and 2.
| `df['do_predict']` | Indication of an outlier data point. The return value is integer between -2 and 2, which lets you know if the prediction is trustworthy or not. `do_predict==1` means that the prediction is trustworthy. If the Dissimilarity Index (DI, see details [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di)) of the input data point is above the threshold defined in the config, FreqAI will subtract 1 from `do_predict`, resulting in `do_predict==0`. If `use_SVM_to_remove_outliers` is active, the Support Vector Machine (SVM, see details [here](freqai-feature-engineering.md#identifying-outliers-using-a-support-vector-machine-svm)) may also detect outliers in training and prediction data. In this case, the SVM will also subtract 1 from `do_predict`. If the input data point was considered an outlier by the SVM but not by the DI, or vice versa, the result will be `do_predict==0`. If both the DI and the SVM considers the input data point to be an outlier, the result will be `do_predict==-1`. As with the SVM, if `use_DBSCAN_to_remove_outliers` is active, DBSCAN (see details [here](freqai-feature-engineering.md#identifying-outliers-with-dbscan)) may also detect outliers and subtract 1 from `do_predict`. Hence, if both the SVM and DBSCAN are active and identify a datapoint that was above the DI threshold as an outlier, the result will be `do_predict==-2`. A particular case is when `do_predict == 2`, which means that the model has expired due to exceeding `expired_hours`. <br>**Datatype:** Integer between -2 and 2.
| `df['DI_values']` | Dissimilarity Index (DI) values are proxies for the level of confidence FreqAI has in the prediction. A lower DI means the prediction is close to the training data, i.e., higher prediction confidence. See details about the DI [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di). <br>**Datatype:** Float.
| `df['%*']` | Any dataframe column prepended with `%` in `feature_engineering_*()` is treated as a training feature. For example, you can include the RSI in the training feature set (similar to in `templates/FreqaiExampleStrategy.py`) by setting `df['%-rsi']`. See more details on how this is done [here](freqai-feature-engineering.md). <br>**Note:** Since the number of features prepended with `%` can multiply very quickly (10s of thousands of features are easily engineered using the multiplictative functionality of, e.g., `include_shifted_candles` and `include_timeframes` as described in the [parameter table](freqai-parameter-table.md)), these features are removed from the dataframe that is returned from FreqAI to the strategy. To keep a particular type of feature for plotting purposes, you would prepend it with `%%`. <br>**Datatype:** Depends on the output of the model.
| `df['%*']` | Any dataframe column prepended with `%` in `feature_engineering_*()` is treated as a training feature. For example, you can include the RSI in the training feature set (similar to in `templates/FreqaiExampleStrategy.py`) by setting `df['%-rsi']`. See more details on how this is done [here](freqai-feature-engineering.md). <br>**Note:** Since the number of features prepended with `%` can multiply very quickly (10s of thousands of features are easily engineered using the multiplictative functionality of, e.g., `include_shifted_candles` and `include_timeframes` as described in the [parameter table](freqai-parameter-table.md)), these features are removed from the dataframe that is returned from FreqAI to the strategy. To keep a particular type of feature for plotting purposes, you would prepend it with `%%` (see details below). <br>**Datatype:** Depends on the feature created by the user.
| `df['%%*']` | Any dataframe column prepended with `%%` in `feature_engineering_*()` is treated as a training feature, just the same as the above `%` prepend. However, in this case, the features are returned back to the strategy for FreqUI/plot-dataframe plotting and monitoring in Dry/Live/Backtesting <br>**Datatype:** Depends on the feature created by the user. Please note that features created in `feature_engineering_expand()` will have automatic FreqAI naming schemas depending on the expansions that you configured (i.e. `include_timeframes`, `include_corr_pairlist`, `indicators_periods_candles`, `include_shifted_candles`). So if you want to plot `%%-rsi` from `feature_engineering_expand_all()`, the final naming scheme for your plotting config would be: `%%-rsi-period_10_ETH/USDT:USDT_1h` for the `rsi` feature with `period=10`, `timeframe=1h`, and `pair=ETH/USDT:USDT` (the `:USDT` is added if you are using futures pairs). It is useful to simply add `print(dataframe.columns)` in your `populate_indicators()` after `self.freqai.start()` to see the full list of available features that are returned to the strategy for plotting purposes.
@@ -7,7 +7,7 @@ Low level feature engineering is performed in the user strategy within a set of
| Function | Description |
|---------------|-------------|
| `feature_engineering_expand_all()` | This optional function will automatically expand the defined features on the config defined `indicator_periods_candles`, `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`.
| `feature_engineering_expand_basic()` | This optional function will automatically expand the defined features on the config defined `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`. Note: this function does *not* expand across `include_periods_candles`.
| `feature_engineering_expand_basic()` | This optional function will automatically expand the defined features on the config defined `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`. Note: this function does *not* expand across `indicator_periods_candles`.
| `feature_engineering_standard()` | This optional function will be called once with the dataframe of the base timeframe. This is the final function to be called, which means that the dataframe entering this function will contain all the features and columns from the base asset created by the other `feature_engineering_expand` functions. This function is a good place to do custom exotic feature extractions (e.g. tsfresh). This function is also a good place for any feature that should not be auto-expanded upon (e.g., day of the week).
| `set_freqai_targets()` | Required function to set the targets for the model. All targets must be prepended with `&` to be recognized by the FreqAI internals.
@@ -178,8 +178,11 @@ You can ask for each of the defined features to be included also for informative
`include_shifted_candles` indicates the number of previous candles to include in the feature set. For example, `include_shifted_candles: 2` tells FreqAI to include the past 2 candles for each of the features in the feature set.
In total, the number of features the user of the presented example strat has created is: length of `include_timeframes`* no. features in `feature_engineering_expand_*()` * length of `include_corr_pairlist` * no. `include_shifted_candles` * length of `indicator_periods_candles`
In total, the number of features the user of the presented example strategy has created is: length of `include_timeframes`* no. features in `feature_engineering_expand_*()` * length of `include_corr_pairlist` * no. `include_shifted_candles` * length of `indicator_periods_candles`
$= 3 * 3 * 3 * 2 * 2 = 108$.
!!! note "Learn more about creative feature engineering"
Check out our [medium article](https://emergentmethods.medium.com/freqai-from-price-to-prediction-6fadac18b665) geared toward helping users learn how to creatively engineer features.
### Gain finer control over `feature_engineering_*` functions with `metadata`
@@ -209,41 +212,7 @@ Another example, where the user wants to use live metrics from the trade databas
You need to set the standard dictionary in the config so that FreqAI can return proper dataframe shapes. These values will likely be overridden by the prediction model, but in the case where the model has yet to set them, or needs a default initial value, the pre-set values are what will be returned.
## Feature normalization
FreqAI is strict when it comes to data normalization. The train features, $X^{train}$, are always normalized to [-1, 1] using a shifted min-max normalization:
All other data (test data and unseen prediction data in dry/live/backtest) is always automatically normalized to the training feature space according to industry standards. FreqAI stores all the metadata required to ensure that test and prediction features will be properly normalized and that predictions are properly denormalized. For this reason, it is not recommended to eschew industry standards and modify FreqAI internals - however - advanced users can do so by inheriting `train()` in their custom `IFreqaiModel` and using their own normalization functions.
## Data dimensionality reduction with Principal Component Analysis
You can reduce the dimensionality of your features by activating the `principal_component_analysis` in the config:
```json
"freqai": {
"feature_parameters" : {
"principal_component_analysis": true
}
}
```
This will perform PCA on the features and reduce their dimensionality so that the explained variance of the data set is >= 0.999. Reducing data dimensionality makes training the model faster and hence allows for more up-to-date models.
## Inlier metric
The `inlier_metric` is a metric aimed at quantifying how similar the features of a data point are to the most recent historical data points.
You define the lookback window by setting `inlier_metric_window` and FreqAI computes the distance between the present time point and each of the previous `inlier_metric_window` lookback points. A Weibull function is fit to each of the lookback distributions and its cumulative distribution function (CDF) is used to produce a quantile for each lookback point. The `inlier_metric` is then computed for each time point as the average of the corresponding lookback quantiles. The figure below explains the concept for an `inlier_metric_window` of 5.
FreqAI adds the `inlier_metric` to the training features and hence gives the model access to a novel type of temporal information.
This function does **not** remove outliers from the data set.
## Weighting features for temporal importance
### Weighting features for temporal importance
FreqAI allows you to set a `weight_factor` to weight recent data more strongly than past data via an exponential function:
@@ -253,13 +222,103 @@ where $W_i$ is the weight of data point $i$ in a total set of $n$ data points. B
## Building the data pipeline
By default, FreqAI builds a dynamic pipeline based on user congfiguration settings. The default settings are robust and designed to work with a variety of methods. These two steps are a `MinMaxScaler(-1,1)` and a `VarianceThreshold` which removes any column that has 0 variance. Users can activate other steps with more configuration parameters. For example if users add `use_SVM_to_remove_outliers: true` to the `freqai` config, then FreqAI will automatically add the [`SVMOutlierExtractor`](#identifying-outliers-using-a-support-vector-machine-svm) to the pipeline. Likewise, users can add `principal_component_analysis: true` to the `freqai` config to activate PCA. The [DissimilarityIndex](#identifying-outliers-with-the-dissimilarity-index-di) is activated with `DI_threshold: 1`. Finally, noise can also be added to the data with `noise_standard_deviation: 0.1`. Finally, users can add [DBSCAN](#identifying-outliers-with-dbscan) outlier removal with `use_DBSCAN_to_remove_outliers: true`.
!!! note "More information available"
Please review the [parameter table](freqai-parameter-table.md) for more information on these parameters.
### Customizing the pipeline
Users are encouraged to customize the data pipeline to their needs by building their own data pipeline. This can be done by simply setting `dk.feature_pipeline` to their desired `Pipeline` object inside their `IFreqaiModel` `train()` function, or if they prefer not to touch the `train()` function, they can override `define_data_pipeline`/`define_label_pipeline` functions in their `IFreqaiModel`:
!!! note "More information available"
FreqAI uses the the [`DataSieve`](https://github.com/emergentmethods/datasieve) pipeline, which follows the SKlearn pipeline API, but adds, among other features, coherence between the X, y, and sample_weight vector point removals, feature removal, feature name following.
```python
from datasieve.transforms import SKLearnWrapper, DissimilarityIndex
from datasieve.pipeline import Pipeline
from sklearn.preprocessing import QuantileTransformer, StandardScaler
from freqai.base_models import BaseRegressionModel
User defines their custom label pipeline here (if they wish)
"""
label_pipeline = Pipeline([
('qt', SKLearnWrapper(StandardScaler())),
])
return label_pipeline
```
Here, you are defining the exact pipeline that will be used for your feature set during training and prediction. You can use *most* SKLearn transformation steps by wrapping them in the `SKLearnWrapper` class as shown above. In addition, you can use any of the transformations available in the [`DataSieve` library](https://github.com/emergentmethods/datasieve).
You can easily add your own transformation by creating a class that inherits from the datasieve `BaseTransform` and implementing your `fit()`, `transform()` and `inverse_transform()` methods:
```python
from datasieve.transforms.base_transform import BaseTransform
# do/dont do something with X, y, sample_weight, or/and feature_list
return X, y, sample_weight, feature_list
```
!!! note "Hint"
You can define this custom class in the same file as your `IFreqaiModel`.
### Migrating a custom `IFreqaiModel` to the new Pipeline
If you have created your own custom `IFreqaiModel` with a custom `train()`/`predict()` function, *and* you still rely on `data_cleaning_train/predict()`, then you will need to migrate to the new pipeline. If your model does *not* rely on `data_cleaning_train/predict()`, then you do not need to worry about this migration.
More details about the migration can be found [here](strategy_migration.md#freqai---new-data-pipeline).
## Outlier detection
Equity and crypto markets suffer from a high level of non-patterned noise in the form of outlier data points. FreqAI implements a variety of methods to identify such outliers and hence mitigate risk.
### Identifying outliers with the Dissimilarity Index (DI)
The Dissimilarity Index (DI) aims to quantify the uncertainty associated with each prediction made by the model.
The Dissimilarity Index (DI) aims to quantify the uncertainty associated with each prediction made by the model.
You can tell FreqAI to remove outlier data points from the training/test data sets using the DI by including the following statement in the config:
@@ -271,7 +330,7 @@ You can tell FreqAI to remove outlier data points from the training/test data se
}
```
The DI allows predictions which are outliers (not existent in the model feature space) to be thrown out due to low levels of certainty. To do so, FreqAI measures the distance between each training data point (feature vector), $X_{a}$, and all other training data points:
Which will add `DissimilarityIndex` step to your `feature_pipeline` and set the threshold to 1. The DI allows predictions which are outliers (not existent in the model feature space) to be thrown out due to low levels of certainty. To do so, FreqAI measures the distance between each training data point (feature vector), $X_{a}$, and all other training data points:
@@ -305,9 +364,9 @@ You can tell FreqAI to remove outlier data points from the training/test data se
}
```
The SVM will be trained on the training data and any data point that the SVM deems to be beyond the feature space will be removed.
Which will add `SVMOutlierExtractor` step to your `feature_pipeline`. The SVM will be trained on the training data and any data point that the SVM deems to be beyond the feature space will be removed.
FreqAI uses `sklearn.linear_model.SGDOneClassSVM` (details are available on scikit-learn's webpage [here](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDOneClassSVM.html) (external website)) and you can elect to provide additional parameters for the SVM, such as `shuffle`, and `nu`.
You can elect to provide additional parameters for the SVM, such as `shuffle`, and `nu` via the `feature_parameters.svm_params` dictionary in the config.
The parameter `shuffle` is by default set to `False` to ensure consistent results. If it is set to `True`, running the SVM multiple times on the same data set might result in different outcomes due to `max_iter` being to low for the algorithm to reach the demanded `tol`. Increasing `max_iter` solves this issue but causes the procedure to take longer time.
@@ -325,7 +384,7 @@ You can configure FreqAI to use DBSCAN to cluster and remove outliers from the t
}
```
DBSCAN is an unsupervised machine learning algorithm that clusters data without needing to know how many clusters there should be.
Which will add the `DataSieveDBSCAN` step to your `feature_pipeline`. This is an unsupervised machine learning algorithm that clusters data without needing to know how many clusters there should be.
Given a number of data points $N$, and a distance $\varepsilon$, DBSCAN clusters the data set by setting all data points that have $N-1$ other data points within a distance of $\varepsilon$ as *core points*. A data point that is within a distance of $\varepsilon$ from a *core point* but that does not have $N-1$ other data points within a distance of $\varepsilon$ from itself is considered an *edge point*. A cluster is then the collection of *core points* and *edge points*. Data points that have no other data points at a distance $<\varepsilon$ are considered outliers. The figure below shows a cluster with $N = 3$.
| `svm_params` | All parameters available in Sklearn's `SGDOneClassSVM()`. See details about some select parameters [here](freqai-feature-engineering.md#identifying-outliers-using-a-support-vector-machine-svm). <br>**Datatype:** Dictionary.
| `use_DBSCAN_to_remove_outliers` | Cluster data using the DBSCAN algorithm to identify and remove outliers from training and prediction data. See details about how it works [here](freqai-feature-engineering.md#identifying-outliers-with-dbscan). <br>**Datatype:** Boolean.
| `inlier_metric_window` | If set, FreqAI adds an `inlier_metric` to the training feature set and set the lookback to be the `inlier_metric_window`, i.e., the number of previous time points to compare the current candle to. Details of how the `inlier_metric` is computed can be found [here](freqai-feature-engineering.md#inlier-metric). <br>**Datatype:** Integer. <br> Default: `0`.
| `noise_standard_deviation` | If set, FreqAI adds noise to the training features with the aim of preventing overfitting. FreqAI generates random deviates from a gaussian distribution with a standard deviation of `noise_standard_deviation` and adds them to all data points. `noise_standard_deviation` should be kept relative to the normalized space, i.e., between -1 and 1. In other words, since data in FreqAI is always normalized to be between -1 and 1, `noise_standard_deviation: 0.05` would result in 32% of the data being randomly increased/decreased by more than 2.5% (i.e., the percent of data falling within the first standard deviation). <br>**Datatype:** Integer. <br> Default: `0`.
| `outlier_protection_percentage` | Enable to prevent outlier detection methods from discarding too much data. If more than `outlier_protection_percentage` % of points are detected as outliers by the SVM or DBSCAN, FreqAI will log a warning message and ignore outlier detection, i.e., the original dataset will be kept intact. If the outlier protection is triggered, no predictions will be made based on the training dataset. <br>**Datatype:** Float. <br> Default: `30`.
| `reverse_train_test_order` | Split the feature dataset (see below) and use the latest data split for training and test on historical split of the data. This allows the model to be trained up to the most recent data point, while avoiding overfitting. However, you should be careful to understand the unorthodox nature of this parameter before employing it. <br>**Datatype:** Boolean. <br> Default: `False` (no reversal).
@@ -75,7 +74,6 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| | **Reinforcement Learning Parameters within the `freqai.rl_config` sub dictionary**
| `rl_config` | A dictionary containing the control parameters for a Reinforcement Learning model. <br>**Datatype:** Dictionary.
| `train_cycles` | Training time steps will be set based on the `train_cycles * number of training data points. <br> **Datatype:** Integer.
| `cpu_count` | Number of processors to dedicate to the Reinforcement Learning training process. <br> **Datatype:** int.
| `max_trade_duration_candles`| Guides the agent training to keep trades below desired length. Example usage shown in `prediction_models/ReinforcementLearner.py` within the customizable `calculate_reward()` function. <br> **Datatype:** int.
| `model_type` | Model string from stable_baselines3 or SBcontrib. Available strings include: `'TRPO', 'ARS', 'RecurrentPPO', 'MaskablePPO', 'PPO', 'A2C', 'DQN'`. User should ensure that `model_training_parameters` match those available to the corresponding stable_baselines3 model by visiting their documentaiton. [PPO doc](https://stable-baselines3.readthedocs.io/en/master/modules/ppo.html) (external website) <br> **Datatype:** string.
| `policy_type` | One of the available policy types from stable_baselines3 <br> **Datatype:** string.
@@ -101,12 +99,12 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
#### trainer_kwargs
| Parameter | Description |
|------------|-------------|
| | **Model training parameters within the `freqai.model_training_parameters.model_kwargs` sub dictionary**
| `max_iters` | The number of training iterations to run. iteration here refers to the number of times we call self.optimizer.step(). used to calculate n_epochs. <br> **Datatype:** int. <br> Default: `100`.
| `batch_size` | The size of the batches to use during training.. <br> **Datatype:** int. <br> Default: `64`.
| `max_n_eval_batches` | The maximum number batches to use for evaluation.. <br> **Datatype:** int, optional. <br> Default: `None`.
| Parameter | Description |
|--------------|-------------|
| | **Model training parameters within the `freqai.model_training_parameters.model_kwargs` sub dictionary**
| `n_epochs` | The `n_epochs` parameter is a crucial setting in the PyTorch training loop that determines the number of times the entire training dataset will be used to update the model's parameters. An epoch represents one full pass through the entire training dataset. Overrides `n_steps`. Either `n_epochs` or `n_steps` must be set. <br><br> **Datatype:** int. optional. <br> Default: `10`.
| `n_steps` | An alternative way of setting `n_epochs` - the number of training iterations to run. Iteration here refer to the number of times we call `optimizer.step()`. Ignored if `n_epochs` is set. A simplified version of the function: <br><br> n_epochs = n_steps / (n_obs / batch_size) <br><br> The motivation here is that `n_steps` is easier to optimize and keep stable across different n_obs - the number of data points. <br> <br> **Datatype:** int. optional. <br> Default: `None`.
| `batch_size` | The size of the batches to use during training. <br><br> **Datatype:** int. <br> Default: `64`.
@@ -20,7 +20,7 @@ With the current framework, we aim to expose the training environment via the co
We envision the majority of users focusing their effort on creative design of the `calculate_reward()` function [details here](#creating-a-custom-reward-function), while leaving the rest of the environment untouched. Other users may not touch the environment at all, and they will only play with the configuration settings and the powerful feature engineering that already exists in FreqAI. Meanwhile, we enable advanced users to create their own model classes entirely.
The framework is built on stable_baselines3 (torch) and OpenAI gym for the base environment class. But generally speaking, the model class is well isolated. Thus, the addition of competing libraries can be easily integrated into the existing framework. For the environment, it is inheriting from `gym.env` which means that it is necessary to write an entirely new environment in order to switch to a different library.
The framework is built on stable_baselines3 (torch) and OpenAI gym for the base environment class. But generally speaking, the model class is well isolated. Thus, the addition of competing libraries can be easily integrated into the existing framework. For the environment, it is inheriting from `gym.Env` which means that it is necessary to write an entirely new environment in order to switch to a different library.
### Important considerations
@@ -173,7 +173,7 @@ class MyCoolRLModel(ReinforcementLearner):
"""
classMyRLEnv(Base5ActionRLEnv):
"""
User made custom environment. This class inherits from BaseEnvironment and gym.env.
User made custom environment. This class inherits from BaseEnvironment and gym.Env.
Users can override any functions from those parent classes. Here is an example
of a user customized `calculate_reward()` function.
@@ -237,11 +237,10 @@ class MyCoolRLModel(ReinforcementLearner):
Reinforcement Learning models benefit from tracking training metrics. FreqAI has integrated Tensorboard to allow users to track training and evaluation performance across all coins and across all retrainings. Tensorboard is activated via the following command:
```bash
cd freqtrade
tensorboard --logdir user_data/models/unique-id
```
where `unique-id` is the `identifier` set in the `freqai` configuration file. This command must be run in a separate shell to view the output in their browser at 127.0.0.1:6006 (6006 is the default port used by Tensorboard).
where `unique-id` is the `identifier` set in the `freqai` configuration file. This command must be run in a separate shell to view the output in the browser at 127.0.0.1:6006 (6006 is the default port used by Tensorboard).
@@ -254,7 +253,7 @@ FreqAI also provides a built in episodic summary logger called `self.tensorboard
```python
classMyRLEnv(Base5ActionRLEnv):
"""
User made custom environment. This class inherits from BaseEnvironment and gym.env.
User made custom environment. This class inherits from BaseEnvironment and gym.Env.
Users can override any functions from those parent classes. Here is an example
of a user customized `calculate_reward()` function.
This way, you can return to using any model you wish by simply specifying the `identifier`.
!!! Note
Backtesting calls `set_freqai_targets()` one time for each backtest window (where the number of windows is the full backtest timerange divided by the `backtest_period_days` parameter). Doing this means that the targets simulate dry/live behavior without look ahead bias. However, the definition of the features in `feature_engineering_*()` is performed once on the entire backtest timerange. This means that you should be sure that features do look-ahead into the future.
Backtesting calls `set_freqai_targets()` one time for each backtest window (where the number of windows is the full backtest timerange divided by the `backtest_period_days` parameter). Doing this means that the targets simulate dry/live behavior without look ahead bias. However, the definition of the features in `feature_engineering_*()` is performed once on the entire training timerange. This means that you should be sure that features do not look-ahead into the future.
More details about look-ahead bias can be found in [Common Mistakes](strategy-customization.md#common-mistakes-when-developing-strategies).
If you are using docker, a dedicated tag with FreqAI dependencies is available as `:freqai`. As such - you can replace the image line in your docker compose file with `image: freqtradeorg/freqtrade:develop_freqai`. This image contains the regular FreqAI dependencies. Similar to native installs, Catboost will not be available on ARM based devices.
If you are using docker, a dedicated tag with FreqAI dependencies is available as `:freqai`. As such - you can replace the image line in your docker compose file with `image: freqtradeorg/freqtrade:develop_freqai`. This image contains the regular FreqAI dependencies. Similar to native installs, Catboost will not be available on ARM based devices. If you would like to use PyTorch or Reinforcement learning, you should use the torch or RL tags, `image: freqtradeorg/freqtrade:develop_freqaitorch`, `image: freqtradeorg/freqtrade:develop_freqairl`.
!!! note "docker-compose-freqai.yml"
We do provide an explicit docker-compose file for this in `docker/docker-compose-freqai.yml` - which can be used via `docker compose -f docker/docker-compose-freqai.yml run ...` - or can be copied to replace the original docker file. This docker-compose file also contains a (disabled) section to enable GPU resources within docker containers. This obviously assumes the system has GPU resources available.
@@ -107,6 +107,18 @@ This is for performance reasons - FreqAI relies on making quick predictions/retr
it needs to download all the training data at the beginning of a dry/live instance. FreqAI stores and appends
new candles automatically for future retrains. This means that if new pairs arrive later in the dry run due to a volume pairlist, it will not have the data ready. However, FreqAI does work with the `ShufflePairlist` or a `VolumePairlist` which keeps the total pairlist constant (but reorders the pairs according to volume).
## Additional learning materials
Here we compile some external materials that provide deeper looks into various components of FreqAI:
- [Real-time head-to-head: Adaptive modeling of financial market data using XGBoost and CatBoost](https://emergentmethods.medium.com/real-time-head-to-head-adaptive-modeling-of-financial-market-data-using-xgboost-and-catboost-995a115a7495)
- [FreqAI - from price to prediction](https://emergentmethods.medium.com/freqai-from-price-to-prediction-6fadac18b665)
## Support
You can find support for FreqAI in a variety of places, including the [Freqtrade discord](https://discord.gg/Jd8JYeWHc4), the dedicated [FreqAI discord](https://discord.gg/7AMWACmbjT), and in [github issues](https://github.com/freqtrade/freqtrade/issues).
## Credits
FreqAI is developed by a group of individuals who all contribute specific skillsets to the project.
@@ -337,11 +337,15 @@ There are four parameter types each suited for different purposes.
* `CategoricalParameter` - defines a parameter with a predetermined number of choices.
* `BooleanParameter` - Shorthand for `CategoricalParameter([True, False])` - great for "enable" parameters.
!!! Tip "Disabling parameter optimization"
Each parameter takes two boolean parameters:
* `load` - when set to `False` it will not load values configured in `buy_params` and `sell_params`.
* `optimize` - when set to `False` parameter will not be included in optimization process.
Use these parameters to quickly prototype various ideas.
### Parameter options
There are two parameter options that can help you to quickly test various ideas:
* `optimize` - when set to `False`, the parameter will not be included in optimization process. (Default: True)
* `load` - when set to `False`, results of a previous hyperopt run (in `buy_params` and `sell_params` either in your strategy or the JSON output file) will not be used as the starting value for subsequent hyperopts. The default value specified in the parameter will be used instead. (Default: True)
!!! Tip "Effects of `load=False` on backtesting"
Be aware that setting the `load` option to `False` will mean backtesting will also use the default value specified in the parameter and *not* the value found through hyperoptimisation.
!!! Warning
Hyperoptable parameters cannot be used in `populate_indicators` - as hyperopt does not recalculate indicators for each epoch, so the starting value would be used in this case.
@@ -433,9 +437,14 @@ While this strategy is most likely too simple to provide consistent profit, it s
`range` property may also be used with `DecimalParameter` and `CategoricalParameter`. `RealParameter` does not provide this property due to infinite search space.
??? Hint "Performance tip"
During normal hyperopting, indicators are calculated once and supplied to each epoch, linearly increasing RAM usage as a factor of increasing cores. As this also has performance implications, hyperopt provides `--analyze-per-epoch` which will move the execution of `populate_indicators()` to the epoch process, calculating a single value per parameter per epoch instead of using the `.range` functionality. In this case, `.range` functionality will only return the actually used value. This will reduce RAM usage, but increase CPU usage. However, your hyperopting run will be less likely to fail due to Out Of Memory (OOM) issues.
During normal hyperopting, indicators are calculated once and supplied to each epoch, linearly increasing RAM usage as a factor of increasing cores. As this also has performance implications, there are two alternatives to reduce RAM usage
In either case, you should try to use space ranges as small as possible this will improve CPU/RAM usage in both scenarios.
* Move `ema_short` and `ema_long` calculations from `populate_indicators()` to `populate_entry_trend()`. Since `populate_entry_trend()` will be calculated every epoch, you don't need to use `.range` functionality.
* hyperopt provides `--analyze-per-epoch` which will move the execution of `populate_indicators()` to the epoch process, calculating a single value per parameter per epoch instead of using the `.range` functionality. In this case, `.range` functionality will only return the actually used value.
These alternatives will reduce RAM usage, but increase CPU usage. However, your hyperopting run will be less likely to fail due to Out Of Memory (OOM) issues.
Whether you are using `.range` functionality or the alternatives above, you should try to use space ranges as small as possible since this will improve CPU/RAM usage.
## Optimizing protections
@@ -917,6 +926,12 @@ Once the optimized strategy has been implemented into your strategy, you should
To achieve same the results (number of trades, their durations, profit, etc.) as during Hyperopt, please use the same configuration and parameters (timerange, timeframe, ...) used for hyperopt `--dmmp`/`--disable-max-market-positions` and `--eps`/`--enable-position-stacking` for Backtesting.
Should results not match, please double-check to make sure you transferred all conditions correctly.
Pay special care to the stoploss, max_open_trades and trailing stoploss parameters, as these are often set in configuration files, which override changes to the strategy.
You should also carefully review the log of your backtest to ensure that there were no parameters inadvertently set by the configuration (like `stoploss`, `max_open_trades` or `trailing_stop`).
### Why do my backtest results not match my hyperopt results?
Should results not match, check the following factors:
* You may have added parameters to hyperopt in `populate_indicators()` where they will be calculated only once **for all epochs**. If you are, for example, trying to optimise multiple SMA timeperiod values, the hyperoptable timeperiod parameter should be placed in `populate_entry_trend()` which is calculated every epoch. See [Optimizing an indicator parameter](https://www.freqtrade.io/en/stable/hyperopt/#optimizing-an-indicator-parameter).
* If you have disabled the auto-export of hyperopt parameters into the JSON parameters file, double-check to make sure you transferred all hyperopted values into your strategy correctly.
* Check the logs to verify what parameters are being set and what values are being used.
* Pay special care to the stoploss, max_open_trades and trailing stoploss parameters, as these are often set in configuration files, which override changes to the strategy. Check the logs of your backtest to ensure that there were no parameters inadvertently set by the configuration (like `stoploss`, `max_open_trades` or `trailing_stop`).
* Verify that you do not have an unexpected parameters JSON file overriding the parameters or the default hyperopt settings in your strategy.
* Verify that any protections that are enabled in backtesting are also enabled when hyperopting, and vice versa. When using `--space protection`, protections are auto-enabled for hyperopting.
@@ -6,7 +6,7 @@ In your configuration, you can use Static Pairlist (defined by the [`StaticPairL
Additionally, [`AgeFilter`](#agefilter), [`PrecisionFilter`](#precisionfilter), [`PriceFilter`](#pricefilter), [`ShuffleFilter`](#shufflefilter), [`SpreadFilter`](#spreadfilter) and [`VolatilityFilter`](#volatilityfilter) act as Pairlist Filters, removing certain pairs and/or moving their positions in the pairlist.
If multiple Pairlist Handlers are used, they are chained and a combination of all Pairlist Handlers forms the resulting pairlist the bot uses for trading and backtesting. Pairlist Handlers are executed in the sequence they are configured. You should always configure either `StaticPairList` or `VolumePairList` as the starting Pairlist Handler.
If multiple Pairlist Handlers are used, they are chained and a combination of all Pairlist Handlers forms the resulting pairlist the bot uses for trading and backtesting. Pairlist Handlers are executed in the sequence they are configured. You can define either `StaticPairList`, `VolumePairList`, `ProducerPairList`, `RemotePairList` or `MarketCapPairList` as the starting Pairlist Handler.
Inactive markets are always removed from the resulting pairlist. Explicitly blacklisted pairs (those in the `pair_blacklist` configuration setting) are also always removed from the resulting pairlist.
@@ -24,7 +24,9 @@ You may also use something like `.*DOWN/BTC` or `.*UP/BTC` to exclude leveraged
* [`VolumePairList`](#volume-pair-list)
* [`ProducerPairList`](#producerpairlist)
* [`RemotePairList`](#remotepairlist)
* [`MarketCapPairList`](#marketcappairlist)
* [`AgeFilter`](#agefilter)
* [`FullTradesFilter`](#fulltradesfilter)
* [`OffsetFilter`](#offsetfilter)
* [`PerformanceFilter`](#performancefilter)
* [`PrecisionFilter`](#precisionfilter)
@@ -66,7 +68,7 @@ When used in the leading position of the chain of Pairlist Handlers, the `pair_w
The `refresh_period` setting allows to define the period (in seconds), at which the pairlist will be refreshed. Defaults to 1800s (30 minutes).
The pairlist cache (`refresh_period`) on `VolumePairList` is only applicable to generating pairlists.
Filtering instances (not the first position in the list) will not apply any cache and will always use up-to-date data.
Filtering instances (not the first position in the list) will not apply any cache (beyond caching candles for the duration of the candle in advanced mode) and will always use up-to-date data.
`VolumePairList` is per default based on the ticker data from exchange, as reported by the ccxt library:
@@ -111,8 +113,8 @@ For convenience `lookback_days` can be specified, which will imply that 1d candl
!!! Warning "Performance implications when using lookback range"
If used in first position in combination with lookback, the computation of the range based volume can be time and resource consuming, as it downloads candles for all tradable pairs. Hence it's highly advised to use the standard approach with `VolumeFilter` to narrow the pairlist down for further range volume calculation.
??? Tip "Unsupported exchanges (Bittrex, Gemini)"
On some exchanges (like Bittrex and Gemini), regular VolumePairList does not work as the api does not natively provide 24h volume. This can be worked around by using candle data to build the volume.
??? Tip "Unsupported exchanges"
On some exchanges (like Gemini), regular VolumePairList does not work as the api does not natively provide 24h volume. This can be worked around by using candle data to build the volume.
To roughly simulate 24h volume, you can use the following configuration.
Please note that These pairlists will only refresh once per day.
@@ -184,24 +186,71 @@ The RemotePairList is defined in the pairlists section of the configuration sett
"pairlists": [
{
"method": "RemotePairList",
"mode": "whitelist",
"processing_mode": "filter",
"pairlist_url": "https://example.com/pairlist",
"number_assets": 10,
"refresh_period": 1800,
"keep_pairlist_on_failure": true,
"read_timeout": 60,
"bearer_token": "my-bearer-token"
"bearer_token": "my-bearer-token",
"save_to_file": "user_data/filename.json"
}
]
```
The optional `mode` option specifies if the pairlist should be used as a `blacklist` or as a `whitelist`. The default value is "whitelist".
The optional `processing_mode` option in the RemotePairList configuration determines how the retrieved pairlist is processed. It can have two values: "filter" or "append". The default value is "filter".
In "filter" mode, the retrieved pairlist is used as a filter. Only the pairs present in both the original pairlist and the retrieved pairlist are included in the final pairlist. Other pairs are filtered out.
In "append" mode, the retrieved pairlist is added to the original pairlist. All pairs from both lists are included in the final pairlist without any filtering.
The `pairlist_url` option specifies the URL of the remote server where the pairlist is located, or the path to a local file (if file:/// is prepended). This allows the user to use either a remote server or a local file as the source for the pairlist.
The `save_to_file` option, when provided with a valid filename, saves the processed pairlist to that file in JSON format. This option is optional, and by default, the pairlist is not saved to a file.
??? Example "Multi bot with shared pairlist example"
`save_to_file` can be used to save the pairlist to a file with Bot1:
```json
"pairlists": [
{
"method": "RemotePairList",
"mode": "whitelist",
"pairlist_url": "https://example.com/pairlist",
"number_assets": 10,
"refresh_period": 1800,
"keep_pairlist_on_failure": true,
"read_timeout": 60,
"save_to_file": "user_data/filename.json"
}
]
```
This saved pairlist file can be loaded by Bot2, or any additional bot with this configuration:
The user is responsible for providing a server or local file that returns a JSON object with the following structure:
```json
{
"pairs": ["XRP/USDT", "ETH/USDT", "LTC/USDT"],
"refresh_period": 1800,
"refresh_period": 1800
}
```
@@ -216,6 +265,25 @@ The optional `bearer_token` will be included in the requests Authorization Heade
!!! Note
In case of a server error the last received pairlist will be kept if `keep_pairlist_on_failure` is set to true, when set to false a empty pairlist is returned.
#### MarketCapPairList
`MarketCapPairList` employs sorting/filtering of pairs by their marketcap rank based of CoinGecko. It will only recognize coins up to the coin placed at rank 250. The returned pairlist will be sorted based of their marketcap ranks.
```json
"pairlists": [
{
"method": "MarketCapPairList",
"number_assets": 20,
"max_rank": 50,
"refresh_period": 86400
}
]
```
`number_assets` defines the maximum number of pairs returned by the pairlist. `max_rank` will determine the maximum rank used in creating/filtering the pairlist. It's expected that some coins within the top `max_rank` marketcap will not be included in the resulting pairlist since not all pairs will have active trading pairs in your preferred market/stake/exchange combination.
`refresh_period` setting defines the period (in seconds) at which the marketcap rank data will be refreshed. Defaults to 86,400s (1 day). The pairlist cache (`refresh_period`) is applicable on both generating pairlists (first position in the list) and filtering instances (not the first position in the list).
#### AgeFilter
Removes pairs that have been listed on the exchange for less than `min_days_listed` days (defaults to `10`) or more than `max_days_listed` days (defaults `None` mean infinity).
@@ -226,6 +294,17 @@ be caught out buying before the pair has finished dropping in price.
This filter allows freqtrade to ignore pairs until they have been listed for at least `min_days_listed` days and listed before `max_days_listed`.
#### FullTradesFilter
Shrink whitelist to consist only in-trade pairs when the trade slots are full (when `max_open_trades` isn't being set to `-1` in the config).
When the trade slots are full, there is no need to calculate indicators of the rest of the pairs (except informative pairs) since no new trade can be opened. By shrinking the whitelist to just the in-trade pairs, you can improve calculation speeds and reduce CPU usage. When a trade slot is free (either a trade is closed or `max_open_trades` value in config is increased), then the whitelist will return to normal state.
When multiple pairlist filters are being used, it's recommended to put this filter at second position directly below the primary pairlist, so when the trade slots are full, the bot doesn't have to download data for the rest of the filters.
!!! Warning "Backtesting"
`FullTradesFilter` does not support backtesting mode.
#### OffsetFilter
Offsets an incoming pairlist by a given `offset` value.
@@ -366,7 +445,7 @@ If the trading range over the last 10 days is <1% or >99%, remove the pair from
"lookback_days": 10,
"min_rate_of_change": 0.01,
"max_rate_of_change": 0.99,
"refresh_period": 1440
"refresh_period": 86400
}
]
```
@@ -421,7 +500,7 @@ The below example blacklists `BNB/BTC`, uses `VolumePairList` with `20` assets,
@@ -40,9 +40,9 @@ Freqtrade is a free and open source crypto trading bot written in Python. It is
Please read the [exchange specific notes](exchanges.md) to learn about eventual, special configurations needed for each exchange.
- [X] [Binance](https://www.binance.com/)
- [X] [Bittrex](https://bittrex.com/)
- [X] [Bitmart](https://bitmart.com/)
- [X] [Gate.io](https://www.gate.io/ref/6266643)
- [X] [Huobi](http://huobi.com/)
- [X] [HTX](https://www.htx.com/) (Former Huobi)
- [X] [Kraken](https://kraken.com/)
- [X] [OKX](https://okx.com/) (Former OKEX)
- [ ] [potentially many others through <img alt="ccxt" width="30px" src="assets/ccxt-logo.svg" />](https://github.com/ccxt/ccxt/). _(We cannot guarantee they will work)_
@@ -63,6 +63,10 @@ Exchanges confirmed working by the community:
- [X] [Bitvavo](https://bitvavo.com/)
- [X] [Kucoin](https://www.kucoin.com/)
## Community showcase
--8<--"includes/showcase.md"
## Requirements
### Hardware requirements
@@ -79,7 +83,7 @@ To run this bot we recommend you a linux cloud instance with a minimum of:
@@ -24,7 +24,7 @@ The easiest way to install and run Freqtrade is to clone the bot Github reposito
The `stable` branch contains the code of the last release (done usually once per month on an approximately one week old snapshot of the `develop` branch to prevent packaging bugs, so potentially it's more stable).
!!! Note
Python3.8 or higher and the corresponding `pip` are assumed to be available. The install-script will warn you and stop if that's not the case. `git` is also needed to clone the Freqtrade repository.
Python3.9 or higher and the corresponding `pip` are assumed to be available. The install-script will warn you and stop if that's not the case. `git` is also needed to clone the Freqtrade repository.
Also, python headers (`python<yourversion>-dev` / `python<yourversion>-devel`) must be available for the installation to complete successfully.
!!! Warning "Up-to-date clock"
@@ -42,7 +42,7 @@ These requirements apply to both [Script Installation](#script-installation) and
Use this csv-filename to store lookahead-analysis-
results
```
!!! Note ""
The above Output was reduced to options `lookahead-analysis` adds on top of regular backtesting commands.
### Summary
Checks a given strategy for look ahead bias via lookahead-analysis
Look ahead bias means that the backtest uses data from future candles thereby not making it viable beyond backtesting
and producing false hopes for the one backtesting.
### Introduction
Many strategies - without the programmer knowing - have fallen prey to look ahead bias.
Any backtest will populate the full dataframe including all time stamps at the beginning.
If the programmer is not careful or oblivious how things work internally
(which sometimes can be really hard to find out) then it will just look into the future making the strategy amazing
but not realistic.
This command is made to try to verify the validity in the form of the aforementioned look ahead bias.
### How does the command work?
It will start with a backtest of all pairs to generate a baseline for indicators and entries/exits.
After the backtest ran, it will look if the `minimum-trade-amount` is met
and if not cancel the lookahead-analysis for this strategy.
After setting the baseline it will then do additional runs for every entry and exit separately.
When a verification-backtest is done, it will compare the indicators as the signal (either entry or exit) and report the bias.
After all signals have been verified or falsified a result-table will be generated for the user to see.
### Caveats
-`lookahead-analysis` can only verify / falsify the trades it calculated and verified.
If the strategy has many different signals / signal types, it's up to you to select appropriate parameters to ensure that all signals have triggered at least once. Not triggered signals will not have been verified.
This could lead to a false-negative (the strategy will then be reported as non-biased).
-`lookahead-analysis` has access to everything that backtesting has too.
Please don't provoke any configs like enabling position stacking.
If you decide to do so, then make doubly sure that you won't ever run out of `max_open_trades` amount and neither leftover money in your wallet.
This page explains how to validate your strategy for inaccuracies due to recursive issues with certain indicators.
A recursive formula defines any term of a sequence relative to its preceding term(s). An example of a recursive formula is a<sub>n</sub> = a<sub>n-1</sub> + b.
Why does this matter for Freqtrade? In backtesting, the bot will get full data of the pairs according to the timerange specified. But in a dry/live run, the bot will be limited by the amount of data each exchanges gives.
For example, to calculate a very basic indicator called `steps`, the first row's value is always 0, while the following rows' values are equal to the value of the previous row plus 1. If I were to calculate it using the latest 1000 candles, then the `steps` value of the first row is 0, and the `steps` value at the last closed candle is 999.
What happens if the calculation is using only the latest 500 candles? Then instead of 999, the `steps` value at last closed candle is 499. The difference of the value means your backtest result can differ from your dry/live run result.
The `recursive-analysis` command requires historic data to be available. To learn how to get data for the pairs and exchange you're interested in,
head over to the [Data Downloading](data-download.md) section of the documentation.
This command is built upon preparing different lengths of data and calculates indicators based on them.
This does not backtest the strategy itself, but rather only calculates the indicators. After calculating the indicators of different startup candle values (`startup_candle_count`) are done, the values of last rows across all specified `startup_candle_count` are compared to see how much variance they show compared to the base calculation.
Command settings:
- Use the `-p` option to set your desired pair to analyze. Since we are only looking at indicator values, using more than one pair is redundant. Preferably use a pair with a relatively high price and at least moderate volatility, such as BTC or ETH, to avoid rounding issues that can make the results inaccurate. If no pair is set on the command, the pair used for this analysis is the first pair in the whitelist.
- It is recommended to set a long timerange (at least 5000 candles) so that the initial indicators' calculation that is going to be used as a benchmark has very small or no recursive issues itself. For example, for a 5m timeframe, a timerange of 5000 candles would be equal to 18 days.
-`--cache` is forced to "none" to avoid loading previous indicators calculation automatically.
In addition to the recursive formula check, this command also carries out a simple lookahead bias check on the indicator values only. For a full lookahead check, use [Lookahead-analysis](lookahead-analysis.md).
### Why are odd-numbered default startup candles used?
The default value for startup candles are odd numbers. When the bot fetches candle data from the exchange's API, the last candle is the one being checked by the bot and the rest of the data are the "startup candles".
For example, Binance allows 1000 candles per API call. When the bot receives 1000 candles, the last candle is the "current candle", and the preceding 999 candles are the "startup candles". By setting the startup candle count as 1000 instead of 999, the bot will try to fetch 1001 candles instead. The exchange API will then send candle data in a paginated form, i.e. in case of the Binance API, this will be two groups- one of length 1000 and another of length 1. This results in the bot thinking the strategy needs 1001 candles of data, and so it will download 2000 candles worth of data instead, which means there will be 1 "current candle" and 1999 "startup candles".
Furthermore, exchanges limit the number of consecutive bulk API calls, e.g. Binance allows 5 calls. In this case, only 5000 candles can be downloaded from Binance API without hitting the API rate limit, which means the max `startup_candle_count` you can have is 4999.
Please note that this candle limit may be changed in the future by the exchanges without any prior notice.
### How does the command work?
- Firstly an initial indicator calculation is carried out using the supplied timerange to generate a benchmark for indicator values.
- After setting the benchmark it will then carry out additional runs for each of the different startup candle count values.
- The command will then compare the indicator values at the last candle rows and report the differences in a table.
## Understanding the recursive-analysis output
This is an example of an output results table where at least one indicator has a recursive formula issue:
The column headers indicate the different `startup_candle_count` used in the analysis. The values in the table indicate the variance of the calculated indicators compared to the benchmark value.
`nan%` means the value of that indicator cannot be calculated due to lack of data. In this example, you cannot calculate RSI with length 30 with just 21 candles (1 current candle + 20 startup candles).
Users should assess the table per indicator to decide if the specified `startup_candle_count` results in a sufficiently small variance so that the indicator does not have any effect on entries and/or exits.
As such, aiming for absolute zero variance (shown by `-` value) might not be the best option, because some indicators might require you to use such a long `startup_candle_count` to have zero variance.
## Caveats
-`recursive-analysis` will only calculate and compare the indicator values at the last row. The output table reports the percentage differences between the different startup candle count calculations and the original benchmark calculation. Whether it has any actual impact on your entries and exits is not included.
- The ideal scenario is that indicators will have no variance (or at least very close to 0%) despite the startup candle being varied. In reality, indicators such as EMA are using a recursive formula to calculate indicator values, so the goal is not necessarily to have zero percentage variance, but to have the variance low enough (and therefore `startup_candle_count` high enough) that the recursion inherent in the indicator will not have any real impact on trading decisions.
-`recursive-analysis` will only run calculations on `populate_indicators` and `@informative` decorator(s). If you put any indicator calculation on `populate_entry_trend` or `populate_exit_trend`, it won't be calculated.
| `reload_config` | Reloads the configuration file.
| `trades` | List last trades. Limited to 500 trades per call.
| `trade/<tradeid>` | Get specific trade.
| `trade/<tradeid>` | DELETE - Remove trade from the database. Tries to close open orders. Requires manual handling of this trade on the exchange.
| `trade/<tradeid>/open-order` | DELETE - Cancel open order for this trade.
| `trade/<tradeid>/reload` | GET - Reload a trade from the Exchange. Only works in live, and can potentially help recover a trade that was manually sold on the exchange.
| `trades/<tradeid>` | DELETE - Remove trade from the database. Tries to close open orders. Requires manual handling of this trade on the exchange.
| `trades/<tradeid>/open-order` | DELETE - Cancel open order for this trade.
| `trades/<tradeid>/reload` | GET - Reload a trade from the Exchange. Only works in live, and can potentially help recover a trade that was manually sold on the exchange.
| `show_config` | Shows part of the current configuration with relevant settings to operation.
| `logs` | Shows last log messages.
| `status` | Lists all open trades.
| `count` | Displays number of trades used and available.
| `entries [pair]` | Shows profit statistics for each enter tags for given pair (or all pairs if pair isn't given). Pair is optional.
| `exits [pair]` | Shows profit statistics for each exit reasons for given pair (or all pairs if pair isn't given). Pair is optional.
| `mix_tags [pair]` | Shows profit statistics for each combinations of enter tag + exit reasons for given pair (or all pairs if pair isn't given). Pair is optional.
| `locks` | Displays currently locked pairs.
| `delete_lock <lock_id>` | Deletes (disables) the lock by id.
| `profit` | Display a summary of your profit/loss from close trades and some stats about your performance.
We also recommend to set `datadir` to something identifying downloaded data as sandbox data, to avoid having sandbox data mixed with data from the real exchange.
This can be done by adding the `"datadir"` key to the configuration.
Now, whenever you use this configuration, your data directory will be set to this directory.
---
## You should now be ready to test your sandbox
Ensure Freqtrade logs show the sandbox URL, and trades made are shown in sandbox. Also make sure to select a pair which shows at least some decent value (which very often is BTC/<somestablecoin>).
## Common problems with sandbox exchanges
Sandbox exchange instances often have very low volume, which can cause some problems which usually are not seen on a real exchange instance.
### Old Candles problem
Since Sandboxes often have low volume, candles can be quite old and show no volume.
To disable the error "Outdated history for pair ...", best increase the parameter `"outdated_offset"` to a number that seems realistic for the sandbox you're using.
### Unfilled orders
Sandboxes often have very low volumes - which means that many trades can go unfilled, or can go unfilled for a very long time.
To mitigate this, you can try to match the first order on the opposite orderbook side using the following configuration:
``` jsonc
"order_types": {
"entry": "limit",
"exit": "limit"
// ...
},
"entry_pricing": {
"price_side": "other",
// ...
},
"exit_pricing":{
"price_side": "other",
// ...
},
```
The configuration is similar to the suggested configuration for market orders - however by using limit-orders you can avoid moving the price too much, and you can set the worst price you might get.
@@ -164,6 +164,31 @@ E.g. If the `current_rate` is 200 USD, then returning `0.02` will set the stoplo
During backtesting, `current_rate` (and `current_profit`) are provided against the candle's high (or low for short trades) - while the resulting stoploss is evaluated against the candle's low (or high for short trades).
The absolute value of the return value is used (the sign is ignored), so returning `0.05` or `-0.05` have the same result, a stoploss 5% below the current price.
Returning None will be interpreted as "no desire to change", and is the only safe way to return when you'd like to not modify the stoploss.
Stoploss on exchange works similar to `trailing_stop`, and the stoploss on exchange is updated as configured in `stoploss_on_exchange_interval` ([More details about stoploss on exchange](stoploss.md#stop-loss-on-exchange-freqtrade)).
!!! Note "Use of dates"
All time-based calculations should be done based on `current_time` - using `datetime.now()` or `datetime.utcnow()` is discouraged, as this will break backtesting support.
!!! Tip "Trailing stoploss"
It's recommended to disable `trailing_stop` when using custom stoploss values. Both can work in tandem, but you might encounter the trailing stop to move the price higher while your custom function would not want this, causing conflicting behavior.
### Adjust stoploss after position adjustments
Depending on your strategy, you may encounter the need to adjust the stoploss in both directions after a [position adjustment](#adjust-trade-position).
For this, freqtrade will make an additional call with `after_fill=True` after an order fills, which will allow the strategy to move the stoploss in any direction (also widening the gap between stoploss and current price, which is otherwise forbidden).
!!! Note "backwards compatibility"
This call will only be made if the `after_fill` parameter is part of the function definition of your `custom_stoploss` function.
As such, this will not impact (and with that, surprise) existing, running strategies.
### Custom stoploss examples
The next section will show some examples on what's possible with the custom stoploss function.
Of course, many more things are possible, and all examples can be combined at will.
#### Trailing stop via custom stoploss
To simulate a regular trailing stoploss of 4% (trailing 4% behind the maximum reached price) you would use the following very simple method:
@@ -179,7 +204,8 @@ class AwesomeStrategy(IStrategy):
Custom stoploss logic, returning the new distance relative to current_rate (as ratio).
e.g. returning -0.05 would create a stoploss 5% below current_rate.
@@ -187,7 +213,7 @@ class AwesomeStrategy(IStrategy):
For full documentation please go to https://www.freqtrade.io/en/latest/strategy-advanced/
When not implemented by a strategy, returns the initial stoploss value
When not implemented by a strategy, returns the initial stoploss value.
Only called when use_custom_stoploss is set to True.
:param pair: Pair that's currently analyzed
@@ -195,25 +221,13 @@ class AwesomeStrategy(IStrategy):
:param current_time: datetime object, containing the current datetime
:param current_rate: Rate, calculated based on pricing settings in exit_pricing.
:param current_profit: Current profit (as ratio), calculated based on current_rate.
:param after_fill: True if the stoploss is called after the order was filled.
:param **kwargs: Ensure to keep this here so updates to this won't break your strategy.
:return float: New stoploss value, relative to the currentrate
:return float: New stoploss value, relative to the current_rate
"""
return -0.04
```
Stoploss on exchange works similar to `trailing_stop`, and the stoploss on exchange is updated as configured in `stoploss_on_exchange_interval` ([More details about stoploss on exchange](stoploss.md#stop-loss-on-exchange-freqtrade)).
!!! Note "Use of dates"
All time-based calculations should be done based on `current_time` - using `datetime.now()` or `datetime.utcnow()` is discouraged, as this will break backtesting support.
!!! Tip "Trailing stoploss"
It's recommended to disable `trailing_stop` when using custom stoploss values. Both can work in tandem, but you might encounter the trailing stop to move the price higher while your custom function would not want this, causing conflicting behavior.
### Custom stoploss examples
The next section will show some examples on what's possible with the custom stoploss function.
Of course, many more things are possible, and all examples can be combined at will.
#### Time based trailing stop
Use the initial stoploss for the first 60 minutes, after this change to 10% trailing stoploss, and after 2 hours (120 minutes) we use a 5% trailing stoploss.
@@ -229,14 +243,45 @@ class AwesomeStrategy(IStrategy):
#### Time based trailing stop with after-fill adjustments
Use the initial stoploss for the first 60 minutes, after this change to 10% trailing stoploss, and after 2 hours (120 minutes) we use a 5% trailing stoploss.
If an additional order fills, set stoploss to -10% below the new `open_rate` ([Averaged across all entries](#position-adjust-calculations)).
# return maximum stoploss value, keeping current stoploss price unchanged
return 1
return None
```
See [Dataframe access](strategy-advanced.md#dataframe-access) for more information about dataframe use in strategy callbacks.
@@ -364,15 +413,89 @@ See [Dataframe access](strategy-advanced.md#dataframe-access) for more informati
#### Stoploss relative to open price
Stoploss values returned from `custom_stoploss()` always specify a percentage relative to `current_rate`. In order to set a stoploss relative to the *open* price, we need to use `current_profit` to calculate what percentage relative to the `current_rate` will give you the same result as if the percentage was specified from the open price.
Stoploss values returned from `custom_stoploss()` must specify a percentage relative to `current_rate`, but sometimes you may want to specify a stoploss relative to the _entry_ price instead.
`stoploss_from_open()` is a helper function to calculate a stoploss value that can be returned from `custom_stoploss` which will be equivalent to the desired trade profit above the entry point.
The helper function [`stoploss_from_open()`](strategy-customization.md#stoploss_from_open) can be used to convert from an open price relative stop, to a current price relative stop which can be returned from `custom_stoploss()`.
??? Example "Returning a stoploss relative to the open price from the customstoploss function"
Say the open price was $100, and `current_price` is $121 (`current_profit` will be `0.21`).
If we want a stop price at 7% above the open price we can call `stoploss_from_open(0.07, current_profit, False)` which will return `0.1157024793`. 11.57% below $121 is $107, which is the same as 7% above $100.
This function will consider leverage - so at 10x leverage, the actual stoploss would be 0.7% above $100 (0.7% * 10x = 7%).
``` python
from datetime import datetime
from freqtrade.persistence import Trade
from freqtrade.strategy import IStrategy, stoploss_from_open
Full examples can be found in the [Custom stoploss](strategy-advanced.md#custom-stoploss) section of the Documentation.
!!! Note
Providing invalid input to `stoploss_from_open()` may produce "CustomStoploss function did not return valid stoploss" warnings.
This may happen if `current_profit` parameter is below specified `open_relative_stop`. Such situations may arise when closing trade
is blocked by `confirm_trade_exit()` method. Warnings can be solved by never blocking stop loss sells by checking `exit_reason` in
`confirm_trade_exit()`, or by using `return stoploss_from_open(...) or 1` idiom, which will request to not change stop loss when
`current_profit <open_relative_stop`.
#### Stoploss percentage from absolute price
Stoploss values returned from `custom_stoploss()` always specify a percentage relative to `current_rate`. In order to set a stoploss at specified absolute price level, we need to use `stop_rate` to calculate what percentage relative to the `current_rate` will give you the same result as if the percentage was specified from the open price.
The helper function [`stoploss_from_absolute()`](strategy-customization.md#stoploss_from_absolute) can be used to convert from an absolute price, to a current price relative stop which can be returned from `custom_stoploss()`.
The helper function `stoploss_from_absolute()` can be used to convert from an absolute price, to a current price relative stop which can be returned from `custom_stoploss()`.
??? Example "Returning a stoploss using absolute price from the custom stoploss function"
If we want to trail a stop price at 2xATR below current price we can call `stoploss_from_absolute(current_rate+(side* candle['atr'] *2),current_rate,is_short=trade.is_short,leverage=trade.leverage)`.
For futures, we need to adjust the direction (up or down), as well as adjust for leverage, since the [`custom_stoploss`](strategy-callbacks.md#custom-stoploss) callback returns the ["risk for this trade"](stoploss.md#stoploss-and-leverage) - not the relative price movement.
``` python
from datetime import datetime
from freqtrade.persistence import Trade
from freqtrade.strategy import IStrategy, stoploss_from_absolute, timeframe_to_prev_date
@@ -387,6 +510,9 @@ Each of these methods are called right before placing an order on the exchange.
!!! Note
If your custom pricing function return None or an invalid value, price will fall back to `proposed_rate`, which is based on the regular pricing configuration.
!!! Note
Using custom_entry_price, the Trade object will be available as soon as the first entry order associated with the trade is created, for the first entry, `trade` parameter value will be `None`.
### Custom order entry and exit price example
``` python
@@ -397,7 +523,7 @@ class AwesomeStrategy(IStrategy):
@@ -634,26 +760,38 @@ The `position_adjustment_enable` strategy property enables the usage of `adjust_
For performance reasons, it's disabled by default and freqtrade will show a warning message on startup if enabled.
`adjust_trade_position()` can be used to perform additional orders, for example to manage risk with DCA (Dollar Cost Averaging) or to increase or decrease positions.
`max_entry_position_adjustment` property is used to limit the number of additional buys per trade (on top of the first buy) that the bot can execute. By default, the value is -1 which means the bot have no limit on number of adjustment buys.
The strategy is expected to return a stake_amount (in stake currency) between `min_stake` and `max_stake` if and when an additional buy order should be made (position is increased).
If there are not enough funds in the wallet (the return value is above `max_stake`) then the signal will be ignored.
Additional orders also result in additional fees and those orders don't count towards `max_open_trades`.
This callback is **not** called when there is an open order (either buy or sell) waiting for execution.
`adjust_trade_position()` is called very frequently for the duration of a trade, so you must keep your implementation as performant as possible.
Additional Buys are ignored once you have reached the maximum amount of extra buys that you have set on `max_entry_position_adjustment`, but the callback is called anyway looking for partial exits.
Position adjustments will always be applied in the direction of the trade, so a positive value will always increase your position (negative values will decrease your position), no matter if it's a long or short trade.
Adjustment orders can be assigned with a tag by returning a 2 element Tuple, with the first element being the adjustment amount, and the 2nd element the tag (e.g. `return250,'increase_favorable_conditions'`).
Position adjustments will always be applied in the direction of the trade, so a positive value will always increase your position (negative values will decrease your position), no matter if it's a long or short trade. Modifications to leverage are not possible, and the stake-amount is assumed to be before applying leverage.
Modifications to leverage are not possible, and the stake-amount returned is assumed to be before applying leverage.
### Increase position
The strategy is expected to return a positive **stake_amount** (in stake currency) between `min_stake` and `max_stake` if and when an additional entry order should be made (position is increased -> buy order for long trades, sell order for short trades).
If there are not enough funds in the wallet (the return value is above `max_stake`) then the signal will be ignored.
`max_entry_position_adjustment` property is used to limit the number of additional entries per trade (on top of the first entry order) that the bot can execute. By default, the value is -1 which means the bot have no limit on number of adjustment entries.
Additional entries are ignored once you have reached the maximum amount of extra entries that you have set on `max_entry_position_adjustment`, but the callback is called anyway looking for partial exits.
### Decrease position
The strategy is expected to return a negative stake_amount (in stake currency) for a partial exit.
Returning the full owned stake at that point (based on the current price) (`-(trade.amount/trade.leverage)*current_exit_rate`) results in a full exit.
Returning a value more than the above (so remaining stake_amount would become negative) will result in the bot ignoring the signal.
!!! Note "About stake size"
Using fixed stake size means it will be the amount used for the first order, just like without position adjustment.
If you wish to buy additional orders with DCA, then make sure to leave enough funds in the wallet for that.
Using 'unlimited' stake amount with DCA orders requires you to also implement the `custom_stake_amount()` callback to avoid allocating all funds to the initial order.
!!! Warning
!!! Warning "Stoploss calculation"
Stoploss is still calculated from the initial opening price, not averaged price.
Regular stoploss rules still apply (cannot move down).
@@ -663,6 +801,11 @@ Position adjustments will always be applied in the direction of the trade, so a
During backtesting this callback is called for each candle in `timeframe` or `timeframe_detail`, so run-time performance will be affected.
This can also cause deviating results between live and backtesting, since backtesting can adjust the trade only once per candle, whereas live could adjust the trade multiple times per candle.
!!! Warning "Performance with many position adjustments"
Position adjustments can be a good approach to increase a strategy's output - but it can also have drawbacks if using this feature extensively.
Each of the orders will be attached to the trade object for the duration of the trade - hence increasing memory usage.
Trades with long duration and 10s or even 100ds of position adjustments are therefore not recommended, and should be closed at regular intervals to not affect performance.
``` python
from freqtrade.persistence import Trade
@@ -696,11 +839,12 @@ class DigDeeperStrategy(IStrategy):
@@ -793,6 +939,8 @@ Returning any other price will cancel the existing order, and replace it with a
The trade open-date (`trade.open_date_utc`) will remain at the time of the very first order placed.
Please make sure to be aware of this - and eventually adjust your logic in other callbacks to account for this, and use the date of the first filled order instead.
If the cancellation of the original order fails, then the order will not be replaced - though the order will most likely have been canceled on exchange. Having this happen on initial entries will result in the deletion of the order, while on position adjustment orders, it'll result in the trade size remaining as is.
!!! Warning "Regular timeout"
Entry `unfilledtimeout` mechanism (as well as `check_entry_timeout()`) takes precedence over this.
Entry Orders that are cancelled via the above methods will not have this callback called. Be sure to update timeout values to match your expectations.
Additional technical libraries can be installed as necessary, or custom indicators may be written / invented by the strategy author.
@@ -168,7 +168,9 @@ Most indicators have an instable startup period, in which they are either not av
To account for this, the strategy can be assigned the `startup_candle_count` attribute.
This should be set to the maximum number of candles that the strategy requires to calculate stable indicators. In the case where a user includes higher timeframes with informative pairs, the `startup_candle_count` does not necessarily change. The value is the maximum period (in candles) that any of the informatives timeframes need to compute stable indicators.
In this example strategy, this should be set to 100 (`startup_candle_count = 100`), since the longest needed history is 100 candles.
You can use [recursive-analysis](recursive-analysis.md) to check and find the correct `startup_candle_count` to be used.
In this example strategy, this should be set to 400 (`startup_candle_count = 400`), since the minimum needed history for ema100 calculation to make sure the value is correct is 400 candles.
Assuming `startup_candle_count` is set to 100, backtesting knows it needs 100 candles to generate valid buy signals. It will load data from `20190101 - (100 * 5m)` - which is ~2018-12-31 15:30:00.
Assuming `startup_candle_count` is set to 400, backtesting knows it needs 400 candles to generate valid buy signals. It will load data from `20190101 - (400 * 5m)` - which is ~2018-12-30 11:40:00.
If this data is available, indicators will be calculated with this extended timerange. The instable startup period (up to 2019-01-01 00:00:00) will then be removed before starting backtesting.
!!! Note
If data for the startup period is not available, then the timerange will be adjusted to account for this startup period - so Backtesting would start at 2019-01-01 08:30:00.
If data for the startup period is not available, then the timerange will be adjusted to account for this startup period - so Backtesting would start at 2019-01-02 09:20:00.
Edit the method `populate_exit_trend()` into your strategy file to update your exit strategy.
The exit-signal is only used for exits if `use_exit_signal` is set to true in the configuration.
The exit-signal can be suppressed by setting `use_exit_signal` to false in the configuration or strategy.
`use_exit_signal` will not influence [signal collision rules](#colliding-signals) - which will still apply and can prevent entries.
It's important to always return the dataframe without removing/modifying the columns `"open", "high", "low", "close", "volume"`, otherwise these fields would contain something unexpected.
@@ -342,16 +344,12 @@ The above configuration would therefore mean:
The calculation does include fees.
To disable ROI completely, set it to an insanely high number:
To disable ROI completely, set it to an empty dictionary:
```python
minimal_roi = {
"0": 100
}
minimal_roi = {}
```
While technically not completely disabled, this would exit once the trade reaches 10000% Profit.
To use times based on candle duration (timeframe), the following snippet can be handy.
This will allow you to change the timeframe for the strategy, and ROI times will still be set as candles (e.g. after 3 candles ...)
@@ -369,6 +367,11 @@ class AwesomeStrategy(IStrategy):
}
```
??? info "Orders that don't fill immediately"
`minimal_roi` will take the `trade.open_date` as reference, which is the time the trade was initialized / the first order for this trade was placed.
This will also hold true for limit orders that don't fill immediately (usually in combination with "off-spot" prices through `custom_entry_price()`), as well as for cases where the initial order is replaced through `adjust_entry_price()`.
The time used will still be from the initial `trade.open_date` (when the initial order was first placed), not from the newly placed order date.
### Stoploss
Setting a stoploss is highly recommended to protect your capital from strong moves against you.
@@ -488,17 +491,18 @@ for more information.
:param timeframe: Informative timeframe. Must always be equal or higher than strategy timeframe.
:param asset: Informative asset, for example BTC, BTC/USDT, ETH/BTC. Do not specify to use
current pair.
current pair. Also supports limited pair format strings (see below)
:param fmt: Column format (str) or column formatter (callable(name, asset, timeframe)). When not
specified, defaults to:
* {base}_{quote}_{column}_{timeframe} if asset is specified.
* {base}_{quote}_{column}_{timeframe} if asset is specified.
* {column}_{timeframe} if asset is not specified.
Format string supports these format variables:
* {asset} - full name of the asset, for example 'BTC/USDT'.
Pair format supports these format variables:
* {base} - base currency in lower case, for example 'eth'.
* {BASE} - same as {base}, except in upper case.
* {quote} - quote currency in lower case, for example 'usdt'.
* {QUOTE} - same as {quote}, except in upper case.
Format string additionally supports this variables.
* {asset} - full name of the asset, for example 'BTC/USDT'.
* {column} - name of dataframe column.
* {timeframe} - timeframe of informative dataframe.
:param ffill: ffill dataframe after merging informative pair.
@@ -590,6 +594,67 @@ for more information.
will overwrite previously defined method and not produce any errors due to limitations of Python programming language. In such cases you will find that indicators
created in earlier-defined methods are not available in the dataframe. Carefully review method names and make sure they are unique!
### *merge_informative_pair()*
This method helps you merge an informative pair to a regular dataframe without lookahead bias.
It's there to help you merge the dataframe in a safe and consistent way.
Options:
- Rename the columns for you to create unique columns
- Merge the dataframe without lookahead bias
- Forward-fill (optional)
For a full sample, please refer to the [complete data provider example](#complete-data-provider-sample) below.
All columns of the informative dataframe will be available on the returning dataframe in a renamed fashion:
!!! Example "Column renaming"
Assuming `inf_tf = '1d'` the resulting columns will be:
``` python
'date', 'open', 'high', 'low', 'close', 'rsi' # from the original dataframe
'date_1d', 'open_1d', 'high_1d', 'low_1d', 'close_1d', 'rsi_1d' # from the informative dataframe
```
??? Example "Column renaming - 1h"
Assuming `inf_tf = '1h'` the resulting columns will be:
``` python
'date', 'open', 'high', 'low', 'close', 'rsi' # from the original dataframe
'date_1h', 'open_1h', 'high_1h', 'low_1h', 'close_1h', 'rsi_1h' # from the informative dataframe
```
??? Example "Custom implementation"
A custom implementation for this is possible, and can be done as follows:
``` python
# Shift date by 1 candle
# This is necessary since the data is always the "open date"
# and a 15m candle starting at 12:15 should not know the close of the 1h candle from 12:00 to 13:00
# FFill to have the 1d value available in every row throughout the day.
# Without this, comparisons would only work once per day.
dataframe = dataframe.ffill()
```
!!! Warning "Informative timeframe < timeframe"
Using informative timeframes smaller than the dataframe timeframe is not recommended with this method, as it will not use any of the additional information this would provide.
To use the more detailed information properly, more advanced methods should be applied (which are out of scope for freqtrade documentation, as it'll depend on the respective need).
## Additional data (DataProvider)
The strategy provides access to the `DataProvider`. This allows you to get additional data to use in your strategy.
@@ -814,146 +879,6 @@ class SampleStrategy(IStrategy):
***
## Helper functions
### *merge_informative_pair()*
This method helps you merge an informative pair to a regular dataframe without lookahead bias.
It's there to help you merge the dataframe in a safe and consistent way.
Options:
- Rename the columns for you to create unique columns
- Merge the dataframe without lookahead bias
- Forward-fill (optional)
For a full sample, please refer to the [complete data provider example](#complete-data-provider-sample) below.
All columns of the informative dataframe will be available on the returning dataframe in a renamed fashion:
!!! Example "Column renaming"
Assuming `inf_tf = '1d'` the resulting columns will be:
``` python
'date', 'open', 'high', 'low', 'close', 'rsi' # from the original dataframe
'date_1d', 'open_1d', 'high_1d', 'low_1d', 'close_1d', 'rsi_1d' # from the informative dataframe
```
??? Example "Column renaming - 1h"
Assuming `inf_tf = '1h'` the resulting columns will be:
``` python
'date', 'open', 'high', 'low', 'close', 'rsi' # from the original dataframe
'date_1h', 'open_1h', 'high_1h', 'low_1h', 'close_1h', 'rsi_1h' # from the informative dataframe
```
??? Example "Custom implementation"
A custom implementation for this is possible, and can be done as follows:
``` python
# Shift date by 1 candle
# This is necessary since the data is always the "open date"
# and a 15m candle starting at 12:15 should not know the close of the 1h candle from 12:00 to 13:00
# FFill to have the 1d value available in every row throughout the day.
# Without this, comparisons would only work once per day.
dataframe = dataframe.ffill()
```
!!! Warning "Informative timeframe < timeframe"
Using informative timeframes smaller than the dataframe timeframe is not recommended with this method, as it will not use any of the additional information this would provide.
To use the more detailed information properly, more advanced methods should be applied (which are out of scope for freqtrade documentation, as it'll depend on the respective need).
***
### *stoploss_from_open()*
Stoploss values returned from `custom_stoploss` must specify a percentage relative to `current_rate`, but sometimes you may want to specify a stoploss relative to the entry point instead. `stoploss_from_open()` is a helper function to calculate a stoploss value that can be returned from `custom_stoploss` which will be equivalent to the desired trade profit above the entry point.
??? Example "Returning a stoploss relative to the open price from the custom stoploss function"
Say the open price was $100, and `current_price` is $121 (`current_profit` will be `0.21`).
If we want a stop price at 7% above the open price we can call `stoploss_from_open(0.07, current_profit, False)` which will return `0.1157024793`. 11.57% below $121 is $107, which is the same as 7% above $100.
This function will consider leverage - so at 10x leverage, the actual stoploss would be 0.7% above $100 (0.7% * 10x = 7%).
``` python
from datetime import datetime
from freqtrade.persistence import Trade
from freqtrade.strategy import IStrategy, stoploss_from_open
Full examples can be found in the [Custom stoploss](strategy-advanced.md#custom-stoploss) section of the Documentation.
!!! Note
Providing invalid input to `stoploss_from_open()` may produce "CustomStoploss function did not return valid stoploss" warnings.
This may happen if `current_profit` parameter is below specified `open_relative_stop`. Such situations may arise when closing trade
is blocked by `confirm_trade_exit()` method. Warnings can be solved by never blocking stop loss sells by checking `exit_reason` in
`confirm_trade_exit()`, or by using `return stoploss_from_open(...) or 1` idiom, which will request to not change stop loss when
`current_profit <open_relative_stop`.
### *stoploss_from_absolute()*
In some situations it may be confusing to deal with stops relative to current rate. Instead, you may define a stoploss level using an absolute price.
??? Example "Returning a stoploss using absolute price from the custom stoploss function"
If we want to trail a stop price at 2xATR below current price we can call `stoploss_from_absolute(current_rate-(candle['atr']*2),current_rate,is_short=trade.is_short)`.
``` python
from datetime import datetime
from freqtrade.persistence import Trade
from freqtrade.strategy import IStrategy, stoploss_from_absolute
The strategy provides access to the `wallets` object. This contains the current balances on the exchange.
@@ -1084,11 +1009,15 @@ This is a common pain-point, which can cause huge differences between backtestin
The following lists some common patterns which should be avoided to prevent frustration:
- don't use `shift(-1)`. This uses data from the future, which is not available.
- don't use `.iloc[-1]` or any other absolute position in the dataframe, this will be different between dry-run and backtesting.
- don't use `shift(-1)` or other negative values. This uses data from the future in backtesting, which is not available in dry or live modes.
- don't use `.iloc[-1]` or any other absolute position in the dataframe within `populate_` functions, as this will be different between dry-run and backtesting. Absolute `iloc` indexing is safe to use in callbacks however - see [Strategy Callbacks](strategy-callbacks.md).
- don't use `dataframe['volume'].mean()`. This uses the full DataFrame for backtesting, including data from the future. Use `dataframe['volume'].rolling(<window>).mean()` instead
- don't use `.resample('1h')`. This uses the left border of the interval, so moves data from an hour to the start of the hour. Use `.resample('1h', label='right')` instead.
!!! Tip "Identifying problems"
You may also want to check the 2 helper commands [lookahead-analysis](lookahead-analysis.md) and [recursive-analysis](recursive-analysis.md), which can each help you figure out problems with your strategy in different ways.
Please treat them as what they are - helpers to identify most common problems. A negative result of each does not guarantee that there's none of the above errors included.
### Colliding signals
When conflicting signals collide (e.g. both `'enter_long'` and `'exit_long'` are 1), freqtrade will do nothing and ignore the entry signal. This will avoid trades that enter, and exit immediately. Obviously, this can potentially lead to missed entries.
1. Features - Move to `feature_engineering_expand_all`
2. Basic features, not expanded across `include_periods_candles` - move to`feature_engineering_expand_basic()`.
2. Basic features, not expanded across `indicator_periods_candles` - move to`feature_engineering_expand_basic()`.
3. Standard features which should not be expanded - move to `feature_engineering_standard()`.
4. Targets - Move this part to `set_freqai_targets()`.
@@ -728,3 +729,86 @@ Targets now get their own, dedicated method.
return dataframe
```
### FreqAI - New data Pipeline
If you have created your own custom `IFreqaiModel` with a custom `train()`/`predict()` function, *and* you still rely on `data_cleaning_train/predict()`, then you will need to migrate to the new pipeline. If your model does *not* rely on `data_cleaning_train/predict()`, then you do not need to worry about this migration. That means that this migration guide is relevant for a very small percentage of power-users. If you stumbled upon this guide by mistake, feel free to inquire in depth about your problem in the Freqtrade discord server.
The conversion involves first removing `data_cleaning_train/predict()` and replacing them with a `define_data_pipeline()` and `define_label_pipeline()` function to your `IFreqaiModel` class:
1. Data normalization and cleaning is now homogenized with the new pipeline definition. This is created in the new `define_data_pipeline()` and `define_label_pipeline()` functions. The `data_cleaning_train()` and `data_cleaning_predict()` functions are no longer used. You can override `define_data_pipeline()` to create your own custom pipeline if you wish.
2. Data normalization and cleaning is now homogenized with the new pipeline definition. This is created in the new `define_data_pipeline()` and `define_label_pipeline()` functions. The `data_cleaning_train()` and `data_cleaning_predict()` functions are no longer used. You can override `define_data_pipeline()` to create your own custom pipeline if you wish.
3. Data denormalization is done with the new pipeline. Replace this with the lines below.
@@ -175,6 +175,7 @@ official commands. You can ask at any moment for help with `/help`.
| `/status` | Lists all open trades
| `/status <trade_id>` | Lists one or more specific trade. Separate multiple <trade_id> with a blank space.
| `/status table` | List all open trades in a table format. Pending buy orders are marked with an asterisk (*) Pending sell orders are marked with a double asterisk (**)
| `/order <trade_id>` | Lists orders of one or more specific trade. Separate multiple <trade_id> with a blank space.
| `/trades [limit]` | List all recently closed trades in a table format.
| `/count` | Displays number of trades used and available
| `/locks` | Show currently locked pairs.
@@ -287,12 +288,17 @@ Return a summary of your profit/loss and performance.
> **Best Performing:** `PAY/BTC: 50.23%`
> **Trading volume:** `0.5 BTC`
> **Profit factor:** `1.04`
> **Win / Loss:** `102 / 36`
> **Winrate:** `73.91%`
> **Expectancy (Ratio):** `4.87 (1.66)`
> **Max Drawdown:** `9.23% (0.01255 BTC)`
The relative profit of `1.2%` is the average profit per trade.
The relative profit of `15.2 Σ%` is be based on the starting capital - so in this case, the starting capital was `0.00485701 * 1.152 = 0.00738 BTC`.
Starting capital is either taken from the `available_capital` setting, or calculated by using current wallet size - profits.
Profit Factor is calculated as gross profits / gross losses - and should serve as an overall metric for the strategy.
Expectancy corresponds to the average return per currency unit at risk, i.e. the winrate and the risk-reward ratio (the average gain of winning trades compared to the average loss of losing trades).
Expectancy Ratio is expected profit or loss of a subsequent trade based on the performance of all past trades.
Max drawdown corresponds to the backtesting metric `Absolute Drawdown (Account)` - calculated as `(Absolute Drawdown) / (DrawdownHigh + startingBalance)`.
Bot started date will refer to the date the bot was first started. For older bots, this will default to the first trade's open date.
@@ -141,7 +141,8 @@ Most properties here can be None as they are dependant on the exchange response.
`amount` | float | Amount in base currency
`filled` | float | Filled amount (in base currency)
`remaining` | float | Remaining amount
`cost` | float | Cost of the order - usually average * filled
`cost` | float | Cost of the order - usually average * filled (*Exchange dependant on futures, may contain the cost with or without leverage and may be in contracts.*)
`stake_amount` | float | Stake amount used for this order. *Added in 2023.7.*
`order_date` | datetime | Order creation date **use `order_date_utc` instead**
`order_date_utc` | datetime | Order creation date (in UTC)
`order_fill_date` | datetime | Order fill date **use `order_fill_utc` instead**
@@ -80,12 +80,18 @@ When using the Form-Encoded or JSON-Encoded configuration you can configure any
The result would be a POST request with e.g. `Status: running` body and `Content-Type: text/plain` header.
Optional parameters are available to enable automatic retries for webhook messages. The `webhook.retries` parameter can be set for the maximum number of retries the webhook request should attempt if it is unsuccessful (i.e. HTTP response status is not 200). By default this is set to `0` which is disabled. An additional `webhook.retry_delay` parameter can be set to specify the time in seconds between retry attempts. By default this is set to `0.1` (i.e. 100ms). Note that increasing the number of retries or retry delay may slow down the trader if there are connectivity issues with the webhook. Example configuration for retries:
## Additional configurations
The `webhook.retries` parameter can be set for the maximum number of retries the webhook request should attempt if it is unsuccessful (i.e. HTTP response status is not 200). By default this is set to `0` which is disabled. An additional `webhook.retry_delay` parameter can be set to specify the time in seconds between retry attempts. By default this is set to `0.1` (i.e. 100ms). Note that increasing the number of retries or retry delay may slow down the trader if there are connectivity issues with the webhook.
You can also specify `webhook.timeout` - which defines how long the bot will wait until it assumes the other host as unresponsive (defaults to 10s).
Example configuration for retries:
```json
"webhook":{
"enabled":true,
"url":"https://<YOURHOOKURL>",
"timeout":10,
"retries":3,
"retry_delay":0.2,
"status":{
@@ -109,6 +115,8 @@ Custom messages can be sent to Webhook endpoints via the `self.dp.send_msg()` fu
Different payloads can be configured for different events. Not all fields are necessary, but you should configure at least one of the dicts, otherwise the webhook will never be called.
## Webhook Message types
### Entry
The fields in `webhook.entry` are filled when the bot executes a long/short. Parameters are filled using string.format.
@@ -126,6 +134,7 @@ Possible parameters are:
*`stake_amount`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`order_type`
*`current_rate`
@@ -147,6 +156,7 @@ Possible parameters are:
*`stake_amount`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`order_type`
*`current_rate`
@@ -168,6 +178,7 @@ Possible parameters are:
*`stake_amount`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`order_type`
*`current_rate`
@@ -191,6 +202,7 @@ Possible parameters are:
*`profit_ratio`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`exit_reason`
*`order_type`
@@ -216,6 +228,7 @@ Possible parameters are:
*`profit_ratio`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`exit_reason`
*`order_type`
@@ -241,6 +254,7 @@ Possible parameters are:
*`profit_ratio`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`exit_reason`
*`order_type`
@@ -294,6 +308,7 @@ You can configure this as follows:
```
The above represents the default (`exit_fill` and `entry_fill` are optional and will default to the above configuration) - modifications are obviously possible.
To disable either of the two default values (`entry_fill` / `exit_fill`), you can assign them an empty array (`exit_fill: []`).
Available fields correspond to the fields for webhooks and are documented in the corresponding webhook sections.
Install ta-lib according to the [ta-lib documentation](https://github.com/mrjbq7/ta-lib#windows).
Install ta-lib according to the [ta-lib documentation](https://github.com/TA-Lib/ta-lib-python#windows).
As compiling from source on windows has heavy dependencies (requires a partial visual studio installation), Freqtrade provides these dependencies (in the binary wheel format) for the latest 3 Python versions (3.8, 3.9, 3.10 and 3.11) and for 64bit Windows.
As compiling from source on windows has heavy dependencies (requires a partial visual studio installation), Freqtrade provides these dependencies (in the binary wheel format) for the latest 3 Python versions (3.9, 3.10 and 3.11) and for 64bit Windows.
These Wheels are also used by CI running on windows, and are therefore tested together with freqtrade.
Other versions must be downloaded from the above link.
``` powershell
cd \path\freqtrade
python -m venv .env
.env\Scripts\activate.ps1
python -m venv .venv
.venv\Scripts\activate.ps1
# optionally install ta-lib from wheel
# Eventually adjust the below filename to match the downloaded wheel
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
