This will explicitly fail if a file (or an invalid symlink) is present.
Freqtrade requires these files to be valid files - so failing here is correct behavior.
closes#10720
We can't rely on the "trades" subarray from the order
to contain everything we need
As such, we need to ensure that required properties are present and not None.
closes#10398
The help command doesn't have consistent formatting.
- /stop: "Description" doesn't conform to the other formatting.
- Statistics header isn't on it's own line.
> The built-in version is signed by Microsoft while newer versions are community-maintained and signed with a different certificate, causing Install-Module to sometimes throw a error requiring us to accept the new publisher certificate.
source: https://pester.dev/docs/introduction/installation#windows
- some work on all pairs, and we don't check protections either so ... just disable them completely
- added info in the docs
Changed pairs-check to if no definition is in the config (but it s maybe in the strategy) it will just force-set it to the proper amount of len(config['pairs']
No default value is specified in the docs for the processing_mode, making it unclear that the default behaviour is to filter out pairs, rather than append.
Fix logical error in the conditional checks for model classes. The `elif` statement that looks for "lightgbm.sklearn" or "xgb" in the model class string is now broken into two separate conditions because the old condition would always evaluate to `True` due to the non-empty string "xgb".
No default value is specified in the docs for the processing_mode, making it unclear that the default behaviour is to filter out pairs, rather than append.
Fix logical error in the conditional checks for model classes. The `elif` statement that looks for "lightgbm.sklearn" or "xgb" in the model class string is now broken into two separate conditions because the old condition would always evaluate to `True` due to the non-empty string "xgb".
We receive a lot of code that fails the `ruff` checks.
@@ -72,12 +72,12 @@ you can manually run pre-commit with `pre-commit run -a`.
mypy freqtrade
```
### 4. Ensure all imports are correct
### 4. Ensure formatting is correct
#### Run isort
#### Run ruff
``` bash
isort .
ruff format .
```
## (Core)-Committer Guide
@@ -125,7 +125,7 @@ Exceptions:
Contributors may be given commit privileges. Preference will be given to those with:
1. Past contributions to Freqtrade and other related open-source projects. Contributions to Freqtrade include both code (both accepted and pending) and friendly participation in the issue tracker and Pull request reviews. Quantity and quality are considered.
1. Past contributions to Freqtrade and other related open-source projects. Contributions to Freqtrade include both code (both accepted and pending) and friendly participation in the issue tracker and Pull request reviews. Both quantity and quality are considered.
1. A coding style that the other core committers find simple, minimal, and clean.
1. Access to resources for cross-platform development and testing.
This will tell freqtrade to output a pickled dictionary of strategy, pairs and corresponding
DataFrame of the candles that resulted in buy signals. Depending on how many buys your strategy
makes, this file may get quite large, so periodically check your `user_data/backtest_results`
folder to delete old exports.
DataFrame of the candles that resulted in entry and exit signals.
Depending on how many entries your strategy makes, this file may get quite large, so periodically check your `user_data/backtest_results` folder to delete old exports.
Before running your next backtest, make sure you either delete your old backtest results or run
backtesting with the `--cache none` option to make sure no cached results are used.
If all goes well, you should now see a `backtest-result-{timestamp}_signals.pkl` file in the
`user_data/backtest_results` folder.
If all goes well, you should now see a `backtest-result-{timestamp}_signals.pkl` and `backtest-result-{timestamp}_exited.pkl` files in the `user_data/backtest_results` folder.
To analyze the entry/exit tags, we now need to use the `freqtrade backtesting-analysis` command
with `--analysis-groups` option provided with space-separated arguments:
`min_rate`, `max_rate`, `is_open`, `enter_tag`, `leverage`, `is_short`, `open_timestamp`, `close_timestamp` and `orders`
#### Filtering Indicators Based on Entry or Exit Signals
The `--indicator-list` option, by default, displays indicator values for both entry and exit signals. To filter the indicator values exclusively for entry signals, you can use the `--entry-only` argument. Similarly, to display indicator values only at exit signals, use the `--exit-only` argument.
Example: Display indicator values at entry signals:
This guide walks you through utilizing public trade data for advanced orderflow analysis in Freqtrade.
!!! Warning "Experimental Feature"
The orderflow feature is currently in beta and may be subject to changes in future releases. Please report any issues or feedback on the [Freqtrade GitHub repository](https://github.com/freqtrade/freqtrade/issues).
!!! Warning "Performance"
Orderflow requires raw trades data. This data is rather large, and can cause a slow initial startup, when freqtrade needs to download the trades data for the last X candles. Additionally, enabling this feature will cause increased memory usage. Please ensure to have sufficient resources available.
## Getting Started
### Enable Public Trades
In your `config.json` file, set the `use_public_trades` option to true under the `exchange` section.
```json
"exchange":{
...
"use_public_trades":true,
}
```
### Configure Orderflow Processing
Define your desired settings for orderflow processing within the orderflow section of config.json. Here, you can adjust factors like:
-`cache_size`: How many previous orderflow candles are saved into cache instead of calculated every new candle
-`max_candles`: Filter how many candles would you like to get trades data for.
-`scale`: This controls the price bin size for the footprint chart.
-`stacked_imbalance_range`: Defines the minimum consecutive imbalanced price levels required for consideration.
-`imbalance_volume`: Filters out imbalances with volume below this threshold.
-`imbalance_ratio`: Filters out imbalances with a ratio (difference between ask and bid volume) lower than this value.
```json
"orderflow":{
"cache_size":1000,
"max_candles":1500,
"scale":0.5,
"stacked_imbalance_range":3,// needs at least this amount of imbalance next to each other
"imbalance_volume":1,// filters out below
"imbalance_ratio":3// filters out ratio lower than
},
```
## Downloading Trade Data for Backtesting
To download historical trade data for backtesting, use the --dl-trades flag with the freqtrade download-data command.
Not all exchanges provide public trade data. For supported exchanges, freqtrade will warn you if public trade data is not available if you start downloading data with the `--dl-trades` flag.
## Accessing Orderflow Data
Once activated, several new columns become available in your dataframe:
``` python
dataframe["trades"] # Contains information about each individual trade.
dataframe["orderflow"] # Represents a footprint chart dict (see below)
dataframe["imbalances"] # Contains information about imbalances in the order flow.
dataframe["bid"] # Total bid volume
dataframe["ask"] # Total ask volume
dataframe["delta"] # Difference between ask and bid volume.
dataframe["min_delta"] # Minimum delta within the candle
dataframe["max_delta"] # Maximum delta within the candle
dataframe["total_trades"] # Total number of trades
dataframe["stacked_imbalances_bid"] # Price level of stacked bid imbalance
dataframe["stacked_imbalances_ask"] # Price level of stacked ask imbalance
```
You can access these columns in your strategy code for further analysis. Here's an example:
This column provides a detailed breakdown of buy and sell orders at different price levels, offering valuable insights into order flow dynamics. The `scale` parameter in your configuration determines the price bin size for this representation
The `orderflow` column contains a dict with the following structure:
``` output
{
"price": {
"bid_amount": 0.0,
"ask_amount": 0.0,
"bid": 0,
"ask": 0,
"delta": 0.0,
"total_volume": 0.0,
"total_trades": 0
}
}
```
#### Orderflow column explanation
- key: Price bin - binned at `scale` intervals
- `bid_amount`: Total volume bought at each price level.
- `ask_amount`: Total volume sold at each price level.
- `bid`: Number of buy orders at each price level.
- `ask`: Number of sell orders at each price level.
- `delta`: Difference between ask and bid volume at each price level.
- `total_volume`: Total volume (ask amount + bid amount) at each price level.
- `total_trades`: Total number of trades (ask + bid) at each price level.
By leveraging these features, you can gain valuable insights into market sentiment and potential trading opportunities based on order flow analysis.
### Raw trades data (`dataframe["trades"]`)
List with the individual trades that occurred during the candle. This data can be used for more granular analysis of order flow dynamics.
Each individual entry contains a dict with the following keys:
- `timestamp`: Timestamp of the trade.
- `date`: Date of the trade.
- `price`: Price of the trade.
- `amount`: Volume of the trade.
- `side`: Buy or sell.
- `id`: Unique identifier for the trade.
- `cost`: Total cost of the trade (price * amount).
### Imbalances (`dataframe["imbalances"]`)
This column provides a dict with information about imbalances in the order flow. An imbalance occurs when there is a significant difference between the ask and bid volume at a given price level.
Each row looks as follows - with price as index, and the corresponding bid and ask imbalance values as columns
You can use whatever naming convention you want, freqtrade1 and 2 are arbitrary. Note, that you will need to use different database files, port mappings and telegram configurations for each instance, as mentioned above.
## Use a different database system
Freqtrade is using SQLAlchemy, which supports multiple different database systems. As such, a multitude of database systems should be supported.
Freqtrade does not depend or install any additional database driver. Please refer to the [SQLAlchemy docs](https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls) on installation instructions for the respective database systems.
The following systems have been tested and are known to work with freqtrade:
* sqlite (default)
* PostgreSQL
* MariaDB
!!! Warning
By using one of the below database systems, you acknowledge that you know how to manage such a system. The freqtrade team will not provide any support with setup or maintenance (or backups) of the below database systems.
Freqtrade will automatically create the tables necessary upon startup.
If you're running different instances of Freqtrade, you must either setup one database per Instance or use different users / schemas for your connections.
### MariaDB / MySQL
Freqtrade supports MariaDB by using SQLAlchemy, which supports multiple different database systems.
The bot has made `429` trades for an average duration of `4:12:00`, with a performance of `76.20%` (profit), that means it has
earned a total of `0.00762792 BTC` starting with a capital of 0.01 BTC.
The column `Avg Profit %` shows the average profit for all trades made while the column `Cum Profit %` sums up all the profits/losses.
The column `Avg Profit %` shows the average profit for all trades made.
The column `Tot Profit %` shows instead the total profit % in relation to the starting balance.
In the above results, we have a starting balance of 0.01 BTC and an absolute profit of 0.00762792 BTC - so the `Tot Profit %` will be `(0.00762792 / 0.01) * 100 ~= 76.2%`.
@@ -398,6 +407,7 @@ It contains some useful key metrics about performance of your strategy on backte
@@ -452,6 +462,7 @@ It contains some useful key metrics about performance of your strategy on backte
-`Backtesting from` / `Backtesting to`: Backtesting range (usually defined with the `--timerange` option).
-`Max open trades`: Setting of `max_open_trades` (or `--max-open-trades`) - or number of pairs in the pairlist (whatever is lower).
-`Trading Mode`: Spot or Futures trading.
-`Total/Daily Avg Trades`: Identical to the total trades of the backtest output table / Total trades divided by the backtesting duration in days (this will give you information about how many trades to expect from the strategy).
-`Starting balance`: Start balance - as given by dry-run-wallet (config or command line).
-`Final balance`: Final balance - starting balance + absolute profit.
@@ -464,7 +475,7 @@ It contains some useful key metrics about performance of your strategy on backte
-`Profit factor`: profit / loss.
-`Avg. stake amount`: Average stake amount, either `stake_amount` or the average when using dynamic stake amount.
-`Total trade volume`: Volume generated on the exchange to reach the above profit.
-`Best Pair` / `Worst Pair`: Best and worst performing pair, and it's corresponding `Cum Profit %`.
-`Best Pair` / `Worst Pair`: Best and worst performing pair, and it's corresponding `Tot Profit %`.
-`Best Trade` / `Worst Trade`: Biggest single winning trade and biggest single losing trade.
-`Best day` / `Worst day`: Best and worst day based on daily profit.
-`Days win/draw/lose`: Winning / Losing days (draws are usually days without closed trade).
@@ -522,21 +533,22 @@ To save time, by default backtest will reuse a cached result from within the las
### Further backtest-result analysis
To further analyze your backtest results, you can [export the trades](#exporting-trades-to-file).
You can then load the trades to perform further analysis as shown in the [data analysis](data-analysis.md#backtesting) backtesting section.
To further analyze your backtest results, freqtrade will export the trades to file by default.
You can then load the trades to perform further analysis as shown in the [data analysis](strategy_analysis_example.md#load-backtest-results-to-pandas-dataframe) backtesting section.
## Assumptions made by backtesting
Since backtesting lacks some detailed information about what happens within a candle, it needs to take a few assumptions:
- Exchange [trading limits](#trading-limits-in-backtesting) are respected
- Entries happen at open-price
- All orders are filled at the requested price (no slippage, no unfilled orders)
- Entries happen at open-price unless a custom price logic has been specified
- All orders are filled at the requested price (no slippage) as long as the price is within the candle's high/low range
- Exit-signal exits happen at open-price of the consecutive candle
- Exits free their trade slot for a new trade with a different pair
- Exit-signal is favored over Stoploss, because exit-signals are assumed to trigger on candle's open
- ROI
- exits are compared to high - but the ROI value is used (e.g. ROI = 2%, high=5% - so the exit will be at 2%)
- exits are never "below the candle", so a ROI of 2% may result in a exit at 2.4% if low was at 2.4% profit
- Exits are compared to high - but the ROI value is used (e.g. ROI = 2%, high=5% - so the exit will be at 2%)
- Exits are never "below the candle", so a ROI of 2% may result in a exit at 2.4% if low was at 2.4% profit
- ROI entries which came into effect on the triggering candle (e.g. `120: 0.02` for 1h candles, from `60: 0.05`) will use the candle's open as exit rate
- Force-exits caused by `<N>=-1` ROI entries use low as exit value, unless N falls on the candle open (e.g. `120: -1` for 1h candles)
- Stoploss exits happen exactly at stoploss price, even if low was lower, but the loss will be `2 * fees` higher than the stoploss price
@@ -554,6 +566,7 @@ Since backtesting lacks some detailed information about what happens within a ca
- Stoploss
- ROI
- Trailing stoploss
- Position reversals (futures only) happen if an entry signal in the other direction than the closing trade triggers at the candle the existing trade closes.
Taking these assumptions, backtesting tries to mirror real trading as closely as possible. However, backtesting will **never** replace running a strategy in dry-run mode.
Also, keep in mind that past results don't guarantee future success.
@@ -568,7 +581,7 @@ These limits are usually listed in the exchange documentation as "trading rules"
Backtesting (as well as live and dry-run) does honor these limits, and will ensure that a stoploss can be placed below this value - so the value will be slightly higher than what the exchange specifies.
Freqtrade has however no information about historic limits.
This can lead to situations where trading-limits are inflated by using a historic price, resulting in minimum amounts > 50$.
This can lead to situations where trading-limits are inflated by using a historic price, resulting in minimum amounts > 50\$.
For example:
@@ -587,7 +600,7 @@ These precision values are based on current exchange limits (as described in the
## Improved backtest accuracy
One big limitation of backtesting is it's inability to know how prices moved intra-candle (was high before close, or viceversa?).
One big limitation of backtesting is it's inability to know how prices moved intra-candle (was high before close, or vice-versa?).
So assuming you run backtesting with a 1h timeframe, there will be 4 prices for that candle (Open, High, Low, Close).
While backtesting does take some assumptions (read above) about this - this can never be perfect, and will always be biased in one way or the other.
@@ -618,22 +631,22 @@ To compare multiple strategies, a list of Strategies can be provided to backtest
This is limited to 1 timeframe value per run. However, data is only loaded once from disk so if you have multiple
strategies you'd like to compare, this will give a nice runtime boost.
All listed Strategies need to be in the same directory.
All listed Strategies need to be in the same directory, unless also `--recursive-strategy-search` is specified, where sub-directories within the strategy directory are also considered.
This will save the results to `user_data/backtest_results/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename.
This will save the results to `user_data/backtest_results/backtest-result-<datetime>.json`, including results for both `Strategy001` and `Strategy002`.
There will be an additional table comparing win/losses of the different strategies (identical to the "Total" row in the first table).
Detailed output for all strategies one after the other will be available, so make sure to scroll up to see the details per strategy.
@@ -33,7 +33,6 @@ For spot pairs, naming will be `base/quote` (e.g. `ETH/USDT`).
For futures pairs, naming will be `base/quote:settle` (e.g. `ETH/USDT:USDT`).
## Bot execution logic
Starting freqtrade in dry-run or live mode (using `freqtrade trade`) will start the bot and start the bot iteration loop.
@@ -50,6 +49,8 @@ By default, the bot loop runs every few seconds (`internals.process_throttle_sec
* Call `populate_indicators()`
* Call `populate_entry_trend()`
* Call `populate_exit_trend()`
* Update trades open order state from exchange.
* Call `order_filled()` strategy callback for filled orders.
* Check timeouts for open orders.
* Calls `check_entry_timeout()` strategy callback for open entry orders.
* Calls `check_exit_timeout()` strategy callback for open exit orders.
@@ -86,8 +87,10 @@ This loop will be repeated again and again until the bot is stopped.
* In Margin and Futures mode, `leverage()` strategy callback is called to determine the desired leverage.
* Determine stake size by calling the `custom_stake_amount()` callback.
* Check position adjustments for open trades if enabled and call `adjust_trade_position()` to determine if an additional order is requested.
* Call `order_filled()` strategy callback for filled entry orders.
* Call `custom_stoploss()` and `custom_exit()` to find custom exit points.
* For exits based on exit-signal, custom-exit and partial exits: Call `custom_exit_price()` to determine exit price (Prices are moved to be within the closing candle).
* Call `order_filled()` strategy callback for filled exit orders.
@@ -14,7 +14,7 @@ You can specify a different configuration file used by the bot with the `-c/--co
If you used the [Quick start](docker_quickstart.md#docker-quick-start) method for installing
the bot, the installation script should have already created the default configuration file (`config.json`) for you.
If the default configuration file is not created we recommend to use `freqtrade new-config --config config.json` to generate a basic configuration file.
If the default configuration file is not created we recommend to use `freqtrade new-config --config user_data/config.json` to generate a basic configuration file.
The Freqtrade configuration file is to be written in JSON format.
Environment variables detected are logged at startup - so if you can't find why a value is not what you think it should be based on the configuration, make sure it's not loaded from an environment variable.
!!! Tip "Validate combined result"
You can use the [show-config subcommand](utils.md#show-config) to see the final, combined configuration.
??? Warning "Loading sequence"
Environment variables are loaded after the initial configuration. As such, you cannot provide the path to the configuration through environment variables. Please use `--config path/to/config.json` for that.
This also applies to user_dir to some degree. while the user directory can be set through environment variables - the configuration will **not** be loaded from that location.
### Multiple configuration files
Multiple configuration files can be specified and used by the bot or the bot can read its configuration parameters from the process standard input stream.
@@ -56,6 +63,9 @@ Multiple configuration files can be specified and used by the bot or the bot can
You can specify additional configuration files in `add_config_files`. Files specified in this parameter will be loaded and merged with the initial config file. The files are resolved relative to the initial configuration file.
This is similar to using multiple `--config` parameters, but simpler in usage as you don't have to specify all files for all commands.
!!! Tip "Validate combined result"
You can use the [show-config subcommand](utils.md#show-config) to see the final, combined configuration.
!!! Tip "Use multiple configuration files to keep secrets secret"
You can use a 2nd configuration file containing your secrets. That way you can share your "primary" configuration file, while still keeping your API keys for yourself.
The 2nd file should only specify what you intend to override.
@@ -113,6 +123,19 @@ This is similar to using multiple `--config` parameters, but simpler in usage as
If multiple files are in the `add_config_files` section, then they will be assumed to be at identical levels, having the last occurrence override the earlier config (unless a parent already defined such a key).
## Editor autocomplete and validation
If you are using an editor that supports JSON schema, you can use the schema provided by Freqtrade to get autocompletion and validation of your configuration file by adding the following line to the top of your configuration file:
The develop schema is available as `https://schema.freqtrade.io/schema_dev.json` - though we recommend to stick to the stable version for the best experience.
## Configuration parameters
The table below will list all configuration parameters available.
@@ -123,10 +146,10 @@ Freqtrade can also load many options via command line (CLI) arguments (check out
The prevalence for all Options is as follows:
- CLI arguments override any other option
- [Environment Variables](#environment-variables)
- Configuration files are used in sequence (the last file wins) and override Strategy configurations.
- Strategy configurations are only used if they are not set via configuration or command-line arguments. These options are marked with [Strategy Override](#parameters-in-the-strategy) in the below table.
* CLI arguments override any other option
* [Environment Variables](#environment-variables)
* Configuration files are used in sequence (the last file wins) and override Strategy configurations.
* Strategy configurations are only used if they are not set via configuration or command-line arguments. These options are marked with [Strategy Override](#parameters-in-the-strategy) in the below table.
### Parameters table
@@ -160,7 +183,7 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `margin_mode` | When trading with leverage, this determines if the collateral owned by the trader will be shared or isolated to each trading pair [leverage documentation](leverage.md). <br> **Datatype:** String
| `liquidation_buffer` | A ratio specifying how large of a safety net to place between the liquidation price and the stoploss to prevent a position from reaching the liquidation price [leverage documentation](leverage.md). <br>*Defaults to `0.05`.* <br> **Datatype:** Float
| | **Unfilled timeout**
| `unfilledtimeout.entry` | **Required.** How long (in minutes or seconds) the bot will wait for an unfilled entry order to complete, after which the order will be cancelled and repeated at current (new) price, as long as there is a signal. [Strategy Override](#parameters-in-the-strategy).<br> **Datatype:** Integer
| `unfilledtimeout.entry` | **Required.** How long (in minutes or seconds) the bot will wait for an unfilled entry order to complete, after which the order will be cancelled. [Strategy Override](#parameters-in-the-strategy).<br> **Datatype:** Integer
| `unfilledtimeout.exit` | **Required.** How long (in minutes or seconds) the bot will wait for an unfilled exit order to complete, after which the order will be cancelled and repeated at current (new) price, as long as there is a signal. [Strategy Override](#parameters-in-the-strategy).<br> **Datatype:** Integer
| `unfilledtimeout.unit` | Unit to use in unfilledtimeout setting. Note: If you set unfilledtimeout.unit to "seconds", "internals.process_throttle_secs" must be inferior or equal to timeout [Strategy Override](#parameters-in-the-strategy). <br> *Defaults to `"minutes"`.* <br> **Datatype:** String
| `unfilledtimeout.exit_timeout_count` | How many times can exit orders time out. Once this number of timeouts is reached, an emergency exit is triggered. 0 to disable and allow unlimited order cancels. [Strategy Override](#parameters-in-the-strategy).<br>*Defaults to `0`.* <br> **Datatype:** Integer
@@ -187,18 +210,18 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `position_adjustment_enable` | Enables the strategy to use position adjustments (additional buys or sells). [More information here](strategy-callbacks.md#adjust-trade-position). <br> [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `false`.*<br> **Datatype:** Boolean
| `max_entry_position_adjustment` | Maximum additional order(s) for each open trade on top of the first entry Order. Set it to `-1` for unlimited additional orders. [More information here](strategy-callbacks.md#adjust-trade-position). <br> [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `-1`.*<br> **Datatype:** Positive Integer or -1
| | **Exchange**
| `exchange.name` | **Required.** Name of the exchange class to use. [List below](#user-content-what-values-for-exchangename). <br> **Datatype:** String
| `exchange.name` | **Required.** Name of the exchange class to use. <br> **Datatype:** String
| `exchange.key` | API key to use for the exchange. Only required when you are in production mode.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
| `exchange.secret` | API secret to use for the exchange. Only required when you are in production mode.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
| `exchange.password` | API password to use for the exchange. Only required when you are in production mode and for exchanges that use password for API requests.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
| `exchange.uid` | API uid to use for the exchange. Only required when you are in production mode and for exchanges that use uid for API requests.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
| `exchange.pair_whitelist` | List of pairs to use by the bot for trading and to check for potential trades during backtesting. Supports regex pairs as `.*/BTC`. Not used by VolumePairList. [More information](plugins.md#pairlists-and-pairlist-handlers). <br> **Datatype:** List
| `exchange.pair_blacklist` | List of pairs the bot must absolutely avoid for trading and backtesting. [More information](plugins.md#pairlists-and-pairlist-handlers). <br> **Datatype:** List
| `exchange.ccxt_config` | Additional CCXT parameters passed to both ccxt instances (sync and async). This is usually the correct place for additional ccxt configurations. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://ccxt.readthedocs.io/en/latest/manual.html#instantiation). Please avoid adding exchange secrets here (use the dedicated fields instead), as they may be contained in logs. <br> **Datatype:** Dict
| `exchange.ccxt_sync_config` | Additional CCXT parameters passed to the regular (sync) ccxt instance. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://ccxt.readthedocs.io/en/latest/manual.html#instantiation) <br> **Datatype:** Dict
| `exchange.ccxt_async_config` | Additional CCXT parameters passed to the async ccxt instance. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://ccxt.readthedocs.io/en/latest/manual.html#instantiation) <br> **Datatype:** Dict
| `exchange.ccxt_config` | Additional CCXT parameters passed to both ccxt instances (sync and async). This is usually the correct place for additional ccxt configurations. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://docs.ccxt.com/#/README?id=overriding-exchange-properties-upon-instantiation). Please avoid adding exchange secrets here (use the dedicated fields instead), as they may be contained in logs. <br> **Datatype:** Dict
| `exchange.ccxt_sync_config` | Additional CCXT parameters passed to the regular (sync) ccxt instance. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://docs.ccxt.com/#/README?id=overriding-exchange-properties-upon-instantiation) <br> **Datatype:** Dict
| `exchange.ccxt_async_config` | Additional CCXT parameters passed to the async ccxt instance. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://docs.ccxt.com/#/README?id=overriding-exchange-properties-upon-instantiation) <br> **Datatype:** Dict
| `exchange.enable_ws` | Enable the usage of Websockets for the exchange. <br>[More information](#consuming-exchange-websockets).<br>*Defaults to `true`.* <br> **Datatype:** Boolean
| `exchange.markets_refresh_interval` | The interval in minutes in which markets are reloaded. <br>*Defaults to `60` minutes.* <br> **Datatype:** Positive Integer
| `exchange.skip_pair_validation` | Skip pairlist validation on startup.<br>*Defaults to `false`*<br> **Datatype:** Boolean
| `exchange.skip_open_order_update` | Skips open order updates on startup should the exchange cause problems. Only relevant in live conditions.<br>*Defaults to `false`*<br> **Datatype:** Boolean
| `exchange.unknown_fee_rate` | Fallback value to use when calculating trading fees. This can be useful for exchanges which have fees in non-tradable currencies. The value provided here will be multiplied with the "fee cost".<br>*Defaults to `None`<br> **Datatype:** float
| `exchange.log_responses` | Log relevant exchange responses. For debug mode only - use with care.<br>*Defaults to `false`*<br> **Datatype:** Boolean
@@ -206,7 +229,6 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| | **Plugins**
| `edge.*` | Please refer to [edge configuration document](edge.md) for detailed explanation of all possible configuration options.
| `pairlists` | Define one or more pairlists to be used. [More information](plugins.md#pairlists-and-pairlist-handlers). <br>*Defaults to `StaticPairList`.* <br> **Datatype:** List of Dicts
| `protections` | Define one or more protections to be used. [More information](plugins.md#protections). <br> **Datatype:** List of Dicts
| | **Telegram**
| `telegram.enabled` | Enable the usage of Telegram. <br> **Datatype:** Boolean
| `telegram.token` | Your Telegram bot token. Only required if `telegram.enabled` is `true`. <br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
@@ -242,7 +264,7 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `disable_dataframe_checks` | Disable checking the OHLCV dataframe returned from the strategy methods for correctness. Only use when intentionally changing the dataframe and understand what you are doing. [Strategy Override](#parameters-in-the-strategy).<br> *Defaults to `False`*. <br> **Datatype:** Boolean
| `internals.process_throttle_secs` | Set the process throttle, or minimum loop duration for one bot iteration loop. Value in second. <br>*Defaults to `5` seconds.* <br> **Datatype:** Positive Integer
| `internals.heartbeat_interval` | Print heartbeat message every N seconds. Set to 0 to disable heartbeat messages. <br>*Defaults to `60` seconds.* <br> **Datatype:** Positive Integer or 0
| `internals.sd_notify` | Enables use of the sd_notify protocol to tell systemd service manager about changes in the bot state and issue keep-alive pings. See [here](installation.md#7-optional-configure-freqtrade-as-a-systemd-service) for more details. <br> **Datatype:** Boolean
| `internals.sd_notify` | Enables use of the sd_notify protocol to tell systemd service manager about changes in the bot state and issue keep-alive pings. See [here](advanced-setup.md#configure-the-bot-running-as-a-systemd-service) for more details. <br> **Datatype:** Boolean
| `strategy` | **Required** Defines Strategy class to use. Recommended to be set via `--strategy NAME`. <br> **Datatype:** ClassName
| `strategy_path` | Adds an additional strategy lookup path (must be a directory). <br> **Datatype:** String
| `recursive_strategy_search` | Set to `true` to recursively search sub-directories inside `user_data/strategies` for a strategy. <br> **Datatype:** Boolean
@@ -273,10 +295,10 @@ Values set in the configuration file always overwrite values set in the strategy
* `order_time_in_force`
* `unfilledtimeout`
* `disable_dataframe_checks`
- `use_exit_signal`
* `use_exit_signal`
* `exit_profit_only`
- `exit_profit_offset`
- `ignore_roi_if_entry_signal`
* `exit_profit_offset`
* `ignore_roi_if_entry_signal`
* `ignore_buying_expired_candle_after`
* `position_adjustment_enable`
* `max_entry_position_adjustment`
@@ -289,12 +311,12 @@ There are several methods to configure how much of the stake currency the bot wi
The minimum stake amount will depend on exchange and pair and is usually listed in the exchange support pages.
Assuming the minimum tradable amount for XRP/USD is 20 XRP (given by the exchange), and the price is 0.6$, the minimum stake amount to buy this pair is `20 * 0.6 ~= 12`.
This exchange has also a limit on USD - where all orders must be > 10$ - which however does not apply in this case.
Assuming the minimum tradable amount for XRP/USD is 20 XRP (given by the exchange), and the price is 0.6\$, the minimum stake amount to buy this pair is `20 * 0.6 ~= 12`.
This exchange has also a limit on USD - where all orders must be > 10\$ - which however does not apply in this case.
To guarantee safe execution, freqtrade will not allow buying with a stake-amount of 10.1$, instead, it'll make sure that there's enough space to place a stoploss below the pair (+ an offset, defined by `amount_reserve_percent`, which defaults to 5%).
To guarantee safe execution, freqtrade will not allow buying with a stake-amount of 10.1\$, instead, it'll make sure that there's enough space to place a stoploss below the pair (+ an offset, defined by `amount_reserve_percent`, which defaults to 5%).
With a reserve of 5%, the minimum stake amount would be ~12.6$ (`12 * (1 + 0.05)`). If we take into account a stoploss of 10% on top of that - we'd end up with a value of ~14$ (`12.6 / (1 - 0.1)`).
With a reserve of 5%, the minimum stake amount would be ~12.6\$ (`12 * (1 + 0.05)`). If we take into account a stoploss of 10% on top of that - we'd end up with a value of ~14\$ (`12.6 / (1 - 0.1)`).
To limit this calculation in case of large stoploss values, the calculated minimum stake-limit will never be more than 50% above the real limit.
@@ -321,11 +343,13 @@ For example, if you have 10 ETH available in your wallet on the exchange and `tr
To fully utilize compounding profits when using multiple bots on the same exchange account, you'll want to limit each bot to a certain starting balance.
This can be accomplished by setting `available_capital` to the desired starting balance.
Assuming your account has 10.000 USDT and you want to run 2 different strategies on this exchange.
Assuming your account has 10000 USDT and you want to run 2 different strategies on this exchange.
You'd set `available_capital=5000` - granting each bot an initial capital of 5000 USDT.
The bot will then split this starting balance equally into `max_open_trades` buckets.
Profitable trades will result in increased stake-sizes for this bot - without affecting the stake-sizes of the other bot.
Adjusting `available_capital` requires reloading the configuration to take effect. Adjusting the `available_capital` adds the difference between the previous `available_capital` and the new `available_capital`. Decreasing the available capital when trades are open doesn't exit the trades. The difference is returned to the wallet when the trades conclude. The outcome of this differs depending on the price movement between the adjustment and exiting the trades.
!!! Warning "Incompatible with `tradable_balance_ratio`"
Setting this option will replace any configuration of `tradable_balance_ratio`.
@@ -338,9 +362,9 @@ To overcome this, the option `amend_last_stake_amount` can be set to `True`, whi
In the example above this would mean:
- Trade1: 400 USDT
- Trade2: 400 USDT
- Trade3: 200 USDT
* Trade1: 400 USDT
* Trade2: 400 USDT
* Trade3: 200 USDT
!!! Note
This option only applies with [Static stake amount](#static-stake-amount) - since [Dynamic stake amount](#dynamic-stake-amount) divides the balances evenly.
@@ -358,7 +382,7 @@ This setting works in combination with `max_open_trades`. The maximum capital en
For example, the bot will at most use (0.05 BTC x 3) = 0.15 BTC, assuming a configuration of `max_open_trades=3` and `stake_amount=0.05`.
!!! Note
This setting respects the [available balance configuration](#available-balance).
This setting respects the [available balance configuration](#tradable-balance).
#### Dynamic stake amount
@@ -397,6 +421,8 @@ Or another example if your position adjustment assumes it can do 1 additional bu
--8<-- "includes/pricing.md"
## Further Configuration details
### Understand minimal_roi
The `minimal_roi` configuration parameter is a JSON object where the key is a duration
@@ -503,13 +529,13 @@ Configuration:
Please carefully read the section [Market order pricing](#market-order-pricing) section when using market orders.
!!! Note "Stoploss on exchange"
`stoploss_on_exchange_interval` is not mandatory. Do not change its value if you are
`order_types.stoploss_on_exchange_interval` is not mandatory. Do not change its value if you are
unsure of what you are doing. For more information about how stoploss works please
refer to [the stoploss documentation](stoploss.md).
If `stoploss_on_exchange` is enabled and the stoploss is cancelled manually on the exchange, then the bot will create a new stoploss order.
If `order_types.stoploss_on_exchange` is enabled and the stoploss is cancelled manually on the exchange, then the bot will create a new stoploss order.
If stoploss on exchange creation fails for some reason, then an "emergency exit" is initiated. By default, this will exit the trade using a market order. The order-type for the emergency-exit can be changed by setting the `emergency_exit` value in the `order_types` dictionary - however, this is not advised.
### Understand order_time_in_force
@@ -535,7 +561,7 @@ is automatically cancelled by the exchange.
**PO (Post only):**
Post only order. The order is either placed as a maker order, or it is canceled.
This means the order must be placed on orderbook for at at least time in an unfilled state.
This means the order must be placed on orderbook for at least time in an unfilled state.
#### time_in_force config
@@ -556,7 +582,14 @@ The possible values are: `GTC` (default), `FOK` or `IOC`.
This is ongoing work. For now, it is supported only for binance, gate and kucoin.
Please don't change the default value unless you know what you are doing and have researched the impact of using different values for your particular exchange.
### What values can be used for fiat_display_currency?
### Fiat conversion
Freqtrade uses the Coingecko API to convert the coin value to it's corresponding fiat value for the Telegram reports.
The FIAT currency can be set in the configuration file as `fiat_display_currency`.
Removing `fiat_display_currency` completely from the configuration will skip initializing coingecko, and will not show any FIAT currency conversion. This has no importance for the correct functioning of the bot.
#### What values can be used for fiat_display_currency?
The `fiat_display_currency` configuration parameter sets the base currency to use for the
conversion from coin to fiat in the bot Telegram reports.
@@ -572,9 +605,53 @@ In addition to fiat currencies, a range of crypto currencies is supported.
The valid values are:
```json
"BTC", "ETH", "XRP", "LTC", "BCH", "USDT"
"BTC", "ETH", "XRP", "LTC", "BCH", "BNB"
```
#### Coingecko Rate limit problems
On some IP ranges, coingecko is heavily rate-limiting.
In such cases, you may want to add your coingecko API key to the configuration.
``` json
{
"fiat_display_currency": "USD",
"coingecko": {
"api_key": "your-api",
"is_demo": true
}
}
```
Freqtrade supports both Demo and Pro coingecko API keys.
The Coingecko API key is NOT required for the bot to function correctly.
It is only used for the conversion of coin to fiat in the Telegram reports, which usually also work without API key.
## Consuming exchange Websockets
Freqtrade can consume websockets through ccxt.pro.
Freqtrade aims ensure data is available at all times.
Should the websocket connection fail (or be disabled), the bot will fall back to REST API calls.
Should you experience problems you suspect are caused by websockets, you can disable these via the setting `exchange.enable_ws`, which defaults to true.
```jsonc
"exchange": {
// ...
"enable_ws": false,
// ...
}
```
Should you be required to use a proxy, please refer to the [proxy section](#using-proxy-with-freqtrade) for more information.
!!! Info "Rollout"
We're implementing this out slowly, ensuring stability of your bots.
Currently, usage is limited to ohlcv data streams.
It's also limited to a few exchanges, with new exchanges being added on an ongoing basis.
## Using Dry-run mode
We recommend starting the bot in the Dry-run mode to see how your bot will
@@ -594,7 +671,7 @@ creating trades on the exchange.
```json
"exchange": {
"name": "bittrex",
"name": "binance",
"key": "key",
"secret": "secret",
...
@@ -611,9 +688,9 @@ Once you will be happy with your bot performance running in the Dry-run mode, yo
* API-keys may or may not be provided. Only Read-Only operations (i.e. operations that do not alter account state) on the exchange are performed in dry-run mode.
* Wallets (`/balance`) are simulated based on `dry_run_wallet`.
* Orders are simulated, and will not be posted to the exchange.
* Market orders fill based on orderbook volume the moment the order is placed.
* Market orders fill based on orderbook volume the moment the order is placed, with a maximum slippage of 5%.
* Limit orders fill once the price reaches the defined level - or time out based on `unfilledtimeout` settings.
* Limit orders will be converted to market orders if they cross the price by more than 1%.
* Limit orders will be converted to market orders if they cross the price by more than 1%, and will be filled immediately based regular market order rules (see point about Market orders above).
* In combination with `stoploss_on_exchange`, the stop_loss price is assumed to be filled.
* Open orders (not trades, which are stored in the database) are kept open after bot restarts, with the assumption that they were not filled while being offline.
@@ -644,7 +721,7 @@ API Keys are usually only required for live trading (trading for real money, bot
@@ -11,9 +11,8 @@ Without provided configuration, `--exchange` becomes mandatory.
You can use a relative timerange (`--days 20`) or an absolute starting point (`--timerange 20200101-`). For incremental downloads, the relative approach should be used.
!!! Tip "Tip: Updating existing data"
If you already have backtesting data available in your data-directory and would like to refresh this data up to today, freqtrade will automatically calculate the data missing for the existing pairs and the download will occur from the latest available point until "now", neither --days or --timerange parameters are required. Freqtrade will keep the available data and only download the missing data.
If you are updating existing data after inserting new pairs that you have no data for, use `--new-pairs-days xx` parameter. Specified number of days will be downloaded for new pairs while old pairs will be updated with missing data only.
If you use `--days xx` parameter alone - data for specified number of days will be downloaded for _all_ pairs. Be careful, if specified number of days is smaller than gap between now and last downloaded candle - freqtrade will delete all existing data to avoid gaps in candle data.
If you already have backtesting data available in your data-directory and would like to refresh this data up to today, freqtrade will automatically calculate the missing timerange for the existing pairs and the download will occur from the latest available point until "now", neither `--days` or `--timerange` parameters are required. Freqtrade will keep the available data and only download the missing data.
If you are updating existing data after inserting new pairs that you have no data for, use the `--new-pairs-days xx` parameter. Specified number of days will be downloaded for new pairs while old pairs will be updated with missing data only.
!!! Tip "Downloading all data for one quote currency"
Often, you'll want to download data for all pairs of a specific quote-currency. In such cases, you can use the following shorthand:
`freqtrade download-data --exchange binance --pairs .*/USDT <...>`. The provided "pairs" string will be expanded to contain all active pairs on the exchange.
`freqtrade download-data --exchange binance --pairs ".*/USDT" <...>`. The provided "pairs" string will be expanded to contain all active pairs on the exchange.
To also download data for inactive (delisted) pairs, add `--include-inactive-pairs` to the command.
* To use a different directory than the exchange specific default, use `--datadir user_data/data/some_directory`.
* To change the exchange used to download the historical data from, please use a different configuration file (you'll probably need to adjust rate limits etc.)
* To change the exchange used to download the historical data from, either use `--exchange <exchange>` - or specify a different configuration file.
* To use `pairs.json` from some other directory, use `--pairs-file some_other_dir/pairs.json`.
* To download historical candle (OHLCV) data for only 10 days, use `--days 10` (defaults to 30 days).
* To download historical candle (OHLCV) data from a fixed starting point, use `--timerange 20200101-` - which will download all data from January 1st, 2020.
* Given starting points are ignored if data is already available, downloading only missing data up to today.
* Use `--timeframes` to specify what timeframe download the historical candle (OHLCV) data for. Default is `--timeframes 1m 5m` which will download 1-minute and 5-minute data.
* To use exchange, timeframe and list of pairs as defined in your configuration file, use the `-c/--config` option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine `-c/--config` with most other options.
@@ -418,7 +423,8 @@ You can get a list of downloaded data using the `list-data` sub-command.
By default, `download-data` sub-command downloads Candles (OHLCV) data. Some exchanges also provide historic trade-data via their API.
By default, `download-data` sub-command downloads Candles (OHLCV) data. Most exchanges also provide historic trade-data via their API.
This data can be useful if you need many different timeframes, since it is only downloaded once, and then resampled locally to the desired timeframes.
Since this data is large by default, the files use the feather file format by default. They are stored in your data-directory with the naming convention of `<pair>-trades.feather` (`ETH_BTC-trades.feather`). Incremental mode is also supported, as for historic OHLCV data, so downloading the data once per week with `--days 8` will create an incremental data-repository.
To use this mode, simply add `--dl-trades` to your call. This will swap the download method to download trades, and resamples the data locally.
To use this mode, simply add `--dl-trades` to your call. This will swap the download method to download trades.
If `--convert` is also provided, the resample step will happen automatically and overwrite eventually existing OHLCV data for the given pair/timeframe combinations.
!!! Warning "do not use"
You should not use this unless you're a kraken user. Most other exchanges provide OHLCV data with sufficient history.
!!! Warning "Do not use"
You should not use this unless you're a kraken user (Kraken does not provide historic OHLCV data).
Most other exchanges provide OHLCV data with sufficient history, so downloading multiple timeframes through that method will still proof to be a lot faster than downloading trades data.
!!! Note "Kraken user"
Kraken users should read [this](exchanges.md#historic-kraken-data) before starting to download data.
While this method uses async calls, it will be slow, since it requires the result of the previous call to generate the next request to the exchange.
!!! Warning
The historic trades are not available during Freqtrade dry-run and live trade modes because all exchanges tested provide this data with a delay of few 100 candles, so it's not suitable for real-time trading.
!!! Note "Kraken user"
Kraken users should read [this](exchanges.md#historic-kraken-data) before starting to download data.
## Next step
Great, you now have backtest data downloaded, so you can now start [backtesting](backtesting.md) your strategy.
Great, you now have some data downloaded, so you can now start [backtesting](backtesting.md) your strategy.
version 2023.3 saw the removal of `populate_any_indicators` in favor of split methods for feature engineering and targets. Please read the [migration document](strategy_migration.md#freqai-strategy) for full details.
## Removal of `protections` from configuration
Setting protections from the configuration via `"protections": [],` has been removed in 2024.10, after having raised deprecation warnings for over 3 years.
@@ -22,7 +22,7 @@ This will spin up a local server (usually on port 8000) so you can see if everyt
## Developer setup
To configure a development environment, you can either use the provided [DevContainer](#devcontainer-setup), or use the `setup.sh` script and answer "y" when asked "Do you want to install dependencies for dev [y/N]? ".
Alternatively (e.g. if your system is not supported by the setup.sh script), follow the manual installation process and run `pip3 install -e .[all]`.
Alternatively (e.g. if your system is not supported by the setup.sh script), follow the manual installation process and run `pip3 install -r requirements-dev.txt` - followed by `pip3 install -e .[all]`.
This will install all required tools for development, including `pytest`, `ruff`, `mypy`, and `coveralls`.
@@ -83,7 +83,7 @@ Details will obviously vary between setups - but this should work to get you sta
``` json
{
"name": "freqtrade trade",
"type": "python",
"type": "debugpy",
"request": "launch",
"module": "freqtrade",
"console": "integratedTerminal",
@@ -116,7 +116,7 @@ A similar setup can also be taken for Pycharm - using `freqtrade` as module name
This assumes that you have the repository checked out, and the editor is started at the repository root level (so setup.py is at the top level of your repository).
This assumes that you have the repository checked out, and the editor is started at the repository root level (so pyproject.toml is at the top level of your repository).
## ErrorHandling
@@ -129,6 +129,8 @@ Below is an outline of exception inheritance hierarchy:
+ FreqtradeException
|
+---+ OperationalException
| |
| +---+ ConfigurationError
|
+---+ DependencyException
| |
@@ -160,7 +162,7 @@ Hopefully you also want to contribute this back upstream.
Whatever your motivations are - This should get you off the ground in trying to develop a new Pairlist Handler.
First of all, have a look at the [VolumePairList](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/pairlist/VolumePairList.py) Handler, and best copy this file with a name of your new Pairlist Handler.
First of all, have a look at the [VolumePairList](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/plugins/pairlist/VolumePairList.py) Handler, and best copy this file with a name of your new Pairlist Handler.
This is a simple Handler, which however serves as a good example on how to start developing.
@@ -203,7 +205,7 @@ This is called with each iteration of the bot (only if the Pairlist Handler is a
It must return the resulting pairlist (which may then be passed into the chain of Pairlist Handlers).
Validations are optional, the parent class exposes a `_verify_blacklist(pairlist)` and `_whitelist_for_active_markets(pairlist)` to do default filtering. Use this if you limit your result to a certain number of pairs - so the end-result is not shorter than expected.
Validations are optional, the parent class exposes a `verify_blacklist(pairlist)` and `_whitelist_for_active_markets(pairlist)` to do default filtering. Use this if you limit your result to a certain number of pairs - so the end-result is not shorter than expected.
#### filter_pairlist
@@ -217,14 +219,14 @@ The default implementation in the base class simply calls the `_validate_pair()`
If overridden, it must return the resulting pairlist (which may then be passed into the next Pairlist Handler in the chain).
Validations are optional, the parent class exposes a `_verify_blacklist(pairlist)` and `_whitelist_for_active_markets(pairlist)` to do default filters. Use this if you limit your result to a certain number of pairs - so the end result is not shorter than expected.
Validations are optional, the parent class exposes a `verify_blacklist(pairlist)` and `_whitelist_for_active_markets(pairlist)` to do default filters. Use this if you limit your result to a certain number of pairs - so the end result is not shorter than expected.
In `VolumePairList`, this implements different methods of sorting, does early validation so only the expected number of pairs is returned.
@@ -239,7 +241,6 @@ No protection should use datetime directly, but use the provided `date_now` vari
!!! Tip "Writing a new Protection"
Best copy one of the existing Protections to have a good example.
Don't forget to register your protection in `constants.py` under the variable `AVAILABLE_PROTECTIONS` - otherwise it will not be selectable.
#### Implementation of a new protection
@@ -259,7 +260,7 @@ For that reason, they must implement the following methods:
The `until` portion should be calculated using the provided `calculate_lock_end()` method.
All Protections should use `"stop_duration"` / `"stop_duration_candles"` to define how long a a pair (or all pairs) should be locked.
All Protections should use `"stop_duration"` / `"stop_duration_candles"` to define how long a pair (or all pairs) should be locked.
The content of this is made available as `self._stop_duration` to the each Protection.
If your protection requires a look-back period, please use `"lookback_period"` / `"lockback_period_candles"` to keep all protections aligned.
@@ -303,7 +304,7 @@ The `IProtection` parent class provides a helper method for this in `calculate_l
Most exchanges supported by CCXT should work out of the box.
To quickly test the public endpoints of an exchange, add a configuration for your exchange to `test_ccxt_compat.py` and run these tests with `pytest --longrun tests/exchange/test_ccxt_compat.py`.
To quickly test the public endpoints of an exchange, add a configuration for your exchange to `tests/exchange_online/conftest.py` and run these tests with `pytest --longrun tests/exchange_online/test_ccxt_compat.py`.
Completing these tests successfully a good basis point (it's a requirement, actually), however these won't guarantee correct exchange functioning, as this only tests public endpoints, but no private endpoint (like generate order or similar).
Also try to use `freqtrade download-data` for an extended timerange (multiple months) and verify that the data downloaded correctly (no holes, the specified timerange was actually downloaded).
@@ -318,6 +319,7 @@ Additional tests / steps to complete:
* Check if balance shows correctly (*)
* Create market order (*)
* Create limit order (*)
* Cancel order (*)
* Complete trade (enter + exit) (*)
* Compare result calculation between exchange and bot
* Ensure fees are applied correctly (check the database against the exchange)
@@ -375,7 +377,7 @@ from pathlib import Path
exchange = ccxt.binance({
'apiKey': '<apikey>',
'secret': '<secret>'
'secret': '<secret>',
'options': {'defaultType': 'swap'}
})
_ = exchange.load_markets()
@@ -418,6 +420,9 @@ This part of the documentation is aimed at maintainers, and shows how to create
### Create release branch
!!! Note
Make sure that the `stable` branch is up-to-date!
First, pick a commit that's about one week old (to not include latest additions to releases).
``` bash
@@ -430,14 +435,11 @@ Determine if crucial bugfixes have been made between this commit and the current
* Merge the release branch (stable) into this branch.
* Edit `freqtrade/__init__.py` and add the version matching the current date (for example `2019.7` for July 2019). Minor versions can be `2019.7.1` should we need to do a second release that month. Version numbers must follow allowed versions from PEP0440 to avoid failures pushing to pypi.
* Commit this part.
* push that branch to the remote and create a PR against the stable branch.
* Push that branch to the remote and create a PR against the **stable branch**.
* Update develop version to next version following the pattern `2019.8-dev`.
### Create changelog from git commits
!!! Note
Make sure that the `stable` branch is up-to-date!
``` bash
# Needs to be done before merging / pulling that branch.
@@ -137,7 +137,7 @@ $$ R = \frac{\text{average_profit}}{\text{average_loss}} = \frac{\mu_{win}}{\mu_
### Expectancy
By combining the Win Rate $W$ and and the Risk Reward ratio $R$ to create an expectancy ratio $E$. A expectance ratio is the expected return of the investment made in a trade. We can compute the value of $E$ as follows:
By combining the Win Rate $W$ and the Risk Reward ratio $R$ to create an expectancy ratio $E$. A expectance ratio is the expected return of the investment made in a trade. We can compute the value of $E$ as follows:
For Binance, it is suggested to add `"BNB/<STAKE>"` to your blacklist to avoid issues, unless you are willing to maintain enough extra `BNB` on the account or unless you're willing to disable using `BNB` for fees.
Binance accounts may use `BNB` for fees, and if a trade happens to be on `BNB`, further trades may consume this position and make the initial BNB trade unsellable as the expected amount is not there anymore.
If not enough `BNB` is available to cover transaction fees, then fees will not be covered by `BNB` and no fee reduction will occur. Freqtrade will never buy BNB to cover for fees. BNB needs to be bought and monitored manually to this end.
### Binance sites
Binance has been split into 2, and users must use the correct ccxt exchange ID for their exchange, otherwise API keys are not recognized.
@@ -125,8 +127,17 @@ These settings will be checked on startup, and freqtrade will show an error if t
Freqtrade will not attempt to change these settings.
## Bingx
BingX supports [time_in_force](configuration.md#understand-order_time_in_force) with settings "GTC" (good till cancelled), "IOC" (immediate-or-cancel) and "PO" (Post only) settings.
!!! Tip "Stoploss on Exchange"
Bingx supports `stoploss_on_exchange` and can use both stop-limit and stop-market orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
## Kraken
Kraken supports [time_in_force](configuration.md#understand-order_time_in_force) with settings "GTC" (good till cancelled), "IOC" (immediate-or-cancel) and "PO" (Post only) settings.
!!! Tip "Stoploss on Exchange"
Kraken supports `stoploss_on_exchange` and can use both stop-loss-market and stop-loss-limit orders. It provides great advantages, so we recommend to benefit from it.
You can use either `"limit"` or `"market"` in the `order_types.stoploss` configuration setting to decide which type to use.
Please pay attention that rateLimit configuration entry holds delay in milliseconds between requests, NOT requests\sec rate.
So, in order to mitigate Kraken API "Rate limit exceeded" exception, this configuration should be increased, NOT decreased.
## Bittrex
### Order types
Bittrex does not support market orders. If you have a message at the bot startup about this, you should change order type values set in your configuration and/or in the strategy from `"market"` to `"limit"`. See some more details on this [here in the FAQ](faq.md#im-getting-the-exchange-bittrex-does-not-support-market-orders-message-and-cannot-run-my-strategy).
Bittrex also does not support `VolumePairlist` due to limited / split API constellation at the moment.
Please use `StaticPairlist`. Other pairlists (other than `VolumePairlist`) should not be affected.
### Volume pairlist
Bittrex does not support the direct usage of VolumePairList. This can however be worked around by using the advanced mode with `lookback_days: 1` (or more), which will emulate 24h volume.
Read more in the [pairlist documentation](plugins.md#volumepairlist-advanced-mode).
### Restricted markets
Bittrex split its exchange into US and International versions.
The International version has more pairs available, however the API always returns all pairs, so there is currently no automated way to detect if you're affected by the restriction.
If you have restricted pairs in your whitelist, you'll get a warning message in the log on Freqtrade startup for each restricted pair.
The warning message will look similar to the following:
If you're an "International" customer on the Bittrex exchange, then this warning will probably not impact you.
If you're a US customer, the bot will fail to create orders for these pairs, and you should remove them from your whitelist.
You can get a list of restricted markets by using the following snippet:
``` python
import ccxt
ct = ccxt.bittrex()
lm = ct.load_markets()
res = [p for p, x in lm.items() if 'US' in x['info']['prohibitedIn']]
print(res)
```
## Kucoin
Kucoin requires a passphrase for each api key, you will therefore need to add this key into the configuration so your exchange section looks as follows:
For Kucoin, it is suggested to add `"KCS/<STAKE>"` to your blacklist to avoid issues, unless you are willing to maintain enough extra `KCS` on the account or unless you're willing to disable using `KCS` for fees.
Kucoin accounts may use `KCS` for fees, and if a trade happens to be on `KCS`, further trades may consume this position and make the initial `KCS` trade unsellable as the expected amount is not there anymore.
## Huobi
## HTX (formerly Huobi)
!!! Tip "Stoploss on Exchange"
Huobi supports `stoploss_on_exchange` and uses `stop-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
HTX supports `stoploss_on_exchange` and uses `stop-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
## OKX (former OKEX)
@@ -283,25 +252,93 @@ OKX requires a passphrase for each api key, you will therefore need to add this
Gate.io allows the use of `POINT` to pay for fees. As this is not a tradable currency (no regular market available), automatic fee calculations will fail (and default to a fee of 0).
The configuration parameter `exchange.unknown_fee_rate` can be used to specify the exchange rate between Point and the stake currency. Obviously, changing the stake-currency will also require changes to this value.
Gate API keys require the following permissions on top of the market type you want to trade:
* "Spot Trade" _or_ "Perpetual Futures" (Read and Write) (either select both, or the one matching the market you want to trade)
* "Wallet" (read only)
* "Account" (read only)
Without these permissions, the bot will not start correctly and show errors like "permission missing".
## Bybit
Futures trading on bybit is currently supported for USDT markets, and will use isolated futures mode.
Users with unified accounts (there's no way back) can create a Sub-account which will start as "non-unified", and can therefore use isolated futures.
On startup, freqtrade will set the position mode to "One-way Mode" for the whole (sub)account. This avoids making this call over and over again (slowing down bot operations), but means that changes to this setting may result in exceptions and errors
On startup, freqtrade will set the position mode to "One-way Mode" for the whole (sub)account. This avoids making this call over and over again (slowing down bot operations), but means that changes to this setting may result in exceptions and errors.
As bybit doesn't provide funding rate history, the dry-run calculation is used for live trades as well.
API Keys for live futures trading (Subaccount on non-unified) must have the following permissions:
API Keys for live futures trading must have the following permissions:
* Read-write
* Contract - Orders
* Contract - Positions
We do strongly recommend to limit all API keys to the IP you're going to use it from.
!!! Warning "Unified accounts"
Freqtrade assumes accounts to be dedicated to the bot.
We therefore recommend the usage of one subaccount per bot. This is especially important when using unified accounts.
Other configurations (multiple bots on one account, manual non-bot trades on the bot account) are not supported and may lead to unexpected behavior.
!!! Tip "Stoploss on Exchange"
Bybit (futures only) supports `stoploss_on_exchange` and uses `stop-loss-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
On futures, Bybit supports both `stop-limit` as well as `stop-market` orders. You can use either `"limit"` or `"market"` in the `order_types.stoploss` configuration setting to decide which type to use.
## Bitmart
Bitmart requires the API key Memo (the name you give the API key) to go along with the exchange key and secret.
It's therefore required to pass the UID as well.
```json
"exchange": {
"name": "bitmart",
"uid": "your_bitmart_api_key_memo",
"secret": "your_exchange_secret",
"password": "your_exchange_api_key_password",
// ...
}
```
!!! Warning "Necessary Verification"
Bitmart requires Verification Lvl2 to successfully trade on the spot market through the API - even though trading via UI works just fine with just Lvl1 verification.
## Hyperliquid
!!! Tip "Stoploss on Exchange"
Hyperliquid supports `stoploss_on_exchange` and uses `stop-loss-limit` orders. It provides great advantages, so we recommend to benefit from it.
Hyperliquid is a Decentralized Exchange (DEX). Decentralized exchanges work a bit different compared to normal exchanges. Instead of authenticating private API calls using an API key, private API calls need to be signed with the private key of your wallet (We recommend using an api Wallet for this, generated either on Hyperliquid or in your wallet of choice).
This needs to be configured like this:
```json
"exchange": {
"name": "hyperliquid",
"walletAddress": "your_eth_wallet_address",
"privateKey": "your_api_private_key",
// ...
}
```
* walletAddress in hex format: `0x<40hexcharacters>` - Can be easily copied from your wallet - and should be your wallet address, not your API Wallet Address.
* privateKey in hex format: `0x<64hexcharacters>` - Use the key the API Wallet shows on creation.
Hyperliquid handles deposits and withdrawals on the Arbitrum One chain, a Layer 2 scaling solution built on top of Ethereum. Hyperliquid uses USDC as quote / collateral. The process of depositing USDC on Hyperliquid requires a couple of steps, see [how to start trading](https://hyperliquid.gitbook.io/hyperliquid-docs/onboarding/how-to-start-trading) for details on what steps are needed.
!!! Note "Hyperliquid general usage Notes"
Hyperliquid does not support market orders, however ccxt will simulate market orders by placing limit orders with a maximum slippage of 5%.
Unfortunately, hyperliquid only offers 5000 historic candles, so backtesting will either need to build candles historically (by waiting and downloading the data incrementally over time) - or will be limited to the last 5000 candles.
!!! Info "Some general best practices (non exhaustive)"
* Beware of supply chain attacks, like pip package poisoning etcetera. Whenever you use your private key, make sure your environment is safe.
* Don't use your actual wallet private key for trading. Use the Hyperliquid [API generator](https://app.hyperliquid.xyz/API) to create a separate API wallet.
* Don't store your actual wallet private key on the server you use for freqtrade. Use the API wallet private key instead. This key won't allow withdrawals, only trading.
* Always keep your mnemonic phrase and private key private.
* Don't use the same mnemonic as the one you had to backup when initializing a hardware wallet, using the same mnemonic basically deletes the security of your hardware wallet.
* Create a different software wallet, only transfer the funds you want to trade with to that wallet, and use that wallet to trade on Hyperliquid.
* If you have funds you don't want to use for trading (after making a profit for example), transfer them back to your hardware wallet.
## All exchanges
Should you experience constant errors with Nonce (like `InvalidNonce`), it is best to regenerate the API keys. Resetting Nonce is difficult and it's usually easier to regenerate the API keys.
@@ -311,7 +348,7 @@ Should you experience constant errors with Nonce (like `InvalidNonce`), it is be
* The Ocean (exchange id: `theocean`) exchange uses Web3 functionality and requires `web3` python package to be installed:
```shell
$ pip3 install web3
pip3 install web3
```
### Getting latest price / Incomplete candles
@@ -319,7 +356,7 @@ $ pip3 install web3
Most exchanges return current incomplete candle via their OHLCV/klines API interface.
By default, Freqtrade assumes that incomplete candle is fetched from the exchange and removes the last candle assuming it's the incomplete candle.
Whether your exchange returns incomplete candles or not can be checked using [the helper script](developer.md#Incomplete-candles) from the Contributor documentation.
Whether your exchange returns incomplete candles or not can be checked using [the helper script](developer.md#incomplete-candles) from the Contributor documentation.
Due to the danger of repainting, Freqtrade does not allow you to use this incomplete candle.
Freqtrade supports spot trading, as well as (isolated) futures trading for some selected exchanges. Please refer to the [documentation start page](index.md#supported-futures-exchanges-experimental) for an uptodate list of supported exchanges.
Freqtrade supports spot trading, as well as (isolated) futures trading for some selected exchanges. Please refer to the [documentation start page](index.md#supported-futures-exchanges-experimental) for an up-to-date list of supported exchanges.
### Can my bot open short positions?
@@ -14,7 +14,7 @@ In spot markets, you can in some cases use leveraged spot tokens, which reflect
### Can my bot trade options or futures?
Futures trading is supported for selected exchanges. Please refer to the [documentation start page](index.md#supported-futures-exchanges-experimental) for an uptodate list of supported exchanges.
Futures trading is supported for selected exchanges. Please refer to the [documentation start page](index.md#supported-futures-exchanges-experimental) for an up-to-date list of supported exchanges.
## Beginner Tips & Tricks
@@ -100,6 +100,19 @@ You can use the `/stopentry` command in Telegram to prevent future trade entry,
Please look at the [advanced setup documentation Page](advanced-setup.md#running-multiple-instances-of-freqtrade).
### I'm getting "Impossible to load Strategy" when starting the bot
This error message is shown when the bot cannot load the strategy.
Usually, you can use `freqtrade list-strategies` to list all available strategies.
The output of this command will also include a status column, showing if the strategy can be loaded.
Please check the following:
* Are you using the correct strategy name? The strategy name is case-sensitive and must correspond to the Strategy class name (not the filename!).
* Is the strategy in the `user_data/strategies` directory, and has the file-ending `.py`?
* Does the bot show other warnings before this error? Maybe you're missing some dependencies for the strategy - which would be highlighted in the log.
* In case of docker - is the strategy directory mounted correctly (check the volumes part of the docker-compose file)?
### I'm getting "Missing data fillup" messages in the log
This message is just a warning that the latest candles had missing candles in them.
@@ -128,15 +141,9 @@ This warning can point to one of the below problems:
* Barely traded pair -> Check the pair on the exchange webpage, look at the timeframe your strategy uses. If the pair does not have any volume in some candles (usually visualized with a "volume 0" bar, and a "_" as candle), this pair did not have any trades in this timeframe. These pairs should ideally be avoided, as they can cause problems with order-filling.
* API problem -> API returns wrong data (this only here for completeness, and should not happen with supported exchanges).
### I'm getting the "RESTRICTED_MARKET" message in the log
Currently known to happen for US Bittrex users.
Read [the Bittrex section about restricted markets](exchanges.md#restricted-markets) for more information.
### I'm getting the "Exchange XXX does not support market orders." message and cannot run my strategy
As the message says, your exchange does not support market orders and you have one of the [order types](configuration.md/#understand-order_types) set to "market". Your strategy was probably written with other exchanges in mind and sets "market" orders for "stoploss" orders, which is correct and preferable for most of the exchanges supporting market orders (but not for Bittrex and Gate.io).
As the message says, your exchange does not support market orders and you have one of the [order types](configuration.md/#understand-order_types) set to "market". Your strategy was probably written with other exchanges in mind and sets "market" orders for "stoploss" orders, which is correct and preferable for most of the exchanges supporting market orders (but not for Gate.io).
To fix this, redefine order types in the strategy to use "limit" instead of "market":
Freqtrade provides a builtin webserver, which can serve [FreqUI](https://github.com/freqtrade/frequi), the freqtrade frontend.
By default, the UI is automatically installed as part of the installation (script, docker).
freqUI can also be manually installed by using the `freqtrade install-ui` command.
This same command can also be used to update freqUI to new new releases.
Once the bot is started in trade / dry-run mode (with `freqtrade trade`) - the UI will be available under the configured API port (by default `http://127.0.0.1:8080`).
??? Note "Looking to contribute to freqUI?"
Developers should not use this method, but instead clone the corresponding use the method described in the [freqUI repository](https://github.com/freqtrade/frequi) to get the source-code of freqUI. A working installation of node will be required to build the frontend.
!!! tip "freqUI is not required to run freqtrade"
freqUI is an optional component of freqtrade, and is not required to run the bot.
It is a frontend that can be used to monitor the bot and to interact with it - but freqtrade itself will work perfectly fine without it.
## Configuration
FreqUI does not have it's own configuration file - but assumes a working setup for the [rest-api](rest-api.md) is available.
Please refer to the corresponding documentation page to get setup with freqUI
## UI
FreqUI is a modern, responsive web application that can be used to monitor and interact with your bot.
FreqUI provides a light, as well as a dark theme.
Themes can be easily switched via a prominent button at the top of the page.
The theme of the screenshots on this page will adapt to the selected documentation Theme, so to see the dark (or light) version, please switch the theme of the Documentation.
### Login
The below screenshot shows the login screen of freqUI.
The Cors error shown in this screenshot is due to the fact that the UI is running on a different port than the API, and [CORS](#cors) has not been setup correctly yet.
### Trade view
The trade view allows you to visualize the trades that the bot is making and to interact with the bot.
On this page, you can also interact with the bot by starting and stopping it and - if configured - force trade entries and exits.
FreqUI Plots can be configured either via a `plot_config` configuration object in the strategy (which can be loaded via "from strategy" button) or via the UI.
Multiple plot configurations can be created and switched at will - allowing for flexible, different views into your charts.
The plot configuration can be accessed via the "Plot Configurator" (Cog icon) button in the top right corner of the trade view.
When freqtrade is started in [webserver mode](utils.md#webserver-mode) (freqtrade started with `freqtrade webserver`), the backtesting view becomes available.
This view allows you to backtest strategies and visualize the results.
You can also load and visualize previous backtest results, as well as compare the results with each other.
@@ -32,6 +32,9 @@ FreqAI is configured through the typical [Freqtrade config file](configuration.m
A full example config is available in `config_examples/config_freqai.example.json`.
!!! Note
The `identifier` is commonly overlooked by newcomers, however, this value plays an important role in your configuration. This value is a unique ID that you choose to describe one of your runs. Keeping it the same allows you to maintain crash resilience as well as faster backtesting. As soon as you want to try a new run (new features, new model, etc.), you should change this value (or delete the `user_data/models/unique-id` folder. More details available in the [parameter table](freqai-parameter-table.md#feature-parameters).
## Building a FreqAI strategy
The FreqAI strategy requires including the following lines of code in the standard [Freqtrade strategy](strategy-customization.md):
@@ -162,7 +165,8 @@ Below are the values you can expect to include/use inside a typical strategy dat
| `df['&*_std/mean']` | Standard deviation and mean values of the defined labels during training (or live tracking with `fit_live_predictions_candles`). Commonly used to understand the rarity of a prediction (use the z-score as shown in `templates/FreqaiExampleStrategy.py` and explained [here](#creating-a-dynamic-target-threshold) to evaluate how often a particular prediction was observed during training or historically with `fit_live_predictions_candles`). <br>**Datatype:** Float.
| `df['do_predict']` | Indication of an outlier data point. The return value is integer between -2 and 2, which lets you know if the prediction is trustworthy or not. `do_predict==1` means that the prediction is trustworthy. If the Dissimilarity Index (DI, see details [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di)) of the input data point is above the threshold defined in the config, FreqAI will subtract 1 from `do_predict`, resulting in `do_predict==0`. If `use_SVM_to_remove_outliers` is active, the Support Vector Machine (SVM, see details [here](freqai-feature-engineering.md#identifying-outliers-using-a-support-vector-machine-svm)) may also detect outliers in training and prediction data. In this case, the SVM will also subtract 1 from `do_predict`. If the input data point was considered an outlier by the SVM but not by the DI, or vice versa, the result will be `do_predict==0`. If both the DI and the SVM considers the input data point to be an outlier, the result will be `do_predict==-1`. As with the SVM, if `use_DBSCAN_to_remove_outliers` is active, DBSCAN (see details [here](freqai-feature-engineering.md#identifying-outliers-with-dbscan)) may also detect outliers and subtract 1 from `do_predict`. Hence, if both the SVM and DBSCAN are active and identify a datapoint that was above the DI threshold as an outlier, the result will be `do_predict==-2`. A particular case is when `do_predict == 2`, which means that the model has expired due to exceeding `expired_hours`. <br>**Datatype:** Integer between -2 and 2.
| `df['DI_values']` | Dissimilarity Index (DI) values are proxies for the level of confidence FreqAI has in the prediction. A lower DI means the prediction is close to the training data, i.e., higher prediction confidence. See details about the DI [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di). <br>**Datatype:** Float.
| `df['%*']` | Any dataframe column prepended with `%` in `feature_engineering_*()` is treated as a training feature. For example, you can include the RSI in the training feature set (similar to in `templates/FreqaiExampleStrategy.py`) by setting `df['%-rsi']`. See more details on how this is done [here](freqai-feature-engineering.md). <br>**Note:** Since the number of features prepended with `%` can multiply very quickly (10s of thousands of features are easily engineered using the multiplictative functionality of, e.g., `include_shifted_candles` and `include_timeframes` as described in the [parameter table](freqai-parameter-table.md)), these features are removed from the dataframe that is returned from FreqAI to the strategy. To keep a particular type of feature for plotting purposes, you would prepend it with `%%`. <br>**Datatype:** Depends on the output of the model.
| `df['%*']` | Any dataframe column prepended with `%` in `feature_engineering_*()` is treated as a training feature. For example, you can include the RSI in the training feature set (similar to in `templates/FreqaiExampleStrategy.py`) by setting `df['%-rsi']`. See more details on how this is done [here](freqai-feature-engineering.md). <br>**Note:** Since the number of features prepended with `%` can multiply very quickly (10s of thousands of features are easily engineered using the multiplictative functionality of, e.g., `include_shifted_candles` and `include_timeframes` as described in the [parameter table](freqai-parameter-table.md)), these features are removed from the dataframe that is returned from FreqAI to the strategy. To keep a particular type of feature for plotting purposes, you would prepend it with `%%` (see details below). <br>**Datatype:** Depends on the feature created by the user.
| `df['%%*']` | Any dataframe column prepended with `%%` in `feature_engineering_*()` is treated as a training feature, just the same as the above `%` prepend. However, in this case, the features are returned back to the strategy for FreqUI/plot-dataframe plotting and monitoring in Dry/Live/Backtesting <br>**Datatype:** Depends on the feature created by the user. Please note that features created in `feature_engineering_expand()` will have automatic FreqAI naming schemas depending on the expansions that you configured (i.e. `include_timeframes`, `include_corr_pairlist`, `indicators_periods_candles`, `include_shifted_candles`). So if you want to plot `%%-rsi` from `feature_engineering_expand_all()`, the final naming scheme for your plotting config would be: `%%-rsi-period_10_ETH/USDT:USDT_1h` for the `rsi` feature with `period=10`, `timeframe=1h`, and `pair=ETH/USDT:USDT` (the `:USDT` is added if you are using futures pairs). It is useful to simply add `print(dataframe.columns)` in your `populate_indicators()` after `self.freqai.start()` to see the full list of available features that are returned to the strategy for plotting purposes.
## Setting the `startup_candle_count`
@@ -289,10 +293,10 @@ class MyCoolPyTorchClassifier(BasePyTorchClassifier):
@@ -7,7 +7,7 @@ Low level feature engineering is performed in the user strategy within a set of
| Function | Description |
|---------------|-------------|
| `feature_engineering_expand_all()` | This optional function will automatically expand the defined features on the config defined `indicator_periods_candles`, `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`.
| `feature_engineering_expand_basic()` | This optional function will automatically expand the defined features on the config defined `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`. Note: this function does *not* expand across `include_periods_candles`.
| `feature_engineering_expand_basic()` | This optional function will automatically expand the defined features on the config defined `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`. Note: this function does *not* expand across `indicator_periods_candles`.
| `feature_engineering_standard()` | This optional function will be called once with the dataframe of the base timeframe. This is the final function to be called, which means that the dataframe entering this function will contain all the features and columns from the base asset created by the other `feature_engineering_expand` functions. This function is a good place to do custom exotic feature extractions (e.g. tsfresh). This function is also a good place for any feature that should not be auto-expanded upon (e.g., day of the week).
| `set_freqai_targets()` | Required function to set the targets for the model. All targets must be prepended with `&` to be recognized by the FreqAI internals.
@@ -224,7 +224,7 @@ where $W_i$ is the weight of data point $i$ in a total set of $n$ data points. B
## Building the data pipeline
By default, FreqAI builds a dynamic pipeline based on user congfiguration settings. The default settings are robust and designed to work with a variety of methods. These two steps are a `MinMaxScaler(-1,1)` and a `VarianceThreshold` which removes any column that has 0 variance. Users can activate other steps with more configuration parameters. For example if users add `use_SVM_to_remove_outliers: true` to the `freqai` config, then FreqAI will automatically add the [`SVMOutlierExtractor`](#identifying-outliers-using-a-support-vector-machine-svm) to the pipeline. Likewise, users can add `principal_component_analysis: true` to the `freqai` config to activate PCA. The [DissimilarityIndex](#identifying-outliers-with-the-dissimilarity-index-di) is activated with `DI_threshold: 1`. Finally, noise can also be added to the data with `noise_standard_deviation: 0.1`. Finally, users can add [DBSCAN](#identifying-outliers-with-dbscan) outlier removal with `use_DBSCAN_to_remove_outliers: true`.
By default, FreqAI builds a dynamic pipeline based on user configuration settings. The default settings are robust and designed to work with a variety of methods. These two steps are a `MinMaxScaler(-1,1)` and a `VarianceThreshold` which removes any column that has 0 variance. Users can activate other steps with more configuration parameters. For example if users add `use_SVM_to_remove_outliers: true` to the `freqai` config, then FreqAI will automatically add the [`SVMOutlierExtractor`](#identifying-outliers-using-a-support-vector-machine-svm) to the pipeline. Likewise, users can add `principal_component_analysis: true` to the `freqai` config to activate PCA. The [DissimilarityIndex](#identifying-outliers-with-the-dissimilarity-index-di) is activated with `DI_threshold: 1`. Finally, noise can also be added to the data with `noise_standard_deviation: 0.1`. Finally, users can add [DBSCAN](#identifying-outliers-with-dbscan) outlier removal with `use_DBSCAN_to_remove_outliers: true`.
!!! note "More information available"
Please review the [parameter table](freqai-parameter-table.md) for more information on these parameters.
@@ -235,7 +235,7 @@ By default, FreqAI builds a dynamic pipeline based on user congfiguration settin
Users are encouraged to customize the data pipeline to their needs by building their own data pipeline. This can be done by simply setting `dk.feature_pipeline` to their desired `Pipeline` object inside their `IFreqaiModel` `train()` function, or if they prefer not to touch the `train()` function, they can override `define_data_pipeline`/`define_label_pipeline` functions in their `IFreqaiModel`:
!!! note "More information available"
FreqAI uses the the [`DataSieve`](https://github.com/emergentmethods/datasieve) pipeline, which follows the SKlearn pipeline API, but adds, among other features, coherence between the X, y, and sample_weight vector point removals, feature removal, feature name following.
FreqAI uses the [`DataSieve`](https://github.com/emergentmethods/datasieve) pipeline, which follows the SKlearn pipeline API, but adds, among other features, coherence between the X, y, and sample_weight vector point removals, feature removal, feature name following.
```python
from datasieve.transforms import SKLearnWrapper, DissimilarityIndex
@@ -391,3 +391,18 @@ Given a number of data points $N$, and a distance $\varepsilon$, DBSCAN clusters
FreqAI uses `sklearn.cluster.DBSCAN` (details are available on scikit-learn's webpage [here](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html) (external website)) with `min_samples` ($N$) taken as 1/4 of the no. of time points (candles) in the feature set. `eps` ($\varepsilon$) is computed automatically as the elbow point in the *k-distance graph* computed from the nearest neighbors in the pairwise distances of all data points in the feature set.
### Data dimensionality reduction with Principal Component Analysis
You can reduce the dimensionality of your features by activating the principal_component_analysis in the config:
```json
"freqai": {
"feature_parameters" : {
"principal_component_analysis": true
}
}
```
This will perform PCA on the features and reduce their dimensionality so that the explained variance of the data set is >= 0.999. Reducing data dimensionality makes training the model faster and hence allows for more up-to-date models.
@@ -22,6 +22,7 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| `write_metrics_to_disk` | Collect train timings, inference timings and cpu usage in json file. <br>**Datatype:** Boolean. <br> Default: `False`
| `data_kitchen_thread_count` | <br> Designate the number of threads you want to use for data processing (outlier methods, normalization, etc.). This has no impact on the number of threads used for training. If user does not set it (default), FreqAI will use max number of threads - 2 (leaving 1 physical core available for Freqtrade bot and FreqUI) <br>**Datatype:** Positive integer.
| `activate_tensorboard` | <br> Indicate whether or not to activate tensorboard for the tensorboard enabled modules (currently Reinforcment Learning, XGBoost, Catboost, and PyTorch). Tensorboard needs Torch installed, which means you will need the torch/RL docker image or you need to answer "yes" to the install question about whether or not you wish to install Torch. <br>**Datatype:** Boolean. <br> Default: `True`.
| `wait_for_training_iteration_on_reload` | <br> When using /reload or ctrl-c, wait for the current training iteration to finish before completing graceful shutdown. If set to `False`, FreqAI will break the current training iteration, allowing you to shutdown gracefully more quickly, but you will lose your current training iteration. <br>**Datatype:** Boolean. <br> Default: `True`.
### Feature parameters
@@ -31,12 +32,12 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| `feature_parameters` | A dictionary containing the parameters used to engineer the feature set. Details and examples are shown [here](freqai-feature-engineering.md). <br>**Datatype:** Dictionary.
| `include_timeframes` | A list of timeframes that all indicators in `feature_engineering_expand_*()` will be created for. The list is added as features to the base indicators dataset. <br>**Datatype:** List of timeframes (strings).
| `include_corr_pairlist` | A list of correlated coins that FreqAI will add as additional features to all `pair_whitelist` coins. All indicators set in `feature_engineering_expand_*()` during feature engineering (see details [here](freqai-feature-engineering.md)) will be created for each correlated coin. The correlated coins features are added to the base indicators dataset. <br>**Datatype:** List of assets (strings).
| `label_period_candles` | Number of candles into the future that the labels are created for. This is used in `feature_engineering_expand_all()` (see `templates/FreqaiExampleStrategy.py` for detailed usage). You can create custom labels and choose whether to make use of this parameter or not. <br>**Datatype:** Positive integer.
| `label_period_candles` | Number of candles into the future that the labels are created for. This can be used in `set_freqai_targets()` (see `templates/FreqaiExampleStrategy.py` for detailed usage). This parameter is not necessarily required, you can create custom labels and choose whether to make use of this parameter or not. Please see `templates/FreqaiExampleStrategy.py` to see the example usage.<br>**Datatype:** Positive integer.
| `include_shifted_candles` | Add features from previous candles to subsequent candles with the intent of adding historical information. If used, FreqAI will duplicate and shift all features from the `include_shifted_candles` previous candles so that the information is available for the subsequent candle. <br>**Datatype:** Positive integer.
| `weight_factor` | Weight training data points according to their recency (see details [here](freqai-feature-engineering.md#weighting-features-for-temporal-importance)). <br>**Datatype:** Positive float (typically <1).
|`indicator_max_period_candles`|**No longer used (#7325)**.Replacedby`startup_candle_count`whichissetinthe [strategy](freqai-configuration.md#building-a-freqai-strategy).`startup_candle_count`istimeframeindependentanddefinesthemaximum*period*usedin`feature_engineering_*()`forindicatorcreation.FreqAIusesthisparametertogetherwiththemaximumtimeframein`include_time_frames`tocalculatehowmanydatapointstodownloadsuchthatthefirstdatapointdoesnotincludeaNaN.<br>**Datatype:** Positive integer.
| `indicator_periods_candles` | Time periods to calculate indicators for. The indicators are added to the base indicator dataset. <br>**Datatype:** List of positive integers.
| `principal_component_analysis` | Automatically reduce the dimensionality of the data set using Principal Component Analysis. See details about how it works [here](#reducing-data-dimensionality-with-principal-component-analysis) <br>**Datatype:** Boolean. <br> Default: `False`.
| `principal_component_analysis` | Automatically reduce the dimensionality of the data set using Principal Component Analysis. See details about how it works [here](freqai-feature-engineering.md#data-dimensionality-reduction-with-principal-component-analysis) <br>**Datatype:** Boolean. <br> Default: `False`.
| `plot_feature_importances` | Create a feature importance plot for each model for the top/bottom `plot_feature_importances` number of features. Plot is stored in `user_data/models/<identifier>/sub-train-<COIN>_<timestamp>.html`. <br>**Datatype:** Integer. <br> Default: `0`.
| `DI_threshold` | Activates the use of the Dissimilarity Index for outlier detection when set to > 0. See details about how it works [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di). <br>**Datatype:** Positive float (typically <1).
@@ -55,7 +56,7 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| | **Data split parameters within the `freqai.data_split_parameters` sub dictionary**
| `data_split_parameters` | Include any additional parameters available from scikit-learn `test_train_split()`, which are shown [here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html) (external website). <br>**Datatype:** Dictionary.
| `test_size` | The fraction of data that should be used for testing instead of training. <br>**Datatype:** Positive float <1.
@@ -74,9 +75,8 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| | **Reinforcement Learning Parameters within the `freqai.rl_config` sub dictionary**
| `rl_config` | A dictionary containing the control parameters for a Reinforcement Learning model. <br>**Datatype:** Dictionary.
| `train_cycles` | Training time steps will be set based on the `train_cycles * number of training data points. <br> **Datatype:** Integer.
| `cpu_count` | Number of processors to dedicate to the Reinforcement Learning training process. <br> **Datatype:** int.
| `max_trade_duration_candles`| Guides the agent training to keep trades below desired length. Example usage shown in `prediction_models/ReinforcementLearner.py` within the customizable `calculate_reward()` function. <br> **Datatype:** int.
| `model_type` | Model string from stable_baselines3 or SBcontrib. Available strings include: `'TRPO', 'ARS', 'RecurrentPPO', 'MaskablePPO', 'PPO', 'A2C', 'DQN'`. User should ensure that `model_training_parameters` match those available to the corresponding stable_baselines3 model by visiting their documentaiton. [PPO doc](https://stable-baselines3.readthedocs.io/en/master/modules/ppo.html) (external website) <br> **Datatype:** string.
| `model_type` | Model string from stable_baselines3 or SBcontrib. Available strings include: `'TRPO', 'ARS', 'RecurrentPPO', 'MaskablePPO', 'PPO', 'A2C', 'DQN'`. User should ensure that `model_training_parameters` match those available to the corresponding stable_baselines3 model by visiting their documentation. [PPO doc](https://stable-baselines3.readthedocs.io/en/master/modules/ppo.html) (external website) <br> **Datatype:** string.
| `policy_type` | One of the available policy types from stable_baselines3 <br> **Datatype:** string.
| `max_training_drawdown_pct` | The maximum drawdown that the agent is allowed to experience during training. <br> **Datatype:** float. <br> Default: 0.8
| `cpu_count` | Number of threads/cpus to dedicate to the Reinforcement Learning training process (depending on if `ReinforcementLearning_multiproc` is selected or not). Recommended to leave this untouched, by default, this value is set to the total number of physical cores minus 1. <br> **Datatype:** int.
@@ -142,7 +142,7 @@ Parameter details can be found [here](freqai-parameter-table.md), but in general
As you begin to modify the strategy and the prediction model, you will quickly realize some important differences between the Reinforcement Learner and the Regressors/Classifiers. Firstly, the strategy does not set a target value (no labels!). Instead, you set the `calculate_reward()` function inside the `MyRLEnv` class (see below). A default `calculate_reward()` is provided inside `prediction_models/ReinforcementLearner.py` to demonstrate the necessary building blocks for creating rewards, but this is *not* designed for production. Users *must* create their own custom reinforcement learning model class or use a pre-built one from outside the Freqtrade source code and save it to `user_data/freqaimodels`. It is inside the `calculate_reward()` where creative theories about the market can be expressed. For example, you can reward your agent when it makes a winning trade, and penalize the agent when it makes a losing trade. Or perhaps, you wish to reward the agent for entering trades, and penalize the agent for sitting in trades too long. Below we show examples of how these rewards are all calculated:
!!! note "Hint"
The best reward functions are ones that are continuously differentiable, and well scaled. In other words, adding a single large negative penalty to a rare event is not a good idea, and the neural net will not be able to learn that function. Instead, it is better to add a small negative penalty to a common event. This will help the agent learn faster. Not only this, but you can help improve the continuity of your rewards/penalties by having them scale with severity according to some linear/exponential functions. In other words, you'd slowly scale the penalty as the duration of the trade increases. This is better than a single large penalty occuring at a single point in time.
The best reward functions are ones that are continuously differentiable, and well scaled. In other words, adding a single large negative penalty to a rare event is not a good idea, and the neural net will not be able to learn that function. Instead, it is better to add a small negative penalty to a common event. This will help the agent learn faster. Not only this, but you can help improve the continuity of your rewards/penalties by having them scale with severity according to some linear/exponential functions. In other words, you'd slowly scale the penalty as the duration of the trade increases. This is better than a single large penalty occurring at a single point in time.
This way, you can return to using any model you wish by simply specifying the `identifier`.
!!! Note
Backtesting calls `set_freqai_targets()` one time for each backtest window (where the number of windows is the full backtest timerange divided by the `backtest_period_days` parameter). Doing this means that the targets simulate dry/live behavior without look ahead bias. However, the definition of the features in `feature_engineering_*()` is performed once on the entire backtest timerange. This means that you should be sure that features do look-ahead into the future.
Backtesting calls `set_freqai_targets()` one time for each backtest window (where the number of windows is the full backtest timerange divided by the `backtest_period_days` parameter). Doing this means that the targets simulate dry/live behavior without look ahead bias. However, the definition of the features in `feature_engineering_*()` is performed once on the entire training timerange. This means that you should be sure that features do not look-ahead into the future.
More details about look-ahead bias can be found in [Common Mistakes](strategy-customization.md#common-mistakes-when-developing-strategies).
---
### Saving prediction data
### Saving backtesting prediction data
To allow for tweaking your strategy (**not** the features!), FreqAI will automatically save the predictions during backtesting so that they can be reused for future backtests and live runs using the same `identifier` model. This provides a performance enhancement geared towards enabling **high-level hyperopting** of entry/exit criteria.
An additional directory called `backtesting_predictions`, which contains all the predictions stored in `hdf` format, will be created in the `unique-id` folder.
An additional directory called `backtesting_predictions`, which contains all the predictions stored in `feather` format, will be created in the `unique-id` folder.
To change your **features**, you **must** set a new `identifier` in the config to signal to FreqAI to train new models.
To save the models generated during a particular backtest so that you can start a live deployment from one of them instead of training a new model, you must set `save_backtest_models` to `True` in the config.
!!! Note
To ensure that the model can be reused, freqAI will call your strategy with a dataframe of length 1.
If your strategy requires more data than this to generate the same features, you can't reuse backtest predictions for live deployment and need to update your `identifier` for each new backtest.
### Backtest live collected predictions
FreqAI allow you to reuse live historic predictions through the backtest parameter `--freqai-backtest-live-models`. This can be useful when you want to reuse predictions generated in dry/run for comparison or other study.
The `--timerange` parameter must not be informed, as it will be automatically calculated through the data in the historic predictions file.
### Downloading data to cover the full backtest period
For live/dry deployments, FreqAI will download the necessary data automatically. However, to use backtesting functionality, you need to download the necessary data using `download-data` (details [here](data-download.md#data-downloading)). You need to pay careful attention to understanding how much *additional* data needs to be downloaded to ensure that there is a sufficient amount of training data *before* the start of the backtesting time range. The amount of additional data can be roughly estimated by moving the start date of the time range backwards by `train_period_days` and the `startup_candle_count` (see the [parameter table](freqai-parameter-table.md) for detailed descriptions of these parameters) from the beginning of the desired backtesting time range.
@@ -114,6 +114,11 @@ Here we compile some external materials that provide deeper looks into various c
- [Real-time head-to-head: Adaptive modeling of financial market data using XGBoost and CatBoost](https://emergentmethods.medium.com/real-time-head-to-head-adaptive-modeling-of-financial-market-data-using-xgboost-and-catboost-995a115a7495)
- [FreqAI - from price to prediction](https://emergentmethods.medium.com/freqai-from-price-to-prediction-6fadac18b665)
## Support
You can find support for FreqAI in a variety of places, including the [Freqtrade discord](https://discord.gg/Jd8JYeWHc4), the dedicated [FreqAI discord](https://discord.gg/7AMWACmbjT), and in [github issues](https://github.com/freqtrade/freqtrade/issues).
## Credits
FreqAI is developed by a group of individuals who all contribute specific skillsets to the project.
Storage format for downloaded candle (OHLCV) data.
(default: `json`).
(default: `feather`).
--max-open-trades INT
Override the value of the `max_open_trades`
configuration setting.
@@ -81,10 +80,6 @@ optional arguments:
--eps, --enable-position-stacking
Allow buying the same pair multiple times (position
stacking).
--dmmp, --disable-max-market-positions
Disable applying `max_open_trades` during backtest
(same as setting `max_open_trades` to a very high
number).
--enable-protections, --enableprotections
Enable protections for backtesting.Will slow
backtesting down by a considerable amount, but will
@@ -134,7 +129,8 @@ optional arguments:
Common arguments:
-v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
--logfile FILE Log to the file specified. Special values are:
--logfile FILE, --log-file FILE
Log to the file specified. Special values are:
'syslog', 'journald'. See the documentation for more
details.
-V, --version show program's version number and exit
@@ -143,7 +139,7 @@ Common arguments:
`userdir/config.json` or `config.json` whichever
exists). Multiple --config options may be used. Can be
set to `-` to read config from stdin.
-d PATH, --datadir PATH
-d PATH, --datadir PATH, --data-dir PATH
Path to directory with historical backtesting data.
--userdir PATH, --user-data-dir PATH
Path to userdata directory.
@@ -337,11 +333,15 @@ There are four parameter types each suited for different purposes.
* `CategoricalParameter` - defines a parameter with a predetermined number of choices.
* `BooleanParameter` - Shorthand for `CategoricalParameter([True, False])` - great for "enable" parameters.
!!! Tip "Disabling parameter optimization"
Each parameter takes two boolean parameters:
* `load` - when set to `False` it will not load values configured in `buy_params` and `sell_params`.
* `optimize` - when set to `False` parameter will not be included in optimization process.
Use these parameters to quickly prototype various ideas.
### Parameter options
There are two parameter options that can help you to quickly test various ideas:
* `optimize` - when set to `False`, the parameter will not be included in optimization process. (Default: True)
* `load` - when set to `False`, results of a previous hyperopt run (in `buy_params` and `sell_params` either in your strategy or the JSON output file) will not be used as the starting value for subsequent hyperopts. The default value specified in the parameter will be used instead. (Default: True)
!!! Tip "Effects of `load=False` on backtesting"
Be aware that setting the `load` option to `False` will mean backtesting will also use the default value specified in the parameter and *not* the value found through hyperoptimisation.
!!! Warning
Hyperoptable parameters cannot be used in `populate_indicators` - as hyperopt does not recalculate indicators for each epoch, so the starting value would be used in this case.
@@ -435,14 +435,13 @@ While this strategy is most likely too simple to provide consistent profit, it s
??? Hint "Performance tip"
During normal hyperopting, indicators are calculated once and supplied to each epoch, linearly increasing RAM usage as a factor of increasing cores. As this also has performance implications, there are two alternatives to reduce RAM usage
* Move `ema_short` and `ema_long` calculations from `populate_indicators()` to `populate_entry_trend()`. Since `populate_entry_trend()` gonna be calculated every epochs, you don't need to use `.range` functionality.
* Move `ema_short` and `ema_long` calculations from `populate_indicators()` to `populate_entry_trend()`. Since `populate_entry_trend()` will be calculated every epoch, you don't need to use `.range` functionality.
* hyperopt provides `--analyze-per-epoch` which will move the execution of `populate_indicators()` to the epoch process, calculating a single value per parameter per epoch instead of using the `.range` functionality. In this case, `.range` functionality will only return the actually used value.
These alternatives will reduce RAM usage, but increase CPU usage. However, your hyperopting run will be less likely to fail due to Out Of Memory (OOM) issues.
Whether you are using `.range` functionality or the alternatives above, you should try to use space ranges as small as possible since this will improve CPU/RAM usage.
## Optimizing protections
Freqtrade can also optimize protections. How you optimize protections is up to you, and the following should be considered as example only.
@@ -586,14 +585,15 @@ Currently, the following loss functions are builtin:
* `ShortTradeDurHyperOptLoss` - (default legacy Freqtrade hyperoptimization loss function) - Mostly for short trade duration and avoiding losses.
* `OnlyProfitHyperOptLoss` - takes only amount of profit into consideration.
* `SharpeHyperOptLoss` - optimizes Sharpe Ratio calculated on trade returns relative to standard deviation.
* `SharpeHyperOptLossDaily` - optimizes Sharpe Ratio calculated on **daily** trade returns relative to standard deviation.
* `SortinoHyperOptLoss` - optimizes Sortino Ratio calculated on trade returns relative to **downside** standard deviation.
* `SharpeHyperOptLoss` - Optimizes Sharpe Ratio calculated on trade returns relative to standard deviation.
* `SharpeHyperOptLossDaily` - Optimizes Sharpe Ratio calculated on **daily** trade returns relative to standard deviation.
* `SortinoHyperOptLoss` - Optimizes Sortino Ratio calculated on trade returns relative to **downside** standard deviation.
* `SortinoHyperOptLossDaily` - optimizes Sortino Ratio calculated on **daily** trade returns relative to **downside** standard deviation.
* `MaxDrawDownHyperOptLoss` - Optimizes Maximum absolute drawdown.
* `MaxDrawDownRelativeHyperOptLoss` - Optimizes both maximum absolute drawdown while also adjusting for maximum relative drawdown.
* `CalmarHyperOptLoss` - Optimizes Calmar Ratio calculated on trade returns relative to max drawdown.
* `ProfitDrawDownHyperOptLoss` - Optimizes by max Profit & min Drawdown objective. `DRAWDOWN_MULT` variable within the hyperoptloss file can be adjusted to be stricter or more flexible on drawdown purposes.
* `MultiMetricHyperOptLoss` - Optimizes by several key metrics to achieve balanced performance. The primary focus is on maximizing Profit and minimizing Drawdown, while also considering additional metrics such as Profit Factor, Expectancy Ratio and Winrate. Moreover, it applies a penalty for epochs with a low number of trades, encouraging strategies with adequate trade frequency.
Creation of a custom loss function is covered in the [Advanced Hyperopt](advanced-hyperopt.md) part of the documentation.
@@ -761,7 +761,7 @@ Override the `roi_space()` method if you need components of the ROI tables to va
A sample for these methods can be found in the [overriding pre-defined spaces section](advanced-hyperopt.md#overriding-pre-defined-spaces).
!!! Note "Reduced search space"
To limit the search space further, Decimals are limited to 3 decimal places (a precision of 0.001). This is usually sufficient, every value more precise than this will usually result in overfitted results. You can however [overriding pre-defined spaces](advanced-hyperopt.md#pverriding-pre-defined-spaces) to change this to your needs.
To limit the search space further, Decimals are limited to 3 decimal places (a precision of 0.001). This is usually sufficient, every value more precise than this will usually result in overfitted results. You can however [overriding pre-defined spaces](advanced-hyperopt.md#overriding-pre-defined-spaces) to change this to your needs.
### Understand Hyperopt Stoploss results
@@ -803,7 +803,7 @@ If you have the `stoploss_space()` method in your custom hyperopt file, remove i
Override the `stoploss_space()` method and define the desired range in it if you need stoploss values to vary in other range during hyperoptimization. A sample for this method can be found in the [overriding pre-defined spaces section](advanced-hyperopt.md#overriding-pre-defined-spaces).
!!! Note "Reduced search space"
To limit the search space further, Decimals are limited to 3 decimal places (a precision of 0.001). This is usually sufficient, every value more precise than this will usually result in overfitted results. You can however [overriding pre-defined spaces](advanced-hyperopt.md#pverriding-pre-defined-spaces) to change this to your needs.
To limit the search space further, Decimals are limited to 3 decimal places (a precision of 0.001). This is usually sufficient, every value more precise than this will usually result in overfitted results. You can however [overriding pre-defined spaces](advanced-hyperopt.md#overriding-pre-defined-spaces) to change this to your needs.
### Understand Hyperopt Trailing Stop results
@@ -864,18 +864,15 @@ You can use the `--print-all` command line option if you would like to see all r
## Position stacking and disabling max market positions
In some situations, you may need to run Hyperopt (and Backtesting) with the
`--eps`/`--enable-position-staking` and `--dmmp`/`--disable-max-market-positions` arguments.
In some situations, you may need to run Hyperopt (and Backtesting) with the `--eps`/`--enable-position-staking` argument, or you may need to set `max_open_trades` to a very high number to disable the limit on the number of open trades.
By default, hyperopt emulates the behavior of the Freqtrade Live Run/Dry Run, where only one
open trade is allowed for every traded pair. The total number of trades open for all pairs
open trade per pair is allowed. The total number of trades open for all pairs
is also limited by the `max_open_trades` setting. During Hyperopt/Backtesting this may lead to
some potential trades to be hidden (or masked) by previously open trades.
potential trades being hidden (or masked) by already open trades.
The `--eps`/`--enable-position-stacking` argument allows emulation of buying the same pair multiple times,
while `--dmmp`/`--disable-max-market-positions` disables applying `max_open_trades`
during Hyperopt/Backtesting (which is equal to setting `max_open_trades` to a very high
number).
The `--eps`/`--enable-position-stacking` argument allows emulation of buying the same pair multiple times.
Using `--max-open-trades` with a very high number will disable the limit on the number of opentrades.
!!! Note
Dry/live runs will **NOT** use position stacking - therefore it does make sense to also validate the strategy without this as it's closer to reality.
@@ -916,12 +913,44 @@ Your epochs should therefore be aligned to the possible values - or you should b
After you run Hyperopt for the desired amount of epochs, you can later list all results for analysis, select only best or profitable once, and show the details for any of the epochs previously evaluated. This can be done with the `hyperopt-list` and `hyperopt-show` sub-commands. The usage of these sub-commands is described in the [Utils](utils.md#list-hyperopt-results) chapter.
## Output debug messages from your strategy
If you want to output debug messages from your strategy, you can use the `logging` module. By default, Freqtrade will output all messages with a level of `INFO` or higher.
Messages printed via `print()` will not be shown in the hyperopt output unless parallelism is disabled (`-j 1`).
It is recommended to use the `logging` module instead.
## Validate backtesting results
Once the optimized strategy has been implemented into your strategy, you should backtest this strategy to make sure everything is working as expected.
To achieve same the results (number of trades, their durations, profit, etc.) as during Hyperopt, please use the same configuration and parameters (timerange, timeframe, ...) used for hyperopt `--dmmp`/`--disable-max-market-positions` and `--eps`/`--enable-position-stacking` for Backtesting.
To achieve same the results (number of trades, their durations, profit, etc.) as during Hyperopt, please use the same configuration and parameters (timerange, timeframe, ...) used for hyperopt for Backtesting.
Should results not match, please double-check to make sure you transferred all conditions correctly.
Pay special care to the stoploss, max_open_trades and trailing stoploss parameters, as these are often set in configuration files, which override changes to the strategy.
You should also carefully review the log of your backtest to ensure that there were no parameters inadvertently set by the configuration (like `stoploss`, `max_open_trades` or `trailing_stop`).
### Why do my backtest results not match my hyperopt results?
Should results not match, check the following factors:
* You may have added parameters to hyperopt in `populate_indicators()` where they will be calculated only once **for all epochs**. If you are, for example, trying to optimise multiple SMA timeperiod values, the hyperoptable timeperiod parameter should be placed in `populate_entry_trend()` which is calculated every epoch. See [Optimizing an indicator parameter](https://www.freqtrade.io/en/stable/hyperopt/#optimizing-an-indicator-parameter).
* If you have disabled the auto-export of hyperopt parameters into the JSON parameters file, double-check to make sure you transferred all hyperopted values into your strategy correctly.
* Check the logs to verify what parameters are being set and what values are being used.
* Pay special care to the stoploss, max_open_trades and trailing stoploss parameters, as these are often set in configuration files, which override changes to the strategy. Check the logs of your backtest to ensure that there were no parameters inadvertently set by the configuration (like `stoploss`, `max_open_trades` or `trailing_stop`).
* Verify that you do not have an unexpected parameters JSON file overriding the parameters or the default hyperopt settings in your strategy.
* Verify that any protections that are enabled in backtesting are also enabled when hyperopting, and vice versa. When using `--space protection`, protections are auto-enabled for hyperopting.
This whole section is only necessary in cross-origin cases (where you multiple bot API's running on `localhost:8081`, `localhost:8082`, ...), and want to combine them into one FreqUI instance.
??? info "Technical explanation"
All web-based front-ends are subject to [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) - Cross-Origin Resource Sharing.
Since most of the requests to the Freqtrade API must be authenticated, a proper CORS policy is key to avoid security problems.
Also, the standard disallows `*` CORS policies for requests with credentials, so this setting must be set appropriately.
Users can allow access from different origin URL's to the bot API via the `CORS_origins` configuration setting.
It consists of a list of allowed URL's that are allowed to consume resources from the bot's API.
Assuming your application is deployed as `https://frequi.freqtrade.io/home/` - this would mean that the following configuration becomes necessary:
```jsonc
{
//...
"jwt_secret_key":"somethingrandom",
"CORS_origins":["https://frequi.freqtrade.io"],
//...
}
```
In the following (pretty common) case, FreqUI is accessible on `http://localhost:8080/trade` (this is what you see in your navbar when navigating to freqUI).
The correct configuration for this case is `http://localhost:8080` - the main part of the URL including the port.
```jsonc
{
//...
"jwt_secret_key":"somethingrandom",
"CORS_origins":["http://localhost:8080"],
//...
}
```
!!! Tip "trailing Slash"
The trailing slash is not allowed in the `CORS_origins` configuration (e.g. `"http://localhots:8080/"`).
Such a configuration will not take effect, and the cors errors will remain.
!!! Note
We strongly recommend to also set `jwt_secret_key` to something random and known only to yourself to avoid unauthorized access to your bot.
Pairlist Handlers define the list of pairs (pairlist) that the bot should trade. They are configured in the `pairlists` section of the configuration settings.
In your configuration, you can use Static Pairlist (defined by the [`StaticPairList`](#static-pair-list) Pairlist Handler) and Dynamic Pairlist (defined by the [`VolumePairList`](#volume-pair-list) Pairlist Handler).
In your configuration, you can use Static Pairlist (defined by the [`StaticPairList`](#static-pair-list) Pairlist Handler) and Dynamic Pairlist (defined by the [`VolumePairList`](#volume-pair-list) and [`PercentChangePairList`](#percent-change-pair-list) Pairlist Handlers).
Additionally, [`AgeFilter`](#agefilter), [`PrecisionFilter`](#precisionfilter), [`PriceFilter`](#pricefilter), [`ShuffleFilter`](#shufflefilter), [`SpreadFilter`](#spreadfilter) and [`VolatilityFilter`](#volatilityfilter) act as Pairlist Filters, removing certain pairs and/or moving their positions in the pairlist.
If multiple Pairlist Handlers are used, they are chained and a combination of all Pairlist Handlers forms the resulting pairlist the bot uses for trading and backtesting. Pairlist Handlers are executed in the sequence they are configured. You should always configure either `StaticPairList` or `VolumePairList` as the starting Pairlist Handler.
If multiple Pairlist Handlers are used, they are chained and a combination of all Pairlist Handlers forms the resulting pairlist the bot uses for trading and backtesting. Pairlist Handlers are executed in the sequence they are configured. You can define either `StaticPairList`, `VolumePairList`, `ProducerPairList`, `RemotePairList`, `MarketCapPairList` or `PercentChangePairList` as the starting Pairlist Handler.
Inactive markets are always removed from the resulting pairlist. Explicitly blacklisted pairs (those in the `pair_blacklist` configuration setting) are also always removed from the resulting pairlist.
@@ -22,8 +22,10 @@ You may also use something like `.*DOWN/BTC` or `.*UP/BTC` to exclude leveraged
* [`StaticPairList`](#static-pair-list) (default, if not configured differently)
@@ -53,7 +55,6 @@ It uses configuration from `exchange.pair_whitelist` and `exchange.pair_blacklis
By default, only currently enabled pairs are allowed.
To skip pair validation against active markets, set `"allow_inactive": true` within the `StaticPairList` configuration.
This can be useful for backtesting expired pairs (like quarterly spot-markets).
This option must be configured along with `exchange.skip_pair_validation` in the exchange configuration.
When used in a "follow-up" position (e.g. after VolumePairlist), all pairs in `'pair_whitelist'` will be added to the end of the pairlist.
@@ -67,7 +68,7 @@ When used in the leading position of the chain of Pairlist Handlers, the `pair_w
The `refresh_period` setting allows to define the period (in seconds), at which the pairlist will be refreshed. Defaults to 1800s (30 minutes).
The pairlist cache (`refresh_period`) on `VolumePairList` is only applicable to generating pairlists.
Filtering instances (not the first position in the list) will not apply any cache and will always use up-to-date data.
Filtering instances (not the first position in the list) will not apply any cache (beyond caching candles for the duration of the candle in advanced mode) and will always use up-to-date data.
`VolumePairList` is per default based on the ticker data from exchange, as reported by the ccxt library:
@@ -80,12 +81,14 @@ Filtering instances (not the first position in the list) will not apply any cach
"number_assets":20,
"sort_key":"quoteVolume",
"min_value":0,
"max_value":8000000,
"refresh_period":1800
}
],
```
You can define a minimum volume with `min_value` - which will filter out pairs with a volume lower than the specified value in the specified timerange.
In addition to that, you can also define a maximum volume with `max_value` - which will filter out pairs with a volume higher than the specified value in the specified timerange.
##### VolumePairList Advanced mode
@@ -112,8 +115,8 @@ For convenience `lookback_days` can be specified, which will imply that 1d candl
!!! Warning "Performance implications when using lookback range"
If used in first position in combination with lookback, the computation of the range based volume can be time and resource consuming, as it downloads candles for all tradable pairs. Hence it's highly advised to use the standard approach with `VolumeFilter` to narrow the pairlist down for further range volume calculation.
??? Tip "Unsupported exchanges (Bittrex, Gemini)"
On some exchanges (like Bittrex and Gemini), regular VolumePairList does not work as the api does not natively provide 24h volume. This can be worked around by using candle data to build the volume.
??? Tip "Unsupported exchanges"
On some exchanges (like Gemini), regular VolumePairList does not work as the api does not natively provide 24h volume. This can be worked around by using candle data to build the volume.
To roughly simulate 24h volume, you can use the following configuration.
Please note that These pairlists will only refresh once per day.
@@ -149,6 +152,89 @@ More sophisticated approach can be used, by using `lookback_timeframe` for candl
!!! Note
`VolumePairList` does not support backtesting mode.
#### Percent Change Pair List
`PercentChangePairList` filters and sorts pairs based on the percentage change in their price over the last 24 hours or any defined timeframe as part of advanced options. This allows traders to focus on assets that have experienced significant price movements, either positive or negative.
**Configuration Options**
* `number_assets`: Specifies the number of top pairs to select based on the 24-hour percentage change.
* `min_value`: Sets a minimum percentage change threshold. Pairs with a percentage change below this value will be filtered out.
* `max_value`: Sets a maximum percentage change threshold. Pairs with a percentage change above this value will be filtered out.
* `sort_direction`: Specifies the order in which pairs are sorted based on their percentage change. Accepts two values: `asc` for ascending order and `desc` for descending order.
* `refresh_period`: Defines the interval (in seconds) at which the pairlist will be refreshed. The default is 1800 seconds (30 minutes).
* `lookback_days`: Number of days to look back. When `lookback_days` is selected, the `lookback_timeframe` is defaulted to 1 day.
* `lookback_timeframe`: Timeframe to use for the lookback period.
* `lookback_period`: Number of periods to look back at.
When PercentChangePairList is used after other Pairlist Handlers, it will operate on the outputs of those handlers. If it is the leading Pairlist Handler, it will select pairs from all available markets with the specified stake currency.
`PercentChangePairList` uses ticker data from the exchange, provided via the ccxt library:
The percentage change is calculated as the change in price over the last 24 hours.
??? Note "Unsupported exchanges"
On some exchanges (like HTX), regular PercentChangePairList does not work as the api does not natively provide 24h percent change in price. This can be worked around by using candle data to calculate the percentage change. To roughly simulate 24h percent change, you can use the following configuration. Please note that these pairlists will only refresh once per day.
```json
"pairlists": [
{
"method": "PercentChangePairList",
"number_assets": 20,
"min_value": 0,
"refresh_period": 86400,
"lookback_days": 1
}
],
```
**Example Configuration to Read from Ticker**
```json
"pairlists": [
{
"method": "PercentChangePairList",
"number_assets": 15,
"min_value": -10,
"max_value": 50
}
],
```
In this configuration:
1. The top 15 pairs are selected based on the highest percentage change in price over the last 24 hours.
2. Only pairs with a percentage change between -10% and 50% are considered.
**Example Configuration to Read from Candles**
```json
"pairlists": [
{
"method": "PercentChangePairList",
"number_assets": 15,
"sort_key": "percentage",
"min_value": 0,
"refresh_period": 3600,
"lookback_timeframe": "1h",
"lookback_period": 72
}
],
```
This example builds the percent change pairs based on a rolling period of 3 days of 1-hour candles by using `lookback_timeframe` for candle size and `lookback_period` which specifies the number of candles.
The percent change in price is calculated using the following formula, which expresses the percentage difference between the current candle's close price and the previous candle's close price, as defined by the specified timeframe and lookback period:
When used in conjunction with `lookback_days` and `lookback_timeframe` the `refresh_period` can not be smaller than the candle size in seconds. As this will result in unnecessary requests to the exchanges API.
!!! Warning "Performance implications when using lookback range"
If used in first position in combination with lookback, the computation of the range-based percent change can be time and resource consuming, as it downloads candles for all tradable pairs. Hence it's highly advised to use the standard approach with `PercentChangePairList` to narrow the pairlist down for further percent-change calculation.
!!! Note "Backtesting"
`PercentChangePairList` does not support backtesting mode.
#### ProducerPairList
With `ProducerPairList`, you can reuse the pairlist from a [Producer](producer-consumer.md) without explicitly defining the pairlist on each consumer.
@@ -192,14 +278,15 @@ The RemotePairList is defined in the pairlists section of the configuration sett
"refresh_period": 1800,
"keep_pairlist_on_failure": true,
"read_timeout": 60,
"bearer_token": "my-bearer-token"
"bearer_token": "my-bearer-token",
"save_to_file": "user_data/filename.json"
}
]
```
The optional `mode` option specifies if the pairlist should be used as a `blacklist` or as a `whitelist`. The default value is "whitelist".
The optional `processing_mode` option in the RemotePairList configuration determines how the retrieved pairlist is processed. It can have two values: "filter" or "append".
The optional `processing_mode` option in the RemotePairList configuration determines how the retrieved pairlist is processed. It can have two values: "filter" or "append". The default value is "filter".
In "filter" mode, the retrieved pairlist is used as a filter. Only the pairs present in both the original pairlist and the retrieved pairlist are included in the final pairlist. Other pairs are filtered out.
@@ -207,6 +294,42 @@ In "append" mode, the retrieved pairlist is added to the original pairlist. All
The `pairlist_url` option specifies the URL of the remote server where the pairlist is located, or the path to a local file (if file:/// is prepended). This allows the user to use either a remote server or a local file as the source for the pairlist.
The `save_to_file` option, when provided with a valid filename, saves the processed pairlist to that file in JSON format. This option is optional, and by default, the pairlist is not saved to a file.
??? Example "Multi bot with shared pairlist example"
`save_to_file` can be used to save the pairlist to a file with Bot1:
```json
"pairlists": [
{
"method": "RemotePairList",
"mode": "whitelist",
"pairlist_url": "https://example.com/pairlist",
"number_assets": 10,
"refresh_period": 1800,
"keep_pairlist_on_failure": true,
"read_timeout": 60,
"save_to_file": "user_data/filename.json"
}
]
```
This saved pairlist file can be loaded by Bot2, or any additional bot with this configuration:
The user is responsible for providing a server or local file that returns a JSON object with the following structure:
```json
@@ -227,6 +350,33 @@ The optional `bearer_token` will be included in the requests Authorization Heade
!!! Note
In case of a server error the last received pairlist will be kept if `keep_pairlist_on_failure` is set to true, when set to false a empty pairlist is returned.
#### MarketCapPairList
`MarketCapPairList` employs sorting/filtering of pairs by their marketcap rank based of CoinGecko. The returned pairlist will be sorted based of their marketcap ranks.
```json
"pairlists": [
{
"method": "MarketCapPairList",
"number_assets": 20,
"max_rank": 50,
"refresh_period": 86400,
"categories": ["layer-1"]
}
]
```
`number_assets` defines the maximum number of pairs returned by the pairlist. `max_rank` will determine the maximum rank used in creating/filtering the pairlist. It's expected that some coins within the top `max_rank` marketcap will not be included in the resulting pairlist since not all pairs will have active trading pairs in your preferred market/stake/exchange combination.
While using a `max_rank` bigger than 250 is supported, it's not recommended, as it'll cause multiple API calls to CoinGecko, which can lead to rate limit issues.
The `refresh_period` setting defines the interval (in seconds) at which the marketcap rank data will be refreshed. The default is 86,400 seconds (1 day). The pairlist cache (`refresh_period`) applies to both generating pairlists (when in the first position in the list) and filtering instances (when not in the first position in the list).
The `categories` setting specifies the [coingecko categories](https://www.coingecko.com/en/categories) from which to select coins from. The default is an empty list `[]`, meaning no category filtering is applied.
If an incorrect category string is chosen, the plugin will print the available categories from CoinGecko and fail. The category should be the ID of the category, for example, for `https://www.coingecko.com/en/categories/layer-1`, the category ID would be `layer-1`. You can pass multiple categories such as `["layer-1", "meme-token"]` to select from several categories.
!!! Warning "Many categories"
Each added category corresponds to one API call to CoinGecko. The more categories you add, the longer the pairlist generation will take, potentially causing rate limit issues.
#### AgeFilter
Removes pairs that have been listed on the exchange for less than `min_days_listed` days (defaults to `10`) or more than `max_days_listed` days (defaults `None` mean infinity).
@@ -312,6 +462,11 @@ As this Filter uses past performance of the bot, it'll have some startup-period
Filters low-value coins which would not allow setting stoplosses.
Namely, pairs are blacklisted if a variance of one percent or more in the stop price would be caused by precision rounding on the exchange, i.e. `rounded(stop_price) <= rounded(stop_price * 0.99)`. The idea is to avoid coins with a value VERY close to their lower trading boundary, not allowing setting of proper stoploss.
!!! Tip "PrecisionFilter is pointless for futures trading"
The above does not apply to shorts. And for longs, in theory the trade will be liquidated first.
!!! Warning "Backtesting"
`PrecisionFilter` does not support backtesting mode using multiple strategies.
@@ -393,6 +548,8 @@ If the trading range over the last 10 days is <1% or >99%, remove the pair from
]
```
Adding `"sort_direction": "asc"` or `"sort_direction": "desc"` enables sorting for this pairlist.
!!! Tip
This Filter can be used to automatically remove stable coin pairs, which have a very low trading range, and are therefore extremely difficult to trade with profit.
Additionally, it can also be used to automatically remove pairs with extreme high/low variance over a given amount of time.
@@ -420,6 +577,8 @@ If the volatility over the last 10 days is not in the range of 0.05-0.50, remove
]
```
Adding `"sort_direction": "asc"` or `"sort_direction": "desc"` enables sorting mode for this pairlist.
### Full example of Pairlist Handlers
The below example blacklists `BNB/BTC`, uses `VolumePairList` with `20` assets, sorting pairs by `quoteVolume` and applies [`PrecisionFilter`](#precisionfilter) and [`PriceFilter`](#pricefilter), filtering all assets where 1 price unit is > 1%. Then the [`SpreadFilter`](#spreadfilter) and [`VolatilityFilter`](#volatilityfilter) is applied and pairs are finally shuffled with the random seed set to some predefined value.
This feature is still in it's testing phase. Should you notice something you think is wrong please let us know via Discord or via Github Issue.
Protections will protect your strategy from unexpected events and market conditions by temporarily stop trading for either one pair, or for all pairs.
All protection end times are rounded up to the next candle to avoid sudden, unexpected intra-candle buys.
!!! Note
!!! Tip "Usage tips"
Not all Protections will work for all strategies, and parameters will need to be tuned for your strategy to improve performance.
!!! Tip
Each Protection can be configured multiple times with different parameters, to allow different levels of protection (short-term / long-term).
!!! Note "Backtesting"
Protections are supported by backtesting and hyperopt, but must be explicitly enabled by using the `--enable-protections` flag.
!!! Warning "Setting protections from the configuration"
Setting protections from the configuration via `"protections": [],` key should be considered deprecated and will be removed in a future version.
It is also no longer guaranteed that your protections apply to the strategy in cases where the strategy defines [protections as property](hyperopt.md#optimizing-protections).
### Available Protections
* [`StoplossGuard`](#stoploss-guard) Stop trading if a certain amount of stoploss occurred within a certain time window.
@@ -36,6 +28,7 @@ All protection end times are rounded up to the next candle to avoid sudden, unex
| `lookback_period_candles` | Only trades that completed within the last `lookback_period_candles` candles will be considered. This setting may be ignored by some Protections. <br>**Datatype:** Positive integer (in candles).
| `lookback_period` | Only trades that completed after `current_time - lookback_period` will be considered. <br>Cannot be used together with `lookback_period_candles`. <br>This setting may be ignored by some Protections. <br>**Datatype:** Float (in minutes)
| `trade_limit` | Number of trades required at minimum (not used by all Protections). <br>**Datatype:** Positive integer
| `unlock_at` | Time when trading will be unlocked regularly (not used by all Protections). <br>**Datatype:** string <br>**Input Format:** "HH:MM" (24-hours)
!!! Note "Durations"
Durations (`stop_duration*` and `lookback_period*` can be defined in either minutes or candles).
@@ -44,7 +37,7 @@ All protection end times are rounded up to the next candle to avoid sudden, unex
#### Stoploss Guard
`StoplossGuard` selects all trades within `lookback_period` in minutes (or in candles when using `lookback_period_candles`).
If `trade_limit` or more trades resulted in stoploss, trading will stop for `stop_duration` in minutes (or in candles when using `stop_duration_candles`).
If `trade_limit` or more trades resulted in stoploss, trading will stop for `stop_duration` in minutes (or in candles when using `stop_duration_candles`, or until the set time when using `unlock_at`).
This applies across all pairs, unless `only_per_pair` is set to true, which will then only look at one pair at a time.
@@ -97,7 +90,7 @@ def protections(self):
#### Low Profit Pairs
`LowProfitPairs` uses all trades for a pair within `lookback_period` in minutes (or in candles when using `lookback_period_candles`) to determine the overall profit ratio.
If that ratio is below `required_profit`, that pair will be locked for `stop_duration` in minutes (or in candles when using `stop_duration_candles`).
If that ratio is below `required_profit`, that pair will be locked for `stop_duration` in minutes (or in candles when using `stop_duration_candles`, or until the set time when using `unlock_at`).
For futures bots, setting `only_per_side` will make the bot only consider one side, and will then only lock this one side, allowing for example shorts to continue after a series of long losses.
@@ -120,7 +113,7 @@ def protections(self):
#### Cooldown Period
`CooldownPeriod` locks a pair for `stop_duration` in minutes (or in candles when using `stop_duration_candles`) after selling, avoiding a re-entry for this pair for `stop_duration` minutes.
`CooldownPeriod` locks a pair for `stop_duration` in minutes (or in candles when using `stop_duration_candles`, or until the set time when using `unlock_at`) after exiting, avoiding a re-entry for this pair for `stop_duration` minutes.
The below example will stop trading a pair for 2 candles after closing a trade, allowing this pair to "cool down".
@@ -28,7 +28,7 @@ Freqtrade is a free and open source crypto trading bot written in Python. It is
- Develop your Strategy: Write your strategy in python, using [pandas](https://pandas.pydata.org/). Example strategies to inspire you are available in the [strategy repository](https://github.com/freqtrade/freqtrade-strategies).
- Download market data: Download historical data of the exchange and the markets your may want to trade with.
- Backtest: Test your strategy on downloaded historical data.
- Optimize: Find the best parameters for your strategy using hyperoptimization which employs machining learning methods. You can optimize buy, sell, take profit (ROI), stop-loss and trailing stop-loss parameters for your strategy.
- Optimize: Find the best parameters for your strategy using hyperoptimization which employs machine learning methods. You can optimize buy, sell, take profit (ROI), stop-loss and trailing stop-loss parameters for your strategy.
- Select markets: Create your static list or use an automatic one based on top traded volumes and/or prices (not available during backtesting). You can also explicitly blacklist markets you don't want to trade.
- Run: Test your strategy with simulated money (Dry-Run mode) or deploy it with real money (Live-Trade mode).
- Run using Edge (optional module): The concept is to find the best historical [trade expectancy](edge.md#expectancy) by markets based on variation of the stop-loss and then allow/reject markets to trade. The sizing of the trade is based on a risk of a percentage of your capital.
@@ -40,9 +40,12 @@ Freqtrade is a free and open source crypto trading bot written in Python. It is
Please read the [exchange specific notes](exchanges.md) to learn about eventual, special configurations needed for each exchange.
- [X] [Binance](https://www.binance.com/)
- [X] [Bittrex](https://bittrex.com/)
- [X] [BingX](https://bingx.com/invite/0EM9RX)
- [X] [Bitmart](https://bitmart.com/)
- [X] [Bybit](https://bybit.com/)
- [X] [Gate.io](https://www.gate.io/ref/6266643)
- [X] [Huobi](http://huobi.com/)
- [X] [HTX](https://www.htx.com/) (Former Huobi)
- [X] [Hyperliquid](https://hyperliquid.xyz/) (A decentralized exchange, or DEX)
- [X] [Kraken](https://kraken.com/)
- [X] [OKX](https://okx.com/) (Former OKEX)
- [ ] [potentially many others through <img alt="ccxt" width="30px" src="assets/ccxt-logo.svg" />](https://github.com/ccxt/ccxt/). _(We cannot guarantee they will work)_
@@ -50,9 +53,10 @@ Please read the [exchange specific notes](exchanges.md) to learn about eventual,
### Supported Futures Exchanges (experimental)
- [X] [Binance](https://www.binance.com/)
- [X] [Gate.io](https://www.gate.io/ref/6266643)
- [X] [OKX](https://okx.com/)
- [X] [Bybit](https://bybit.com/)
- [X] [Gate.io](https://www.gate.io/ref/6266643)
- [X] [Hyperliquid](https://hyperliquid.xyz/) (A decentralized exchange, or DEX)
- [X] [OKX](https://okx.com/)
Please make sure to read the [exchange specific notes](exchanges.md), as well as the [trading with leverage](leverage.md) documentation before diving in.
@@ -83,7 +87,7 @@ To run this bot we recommend you a linux cloud instance with a minimum of:
@@ -24,7 +24,7 @@ The easiest way to install and run Freqtrade is to clone the bot Github reposito
The `stable` branch contains the code of the last release (done usually once per month on an approximately one week old snapshot of the `develop` branch to prevent packaging bugs, so potentially it's more stable).
!!! Note
Python3.9 or higher and the corresponding `pip` are assumed to be available. The install-script will warn you and stop if that's not the case. `git` is also needed to clone the Freqtrade repository.
Python3.10 or higher and the corresponding `pip` are assumed to be available. The install-script will warn you and stop if that's not the case. `git` is also needed to clone the Freqtrade repository.
Also, python headers (`python<yourversion>-dev` / `python<yourversion>-devel`) must be available for the installation to complete successfully.
!!! Warning "Up-to-date clock"
@@ -42,7 +42,7 @@ These requirements apply to both [Script Installation](#script-installation) and
You will run freqtrade in separated `virtual environment`
@@ -232,19 +242,18 @@ python3 -m venv .venv
source .venv/bin/activate
```
#### Install python dependencies
### Install python dependencies
```bash
python3 -m pip install --upgrade pip
python3 -m pip install -r requirements.txt
# install freqtrade
python3 -m pip install -e .
```
### Congratulations
[You are now ready](#you-are-ready) to run the bot.
[You are ready](#you-are-ready), and run the bot
#### (Optional) Post-installation Tasks
### (Optional) Post-installation Tasks
!!! Note
If you run the bot on a server, you should consider using [Docker](docker_quickstart.md) or a terminal multiplexer like `screen` or [`tmux`](https://en.wikipedia.org/wiki/Tmux) to avoid that the bot is stopped on logout.
@@ -286,7 +295,7 @@ cd freqtrade
#### Freqtrade install: Conda Environment
```bash
conda create --name freqtrade python=3.11
conda create --name freqtrade python=3.12
```
!!! Note "Creating Conda Environment"
@@ -333,9 +342,7 @@ cd build_helpers
bash install_ta-lib.sh ${CONDA_PREFIX} nosudo
```
### Congratulations
[You are ready](#you-are-ready), and run the bot
[You are now ready](#you-are-ready) to run the bot.
@@ -17,7 +17,7 @@ If you already have an existing strategy, please read the [strategy migration gu
## Shorting
Shorting is not possible when trading with [`trading_mode`](#understand-tradingmode) set to `spot`. To short trade, `trading_mode` must be set to `margin`(currently unavailable) or [`futures`](#futures), with [`margin_mode`](#margin-mode) set to `cross`(currently unavailable) or [`isolated`](#isolated-margin-mode)
Shorting is not possible when trading with [`trading_mode`](#leverage-trading-modes) set to `spot`. To short trade, `trading_mode` must be set to `margin`(currently unavailable) or [`futures`](#futures), with [`margin_mode`](#margin-mode) set to `cross`(currently unavailable) or [`isolated`](#isolated-margin-mode)
For a strategy to short, the strategy class must set the class variable `can_short = True`
@@ -101,3 +101,4 @@ This could lead to a false-negative (the strategy will then be reported as non-b
-`lookahead-analysis` has access to everything that backtesting has too.
Please don't provoke any configs like enabling position stacking.
If you decide to do so, then make doubly sure that you won't ever run out of `max_open_trades` amount and neither leftover money in your wallet.
- In the results table, the `biased_indicators` column will falsely flag FreqAI target indicators defined in `set_freqai_targets()` as biased. These are not biased and can safely be ignored.
This page explains how to plot prices, indicators and profits.
!!! Warning "Deprecated"
The commands described in this page (`plot-dataframe`, `plot-profit`) should be considered deprecated and are in maintenance mode.
This is mostly for the performance problems even medium sized plots can cause, but also because "store a file and open it in a browser" isn't very intuitive from a UI perspective.
While there are no immediate plans to remove them, they are not actively maintained - and may be removed short-term should major changes be required to keep them working.
Please use [FreqUI](freq-ui.md) for plotting needs, which doesn't struggle with the same performance problems.
## Installation / Setup
Plotting modules use the Plotly library. You can install / upgrade this by running the following command:
Freqtrade provides a builtin webserver, which can serve [FreqUI](https://github.com/freqtrade/frequi), the freqtrade UI.
By default, the UI is not included in the installation (except for docker images), and must be installed explicitly with `freqtrade install-ui`.
This same command can also be used to update freqUI, should there be a new release.
Once the bot is started in trade / dry-run mode (with `freqtrade trade`) - the UI will be available under the configured port below (usually `http://127.0.0.1:8080`).
!!! Note "developers"
Developers should not use this method, but instead use the method described in the [freqUI repository](https://github.com/freqtrade/frequi) to get the source-code of freqUI.
FreqUI now has it's own dedicated [documentation section](freq-ui.md) - please refer to that section for all information regarding the FreqUI.
## Configuration
@@ -89,17 +81,20 @@ Make sure that the following 2 lines are available in your docker-compose file:
```
!!! Danger "Security warning"
By using `8080:8080` in the docker port mapping, the API will be available to everyone connecting to the server under the correct port, so others may be able to control your bot.
By using `"8080:8080"` (or `"0.0.0.0:8080:8080"`) in the docker port mapping, the API will be available to everyone connecting to the server under the correct port, so others may be able to control your bot.
This **may** be safe if you're running the bot in a secure environment (like your home network), but it's not recommended to expose the API to the internet.
## Rest API
### Consuming the API
You can consume the API by using the script `scripts/rest_client.py`.
The client script only requires the `requests` module, so Freqtrade does not need to be installed on the system.
You can consume the API by using `freqtrade-client` (also available as `scripts/rest_client.py`).
This command can be installed independent of the bot by using `pip install freqtrade-client`.
This module is designed to be lightweight, and only depends on the `requests` and `python-rapidjson` modules, skipping all heavy dependencies freqtrade otherwise needs.
By default, the script assumes `127.0.0.1` (localhost) and port `8080` to be used, however you can specify a configuration file to override this behaviour.
@@ -120,9 +115,35 @@ By default, the script assumes `127.0.0.1` (localhost) and port `8080` to be use
| `reload_config` | Reloads the configuration file.
| `trades` | List last trades. Limited to 500 trades per call.
| `trade/<tradeid>` | Get specific trade.
| `trade/<tradeid>` | DELETE - Remove trade from the database. Tries to close open orders. Requires manual handling of this trade on the exchange.
| `trade/<tradeid>/open-order` | DELETE - Cancel open order for this trade.
| `trade/<tradeid>/reload` | GET - Reload a trade from the Exchange. Only works in live, and can potentially help recover a trade that was manually sold on the exchange.
| `trades/<tradeid>` | DELETE - Remove trade from the database. Tries to close open orders. Requires manual handling of this trade on the exchange.
| `trades/<tradeid>/open-order` | DELETE - Cancel open order for this trade.
| `trades/<tradeid>/reload` | GET - Reload a trade from the Exchange. Only works in live, and can potentially help recover a trade that was manually sold on the exchange.
| `show_config` | Shows part of the current configuration with relevant settings to operation.
| `logs` | Shows last log messages.
| `status` | Lists all open trades.
| `count` | Displays number of trades used and available.
| `entries [pair]` | Shows profit statistics for each enter tags for given pair (or all pairs if pair isn't given). Pair is optional.
| `exits [pair]` | Shows profit statistics for each exit reasons for given pair (or all pairs if pair isn't given). Pair is optional.
| `mix_tags [pair]` | Shows profit statistics for each combinations of enter tag + exit reasons for given pair (or all pairs if pair isn't given). Pair is optional.
| `locks` | Displays currently locked pairs.
| `delete_lock <lock_id>` | Deletes (disables) the lock by id.
| `locks add <pair>, <until>, [side], [reason]` | Locks a pair until "until". (Until will be rounded up to the nearest timeframe).
| `profit` | Display a summary of your profit/loss from close trades and some stats about your performance.
| `forceexit <trade_id>` | Instantly exits the given trade (Ignoring `minimum_roi`).
| `forceexit <trade_id> [order_type] [amount]` | Instantly exits the given trade (ignoring `minimum_roi`), using the given order type ("market" or "limit", uses your config setting if not specified), and the chosen amount (full sell if not specified).
| `forceexit all` | Instantly exits all open trades (Ignoring `minimum_roi`).
| `forceenter <pair> [rate]` | Instantly enters the given pair. Rate is optional. (`force_entry_enable` must be set to True)
| `forceenter <pair><side> [rate]` | Instantly longs or shorts the given pair. Rate is optional. (`force_entry_enable` must be set to True)
Possible commands can be listed from the rest-client script using the `help` command.
``` bash
python3 scripts/rest_client.py help
freqtrade-client help
```
``` output
@@ -430,7 +455,7 @@ To properly configure your reverse proxy (securely), please consult it's documen
- **Caddy**: Caddy v2 supports websockets out of the box, see the [documentation](https://caddyserver.com/docs/v2-upgrade#proxy)
!!! Tip "SSL certificates"
You can use tools like certbot to setup ssl certificates to access your bot's UI through encrypted connection by using any fo the above reverse proxies.
You can use tools like certbot to setup ssl certificates to access your bot's UI through encrypted connection by using any of the above reverse proxies.
While this will protect your data in transit, we do not recommend to run the freqtrade API outside of your private network (VPN, SSH tunnel).
### OpenAPI interface
@@ -463,42 +488,4 @@ Since the access token has a short timeout (15 min) - the `token/refresh` reques
This whole section is only necessary in cross-origin cases (where you multiple bot API's running on `localhost:8081`, `localhost:8082`, ...), and want to combine them into one FreqUI instance.
??? info "Technical explanation"
All web-based front-ends are subject to [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) - Cross-Origin Resource Sharing.
Since most of the requests to the Freqtrade API must be authenticated, a proper CORS policy is key to avoid security problems.
Also, the standard disallows `*` CORS policies for requests with credentials, so this setting must be set appropriately.
Users can allow access from different origin URL's to the bot API via the `CORS_origins` configuration setting.
It consists of a list of allowed URL's that are allowed to consume resources from the bot's API.
Assuming your application is deployed as `https://frequi.freqtrade.io/home/` - this would mean that the following configuration becomes necessary:
```jsonc
{
//...
"jwt_secret_key": "somethingrandom",
"CORS_origins": ["https://frequi.freqtrade.io"],
//...
}
```
In the following (pretty common) case, FreqUI is accessible on `http://localhost:8080/trade` (this is what you see in your navbar when navigating to freqUI).
The correct configuration for this case is `http://localhost:8080` - the main part of the URL including the port.
```jsonc
{
//...
"jwt_secret_key": "somethingrandom",
"CORS_origins": ["http://localhost:8080"],
//...
}
```
!!! Note
We strongly recommend to also set `jwt_secret_key` to something random and known only to yourself to avoid unauthorized access to your bot.
This page contains some help if you want to edit your sqlite db.
This page contains some help if you want to query your sqlite db.
!!! Tip "Other Database systems"
To use other Database Systems like PostgreSQL or MariaDB, you can use the same queries, but you need to use the respective client for the database system. [Click here](advanced-setup.md#use-a-different-database-system) to learn how to setup a different database system with freqtrade.
!!! Warning
If you are not familiar with SQL, you should be very careful when running queries on your database.
Always make sure to have a backup of your database before running any queries.
## Install sqlite3
@@ -43,13 +50,25 @@ sqlite3
.schema <table_name>
```
## Get all trades in the table
### Get all trades in the table
```sql
SELECT * FROM trades;
```
## Fix trade still open after a manual exit on the exchange
## Destructive queries
Queries that write to the database.
These queries should usually not be necessary as freqtrade tries to handle all database operations itself - or exposes them via API or telegram commands.
!!! Warning
Please make sure you have a backup of your database before running any of the below queries.
!!! Danger
You should also **never** run any writing query (`update`, `insert`, `delete`) while a bot is connected to the database.
This can and will lead to data corruption - most likely, without the possibility of recovery.
### Fix trade still open after a manual exit on the exchange
!!! Warning
Manually selling a pair on the exchange will not be detected by the bot and it will try to sell anyway. Whenever possible, /forceexit <tradeid> should be used to accomplish the same thing.
@@ -69,7 +88,7 @@ SET is_open=0,
WHERE id=<trade_ID_to_update>;
```
### Example
#### Example
```sql
UPDATE trades
@@ -82,7 +101,7 @@ SET is_open=0,
WHERE id=31;
```
## Remove trade from the database
### Remove trade from the database
!!! Tip "Use RPC Methods to delete trades"
Consider using `/delete <tradeid>` via telegram or rest API. That's the recommended way to deleting trades.
@@ -100,39 +119,3 @@ DELETE FROM trades WHERE id = 31;
!!! Warning
This will remove this trade from the database. Please make sure you got the correct id and **NEVER** run this query without the `where` clause.
## Use a different database system
Freqtrade is using SQLAlchemy, which supports multiple different database systems. As such, a multitude of database systems should be supported.
Freqtrade does not depend or install any additional database driver. Please refer to the [SQLAlchemy docs](https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls) on installation instructions for the respective database systems.
The following systems have been tested and are known to work with freqtrade:
* sqlite (default)
* PostgreSQL)
* MariaDB
!!! Warning
By using one of the below database systems, you acknowledge that you know how to manage such a system. The freqtrade team will not provide any support with setup or maintenance (or backups) of the below database systems.
Freqtrade will automatically create the tables necessary upon startup.
If you're running different instances of Freqtrade, you must either setup one database per Instance or use different users / schemas for your connections.
### MariaDB / MySQL
Freqtrade supports MariaDB by using SQLAlchemy, which supports multiple different database systems.
@@ -30,11 +30,13 @@ The Order-type will be ignored if only one mode is available.
|----------|-------------|
| Binance | limit |
| Binance Futures | market, limit |
| Huobi | limit |
| Bingx | market, limit |
| HTX (former Huobi) | limit |
| kraken | market, limit |
| Gate | limit |
| Okx | limit |
| Kucoin | stop-limit, stop-market|
| Hyperliquid (futures only) | limit |
!!! Note "Tight stoploss"
<ins>Do not set too low/tight stoploss value when using stop loss on exchange!</ins>
@@ -158,7 +160,7 @@ You could also have a default stop loss when you are in the red with your buy (b
For example, your default stop loss is -10%, but once you have more than 0% profit (example 0.1%) a different trailing stoploss will be used.
!!! Note
If you want the stoploss to only be changed when you break even of making a profit (what most users want) please refer to next section with [offset enabled](#Trailing-stop-loss-only-once-the-trade-has-reached-a-certain-offset).
If you want the stoploss to only be changed when you break even of making a profit (what most users want) please refer to next section with [offset enabled](#trailing-stop-loss-only-once-the-trade-has-reached-a-certain-offset).
Both values require `trailing_stop` to be set to true and `trailing_stop_positive` with a value.
@@ -240,7 +242,7 @@ When using leverage, the same principle is applied - with stoploss defining the
Therefore, a stoploss of 10% on a 10x trade would trigger on a 1% price move.
If your stake amount (own capital) was 100$ - this trade would be 1000$ at 10x (after leverage).
If price moves 1% - you've lost 10$ of your own capital - therfore stoploss will trigger in this case.
If price moves 1% - you've lost 10$ of your own capital - therefore stoploss will trigger in this case.
Make sure to be aware of this, and avoid using too tight stoploss (at 10x leverage, 10% risk may be too little to allow the trade to "breath" a little).
# Freqtrade Strategies 101: A Quick Start for Strategy Development
For the purposes of this quick start, we are assuming you are familiar with the basics of trading, and have read the
[Freqtrade basics](bot-basics.md) page.
## Required Knowledge
A strategy in Freqtrade is a Python class that defines the logic for buying and selling cryptocurrency `assets`.
Assets are defined as `pairs`, which represent the `coin` and the `stake`. The coin is the asset you are trading using another currency as the stake.
Data is supplied by the exchange in the form of `candles`, which are made up of a six values: `date`, `open`, `high`, `low`, `close` and `volume`.
`Technical analysis` functions analyse the candle data using various computational and statistical formulae, and produce secondary values called `indicators`.
Indicators are analysed on the asset pair candles to generate `signals`.
Signals are turned into `orders` on a cryptocurrency `exchange`, i.e. `trades`.
We use the terms `entry` and `exit` instead of `buying` and `selling` because Freqtrade supports both `long` and `short` trades.
- **long**: You buy the coin based on a stake, e.g. buying the coin BTC using USDT as your stake, and you make a profit by selling the coin at a higher rate than you paid for. In long trades, profits are made by the coin value going up versus the stake.
- **short**: You borrow capital from the exchange in the form of the coin, and you pay back the stake value of the coin later. In short trades profits are made by the coin value going down versus the stake (you pay the loan off at a lower rate).
Whilst Freqtrade supports spot and futures markets for certain exchanges, for simplicity we will focus on spot (long) trades only.
## Structure of a Basic Strategy
### Main dataframe
Freqtrade strategies use a tabular data structure with rows and columns known as a `dataframe` to generate signals to enter and exit trades.
Each pair in your configured pairlist has its own dataframe. Dataframes are indexed by the `date` column, e.g. `2024-06-31 12:00`.
The next 5 columns represent the `open`, `high`, `low`, `close` and `volume` (OHLCV) data.
### Populate indicator values
The `populate_indicators` function adds columns to the dataframe that represent the technical analysis indicator values.
Examples of common indicators include Relative Strength Index, Bollinger Bands, Money Flow Index, Moving Average, and Average True Range.
Columns are added to the dataframe by calling technical analysis functions, e.g. ta-lib's RSI function `ta.RSI()`, and assigning them to a column name, e.g. `rsi`
```python
dataframe['rsi']=ta.RSI(dataframe)
```
??? Hint "Technical Analysis libraries"
Different libraries work in different ways to generate indicator values. Please check the documentation of each library to understand
how to integrate it into your strategy. You can also check the [Freqtrade example strategies](https://github.com/freqtrade/freqtrade-strategies) to give you ideas.
### Populate entry signals
The `populate_entry_trend` function defines conditions for an entry signal.
The dataframe column `enter_long` is added to the dataframe, and when a value of `1` is in this column, Freqtrade sees an entry signal.
??? Hint "Shorting"
To enter short trades, use the `enter_short` column.
### Populate exit signals
The `populate_exit_trend` function defines conditions for an exit signal.
The dataframe column `exit_long` is added to the dataframe, and when a value of `1` is in this column, Freqtrade sees an exit signal.
??? Hint "Shorting"
To exit short trades, use the `exit_short` column.
## A simple strategy
Here is a minimal example of a Freqtrade strategy:
```python
fromfreqtrade.strategyimportIStrategy
frompandasimportDataFrame
importtalib.abstractasta
classMyStrategy(IStrategy):
# set the initial stoploss to -10%
stoploss=-0.10
# exit profitable positions at any time when the profit is greater than 1%
When a signal is found (a `1` in an entry or exit column), Freqtrade will attempt to make an order, i.e. a `trade` or `position`.
Each new trade position takes up a `slot`. Slots represent the maximum number of concurrent new trades that can be opened.
The number of slots is defined by the `max_open_trades` [configuration](configuration.md) option.
However, there can be a range of scenarios where generating a signal does not always create a trade order. These include:
- not enough remaining stake to buy an asset, or funds in your wallet to sell an asset (including any fees)
- not enough remaining free slots for a new trade to be opened (the number of positions you have open equals the `max_open_trades` option)
- there is already an open trade for a pair (Freqtrade cannot stack positions - however it can [adjust existing positions](strategy-callbacks.md#adjust-trade-position))
- if an entry and exit signal is present on the same candle, they are considered as [colliding](strategy-customization.md#colliding-signals), and no order will be raised
- the strategy actively rejects the trade order due to logic you specify by using one of the relevant [entry](strategy-callbacks.md#trade-entry-buy-order-confirmation) or [exit](strategy-callbacks.md#trade-exit-sell-order-confirmation) callbacks
Read through the [strategy customization](strategy-customization.md) documentation for more details.
## Backtesting and forward testing
Strategy development can be a long and frustrating process, as turning our human "gut instincts" into a working computer-controlled
("algo") strategy is not always straightforward.
Therefore a strategy should be tested to verify that it is going to work as intended.
Freqtrade has two testing modes:
- **backtesting**: using historical data that you [download from an exchange](data-download.md), backtesting is a quick way to assess performance of a strategy. However, it can be very easy to distort results so a strategy will look a lot more profitable than it really is. Check the [backtesting documentation](backtesting.md) for more information.
- **dry run**: often referred to as _forward testing_, dry runs use real time data from the exchange. However, any signals that would result in trades are tracked as normal by Freqtrade, but do not have any trades opened on the exchange itself. Forward testing runs in real time, so whilst it takes longer to get results it is a much more reliable indicator of **potential** performance than backtesting.
Dry runs are enabled by setting `dry_run` to true in your [configuration](configuration.md#using-dry-run-mode).
!!! Warning "Backtests can be very inaccurate"
There are many reasons why backtest results may not match reality. Please check the [backtesting assumptions](backtesting.md#assumptions-made-by-backtesting) and [common strategy mistakes](strategy-customization.md#common-mistakes-when-developing-strategies) documentation.
Some websites that list and rank Freqtrade strategies show impressive backtest results. Do not assume these results are achieveable or realistic.
??? Hint "Useful commands"
Freqtrade includes two useful commands to check for basic flaws in strategies: [lookahead-analysis](lookahead-analysis.md) and [recursive-analysis](recursive-analysis.md).
### Assessing backtesting and dry run results
Always dry run your strategy after backtesting it to see if backtesting and dry run results are sufficiently similar.
If there is any significant difference, verify that your entry and exit signals are consistent and appear on the same candles between the two modes. However, there will always be differences between dry runs and backtests:
- Backtesting assumes all orders fill. In dry runs this might not be the case if using limit orders or there is no volume on the exchange.
- Following an entry signal on candle close, backtesting assumes trades enter at the next candle's open price (unless you have custom pricing callbacks in your strategy). In dry runs, there is often a delay between signals and trades opening.
This is because when new candles come in on your main timeframe, e.g. every 5 minutes, it takes time for Freqtrade to analyse all pair dataframes. Therefore, Freqtrade will attempt to open trades a few seconds (ideally a small a delay as possible)
after candle open.
- As entry rates in dry runs might not match backtesting, this means profit calculations will also differ. Therefore, it is normal if ROI, stoploss, trailing stoploss and callback exits are not identical.
- The more computational "lag" you have between new candles coming in and your signals being raised and trades being opened will result in greater price unpredictability. Make sure your computer is powerful enough to process the data for the number
of pairs you have in your pairlist within a reasonable time. Freqtrade will warn you in the logs if there are significant data processing delays.
## Controlling or monitoring a running bot
Once your bot is running in dry or live mode, Freqtrade has five mechanisms to control or monitor a running bot:
- **[FreqUI](freq-ui.md)**: The easiest to get started with, FreqUI is a web interface to see and control current activity of your bot.
- **[Telegram](telegram-usage.md)**: On mobile devices, Telegram integration is available to get alerts about your bot activity and to control certain aspects.
- **[FTUI](https://github.com/freqtrade/ftui)**: FTUI is a terminal (command line) interface to Freqtrade, and allows monitoring of a running bot only.
- **[REST API](rest-api.md)**: The REST API allows programmers to develop their own tools to interact with a Freqtrade bot.
- **[Webhooks](webhook-config.md)**: Freqtrade can send information to other services, e.g. discord, by webhooks.
### Logs
Freqtrade generates extensive debugging logs to help you understand what's happening. Please familiarise yourself with the information and error messages you might see in your bot logs.
## Final Thoughts
Algo trading is difficult, and most public strategies are not good performers due to the time and effort to make a strategy work profitably in multiple scenarios.
Therefore, taking public strategies and using backtests as a way to assess performance is often problematic. However, Freqtrade provides useful ways to help you make decisions and do your due diligence.
There are many different ways to achieve profitability, and there is no one single tip, trick or config option that will fix a poorly performing strategy.
Freqtrade is an open source platform with a large and helpful community - make sure to visit our [discord channel](https://discord.gg/p7nuUNVfP7) to discuss your strategy with others!
As always, only invest what you are willing to lose.
## Conclusion
Developing a strategy in Freqtrade involves defining entry and exit signals based on technical indicators. By following the structure and methods outlined above, you can create and test your own trading strategies.
Common questions and answers are available on our [FAQ](faq.md).
To continue, refer to the more in-depth [Freqtrade strategy customization documentation](strategy-customization.md).
@@ -11,8 +11,103 @@ The call sequence of the methods described here is covered under [bot execution
!!! Tip
Start off with a strategy template containing all available callback methods by running `freqtrade new-strategy --strategy MyAwesomeStrategy --template advanced`
## Storing information
## Storing information (Persistent)
Freqtrade allows storing/retrieving user custom information associated with a specific trade in the database.
Using a trade object, information can be stored using `trade.set_custom_data(key='my_key', value=my_value)` and retrieved using `trade.get_custom_data(key='my_key')`. Each data entry is associated with a trade and a user supplied key (of type `string`). This means that this can only be used in callbacks that also provide a trade object.
For the data to be able to be stored within the database, freqtrade must serialized the data. This is done by converting the data to a JSON formatted string.
Freqtrade will attempt to reverse this action on retrieval, so from a strategy perspective, this should not be relevant.
The above is a simple example - there are simpler ways to retrieve trade data like entry-adjustments.
!!! Note
It is recommended that simple data types are used `[bool, int, float, str]` to ensure no issues when serializing the data that needs to be stored.
Storing big junks of data may lead to unintended side-effects, like a database becoming big (and as a consequence, also slow).
!!! Warning "Non-serializable data"
If supplied data cannot be serialized a warning is logged and the entry for the specified `key` will contain `None` as data.
??? Note "All attributes"
custom-data has the following accessors through the Trade object (assumed as `trade` below):
*`trade.get_custom_data(key='something', default=0)` - Returns the actual value given in the type provided.
*`trade.get_custom_data_entry(key='something')` - Returns the entry - including metadata. The value is accessible via `.value` property.
*`trade.set_custom_data(key='something', value={'some': 'value'})` - set or update the corresponding key for this trade. Value must be serializable - and we recommend to keep the stored data relatively small.
"value" can be any type (both in setting and receiving) - but must be json serializable.
## Storing information (Non-Persistent)
!!! Warning "Deprecated"
This method of storing information is deprecated and we do advise against using non-persistent storage.
Please use [Persistent Storage](#storing-information-persistent) instead.
It's content has therefore been collapsed.
??? Abstract "Storing information"
Storing information can be accomplished by creating a new dictionary within the strategy class.
The name of the variable can be chosen at will, but should be prefixed with `custom_` to avoid naming collisions with predefined strategy variables.
@@ -231,4 +326,4 @@ for val in self.buy_ema_short.range:
dataframe = pd.concat(frames, axis=1)
```
Freqtrade does however also counter this by running `dataframe.copy()` on the dataframe right after the `populate_indicators()` method - so performance implications of this should be low to non-existant.
Freqtrade does however also counter this by running `dataframe.copy()` on the dataframe right after the `populate_indicators()` method - so performance implications of this should be low to non-existent.
# Between 2% and 10%, sell if EMA-long above EMA-short
if 0.02 < current_profit < 0.1:
if last_candle['emalong'] > last_candle['emashort']:
return 'ema_long_below_80'
if last_candle["emalong"] > last_candle["emashort"]:
return "ema_long_below_80"
# Sell any positions at a loss if they are held for more than one day.
if current_profit < 0.0 and (current_time - trade.open_date_utc).days >= 1:
return 'unclog'
return "unclog"
```
See [Dataframe access](strategy-advanced.md#dataframe-access) for more information about dataframe use in strategy callbacks.
@@ -158,15 +166,17 @@ Called for open trade every iteration (roughly every 5 seconds) until a trade is
The usage of the custom stoploss method must be enabled by setting `use_custom_stoploss=True` on the strategy object.
The stoploss price can only ever move upwards - if the stoploss value returned from `custom_stoploss` would result in a lower stoploss price than was previously set, it will be ignored. The traditional `stoploss` value serves as an absolute lower level and will be instated as the initial stoploss (before this method is called for the first time for a trade), and is still mandatory.
As custom stoploss acts as regular, changing stoploss, it will behave similar to `trailing_stop` - and trades exiting due to this will have the exit_reason of `"trailing_stop_loss"`.
The method must return a stoploss value (float / number) as a percentage of the current price.
E.g. If the `current_rate` is 200 USD, then returning `0.02` will set the stoploss price 2% lower, at 196 USD.
During backtesting, `current_rate` (and `current_profit`) are provided against the candle's high (or low for short trades) - while the resulting stoploss is evaluated against the candle's low (or high for short trades).
The absolute value of the return value is used (the sign is ignored), so returning `0.05` or `-0.05` have the same result, a stoploss 5% below the current price.
Returning None will be interpreted as "no desire to change", and is the only safe way to return when you'd like to not modify the stoploss.
Returning `None` will be interpreted as "no desire to change", and is the only safe way to return when you'd like to not modify the stoploss.
`NaN` and `inf` values are considered invalid and will be ignored (identical to `None`).
Stoploss on exchange works similar to `trailing_stop`, and the stoploss on exchange is updated as configured in `stoploss_on_exchange_interval` ([More details about stoploss on exchange](stoploss.md#stop-loss-on-exchange-freqtrade)).
Stoploss on exchange works similar to `trailing_stop`, and the stoploss on exchange is updated as configured in `stoploss_on_exchange_interval` ([More details about stoploss on exchange](stoploss.md#stop-loss-on-exchangefreqtrade)).
!!! Note "Use of dates"
All time-based calculations should be done based on `current_time` - using `datetime.now()` or `datetime.utcnow()` is discouraged, as this will break backtesting support.
@@ -193,9 +203,7 @@ Of course, many more things are possible, and all examples can be combined at wi
To simulate a regular trailing stoploss of 4% (trailing 4% behind the maximum reached price) you would use the following very simple method:
``` python
# additional imports required
from datetime import datetime
from freqtrade.persistence import Trade
# Default imports
class AwesomeStrategy(IStrategy):
@@ -203,9 +211,9 @@ class AwesomeStrategy(IStrategy):
Custom stoploss logic, returning the new distance relative to current_rate (as ratio).
e.g. returning -0.05 would create a stoploss 5% below current_rate.
@@ -233,8 +241,7 @@ class AwesomeStrategy(IStrategy):
Use the initial stoploss for the first 60 minutes, after this change to 10% trailing stoploss, and after 2 hours (120 minutes) we use a 5% trailing stoploss.
``` python
from datetime import datetime, timedelta
from freqtrade.persistence import Trade
# Default imports
class AwesomeStrategy(IStrategy):
@@ -242,9 +249,9 @@ class AwesomeStrategy(IStrategy):
# After an additional order, start with a stoploss of 10% below the new open rate
@@ -290,8 +296,7 @@ Use a different stoploss depending on the pair.
In this example, we'll trail the highest price with 10% trailing stoploss for `ETH/BTC` and `XRP/BTC`, with 5% trailing stoploss for `LTC/BTC` and with 15% for all other pairs.
``` python
from datetime import datetime
from freqtrade.persistence import Trade
# Default imports
class AwesomeStrategy(IStrategy):
@@ -299,13 +304,13 @@ class AwesomeStrategy(IStrategy):
# once the profit has risen above 10%, keep the stoploss at 7% above the open price
if current_profit > 0.10:
@@ -449,7 +450,7 @@ Stoploss values returned from `custom_stoploss()` must specify a percentage rela
```
Full examples can be found in the [Custom stoploss](strategy-advanced.md#custom-stoploss) section of the Documentation.
Full examples can be found in the [Custom stoploss](strategy-callbacks.md#custom-stoploss) section of the Documentation.
!!! Note
Providing invalid input to `stoploss_from_open()` may produce "CustomStoploss function did not return valid stoploss" warnings.
@@ -466,37 +467,34 @@ The helper function `stoploss_from_absolute()` can be used to convert from an ab
??? Example "Returning a stoploss using absolute price from the custom stoploss function"
If we want to trail a stop price at 2xATR below current price we can call `stoploss_from_absolute(current_rate+(side* candle['atr'] *2),current_rate,is_short=trade.is_short,leverage=trade.leverage)`.
If we want to trail a stop price at 2xATR below current price we can call `stoploss_from_absolute(current_rate+(side* candle["atr"] *2),current_rate=current_rate,is_short=trade.is_short,leverage=trade.leverage)`.
For futures, we need to adjust the direction (up or down), as well as adjust for leverage, since the [`custom_stoploss`](strategy-callbacks.md#custom-stoploss) callback returns the ["risk for this trade"](stoploss.md#stoploss-and-leverage) - not the relative price movement.
``` python
from datetime import datetime
from freqtrade.persistence import Trade
from freqtrade.strategy import IStrategy, stoploss_from_absolute, timeframe_to_prev_date
@@ -685,7 +682,7 @@ class AwesomeStrategy(IStrategy):
:param time_in_force: Time in force. Defaults to GTC (Good-til-cancelled).
:param current_time: datetime object, containing the current datetime
:param entry_tag: Optional entry_tag (buy_tag) if provided with the buy signal.
:param side: 'long' or 'short' - indicating the direction of the proposed trade
:param side: "long" or "short" - indicating the direction of the proposed trade
:param **kwargs: Ensure to keep this here so updates to this won't break your strategy.
:return bool: When True is returned, then the buy-order is placed on the exchange.
False aborts the process
@@ -707,8 +704,7 @@ The exit-reasons (if applicable) will be in the following sequence:
* `trailing_stop_loss`
``` python
from freqtrade.persistence import Trade
# Default imports
class AwesomeStrategy(IStrategy):
@@ -734,14 +730,14 @@ class AwesomeStrategy(IStrategy):
or current rate for market orders.
:param time_in_force: Time in force. Defaults to GTC (Good-til-cancelled).
:param exit_reason: Exit reason.
Can be any of ['roi', 'stop_loss', 'stoploss_on_exchange', 'trailing_stop_loss',
'exit_signal', 'force_exit', 'emergency_exit']
Can be any of ["roi", "stop_loss", "stoploss_on_exchange", "trailing_stop_loss",
"exit_signal", "force_exit", "emergency_exit"]
:param current_time: datetime object, containing the current datetime
:param **kwargs: Ensure to keep this here so updates to this won't break your strategy.
:return bool: When True, then the exit-order is placed on the exchange.
False aborts the process
"""
if exit_reason == 'force_exit' and trade.calc_profit_ratio(rate) < 0:
if exit_reason == "force_exit" and trade.calc_profit_ratio(rate) < 0:
# Reject force-sells with negative profit
# This is just a sample, please adjust to your needs
# (this does not necessarily make sense, assuming you know when you're force-selling)
@@ -760,26 +756,38 @@ The `position_adjustment_enable` strategy property enables the usage of `adjust_
For performance reasons, it's disabled by default and freqtrade will show a warning message on startup if enabled.
`adjust_trade_position()` can be used to perform additional orders, for example to manage risk with DCA (Dollar Cost Averaging) or to increase or decrease positions.
`max_entry_position_adjustment` property is used to limit the number of additional buys per trade (on top of the first buy) that the bot can execute. By default, the value is -1 which means the bot have no limit on number of adjustment buys.
The strategy is expected to return a stake_amount (in stake currency) between `min_stake` and `max_stake` if and when an additional buy order should be made (position is increased).
If there are not enough funds in the wallet (the return value is above `max_stake`) then the signal will be ignored.
Additional orders also result in additional fees and those orders don't count towards `max_open_trades`.
This callback is **not** called when there is an open order (either buy or sell) waiting for execution.
`adjust_trade_position()` is called very frequently for the duration of a trade, so you must keep your implementation as performant as possible.
Additional Buys are ignored once you have reached the maximum amount of extra buys that you have set on `max_entry_position_adjustment`, but the callback is called anyway looking for partial exits.
Position adjustments will always be applied in the direction of the trade, so a positive value will always increase your position (negative values will decrease your position), no matter if it's a long or short trade.
Adjustment orders can be assigned with a tag by returning a 2 element Tuple, with the first element being the adjustment amount, and the 2nd element the tag (e.g. `return250,"increase_favorable_conditions"`).
Position adjustments will always be applied in the direction of the trade, so a positive value will always increase your position (negative values will decrease your position), no matter if it's a long or short trade. Modifications to leverage are not possible, and the stake-amount is assumed to be before applying leverage.
Modifications to leverage are not possible, and the stake-amount returned is assumed to be before applying leverage.
### Increase position
The strategy is expected to return a positive **stake_amount** (in stake currency) between `min_stake` and `max_stake` if and when an additional entry order should be made (position is increased -> buy order for long trades, sell order for short trades).
If there are not enough funds in the wallet (the return value is above `max_stake`) then the signal will be ignored.
`max_entry_position_adjustment` property is used to limit the number of additional entries per trade (on top of the first entry order) that the bot can execute. By default, the value is -1 which means the bot have no limit on number of adjustment entries.
Additional entries are ignored once you have reached the maximum amount of extra entries that you have set on `max_entry_position_adjustment`, but the callback is called anyway looking for partial exits.
### Decrease position
The strategy is expected to return a negative stake_amount (in stake currency) for a partial exit.
Returning the full owned stake at that point (`-trade.stake_amount`) results in a full exit.
Returning a value more than the above (so remaining stake_amount would become negative) will result in the bot ignoring the signal.
!!! Note "About stake size"
Using fixed stake size means it will be the amount used for the first order, just like without position adjustment.
If you wish to buy additional orders with DCA, then make sure to leave enough funds in the wallet for that.
Using 'unlimited' stake amount with DCA orders requires you to also implement the `custom_stake_amount()` callback to avoid allocating all funds to the initial order.
Using `"unlimited"` stake amount with DCA orders requires you to also implement the `custom_stake_amount()` callback to avoid allocating all funds to the initial order.
!!! Warning
!!! Warning "Stoploss calculation"
Stoploss is still calculated from the initial opening price, not averaged price.
Regular stoploss rules still apply (cannot move down).
@@ -789,9 +797,13 @@ Position adjustments will always be applied in the direction of the trade, so a
During backtesting this callback is called for each candle in `timeframe` or `timeframe_detail`, so run-time performance will be affected.
This can also cause deviating results between live and backtesting, since backtesting can adjust the trade only once per candle, whereas live could adjust the trade multiple times per candle.
``` python
from freqtrade.persistence import Trade
!!! Warning "Performance with many position adjustments"
Position adjustments can be a good approach to increase a strategy's output - but it can also have drawbacks if using this feature extensively.
Each of the orders will be attached to the trade object for the duration of the trade - hence increasing memory usage.
Trades with long duration and 10s or even 100ds of position adjustments are therefore not recommended, and should be closed at regular intervals to not affect performance.
``` python
# Default imports
class DigDeeperStrategy(IStrategy):
@@ -809,8 +821,8 @@ class DigDeeperStrategy(IStrategy):
# This is called when placing the initial order (opening trade)
Customize leverage for each new trade. This method is only called in futures mode.
@@ -995,7 +1015,7 @@ class AwesomeStrategy(IStrategy):
:param proposed_leverage: A leverage proposed by the bot.
:param max_leverage: Max leverage allowed on this pair
:param entry_tag: Optional entry_tag (buy_tag) if provided with the buy signal.
:param side: 'long' or 'short' - indicating the direction of the proposed trade
:param side: "long" or "short" - indicating the direction of the proposed trade
:return: A leverage amount, which is between 1.0 and max_leverage.
"""
return 1.0
@@ -1003,3 +1023,35 @@ class AwesomeStrategy(IStrategy):
All profit calculations include leverage. Stoploss / ROI also include leverage in their calculation.
Defining a stoploss of 10% at 10x leverage would trigger the stoploss with a 1% move to the downside.
## Order filled Callback
The `order_filled()` callback may be used to perform specific actions based on the current trade state after an order is filled.
It will be called independent of the order type (entry, exit, stoploss or position adjustment).
Assuming that your strategy needs to store the high value of the candle at trade entry, this is possible with this callback as the following example show.
This page explains how to customize your strategies, add new indicators and set up trading rules.
Please familiarize yourself with [Freqtrade basics](bot-basics.md) first, which provides overall info on how the bot operates.
If you haven't already, please familiarize yourself with:
- the [Freqtrade strategy 101](strategy-101.md), which provides a quick start to strategy development
- the [Freqtrade bot basics](bot-basics.md), which provides overall info on how the bot operates
## Develop your own strategy
The bot includes a default strategy file.
Also, several other strategies are available in the [strategy repository](https://github.com/freqtrade/freqtrade-strategies).
You will however most likely have your own idea for a strategy.
This document intends to help you convert your strategy idea into your own strategy.
To get started, use `freqtrade new-strategy --strategy AwesomeStrategy` (you can obviously use your own naming for your strategy).
This will create a new strategy file from a template, which will be located under `user_data/strategies/AwesomeStrategy.py`.
This document intends to help you convert your ideas into a working strategy.
### Generating a strategy template
To get started, you can use the command:
```bash
freqtrade new-strategy --strategy AwesomeStrategy
```
This will create a new strategy called `AwesomeStrategy` from a template, which will be located using the filename `user_data/strategies/AwesomeStrategy.py`.
!!! Note
This is just a template file, which will most likely not be profitable out of the box.
There is a difference between the *name* of the strategy and the filename. In most commands, Freqtrade uses the *name* of the strategy, *not the filename*.
!!! Note
The `new-strategy` command generates starting examples which will not be profitable out of the box.
??? Hint "Different template levels"
`freqtrade new-strategy` has an additional parameter, `--template`, which controls the amount of pre-build information you get in the created strategy. Use `--template minimal` to get an empty strategy without any indicator examples, or `--template advanced` to get a template with most callbacks defined.
`freqtrade new-strategy` has an additional parameter, `--template`, which controls the amount of pre-build information you get in the created strategy. Use `--template minimal` to get an empty strategy without any indicator examples, or `--template advanced` to get a template with more complicated features defined.
### Anatomy of a strategy
A strategy file contains all the information needed to build a good strategy:
A strategy file contains all the information needed to build the strategy logic:
- Candle data in OHLCV format
- Indicators
- Entry strategy rules
-Exit strategy rules
-Minimal ROI recommended
- Stoploss strongly recommended
- Entry logic
-Signals
-Exit logic
- Signals
- Minimal ROI
- Callbacks ("custom functions")
- Stoploss
- Fixed/absolute
- Trailing
- Callbacks ("custom functions")
- Pricing [optional]
- Position adjustment [optional]
The bot also include a sample strategy called `SampleStrategy` you can update: `user_data/strategies/sample_strategy.py`.
You can test it with the parameter: `--strategy SampleStrategy`
The bot includes a sample strategy called `SampleStrategy` that you can use as a basis: `user_data/strategies/sample_strategy.py`.
You can test it with the parameter: `--strategy SampleStrategy`. Remember that you use the strategy class name, not the filename.
Additionally, there is an attribute called `INTERFACE_VERSION`, which defines the version of the strategy interface the bot should use.
The current version is 3 - which is also the default when it's not set explicitly in the strategy.
Future versions will require this to be set.
You may see older strategies set to interface version 2, and these will need to be updated to v3 terminology as future versions will require this to be set.
Starting the bot in dry or live mode is accomplished using the `trade` command:
```bash
freqtrade trade --strategy AwesomeStrategy
```
### Bot modes
Freqtrade strategies can be processed by the Freqtrade bot in 5 main modes:
- backtesting
- hyperopting
- dry ("forward testing")
- live
- FreqAI (not covered here)
Check the [configuration documentation](configuration.md) about how to set the bot to dry or live mode.
**Always use dry mode when testing as this gives you an idea of how your strategy will work in reality without risking capital.**
## Diving in deeper
**For the following section we will use the [user_data/strategies/sample_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/templates/sample_strategy.py)
file as reference.**
!!! Note "Strategies and Backtesting"
To avoid problems and unexpected differences between Backtesting and dry/live modes, please be aware
To avoid problems and unexpected differences between backtesting and dry/live modes, please be aware
that during backtesting the full time range is passed to the `populate_*()` methods at once.
It is therefore best to use vectorized operations (across the whole dataframe, not loops) and
avoid index referencing (`df.iloc[-1]`), but instead use `df.shift()` to get to the previous candle.
@@ -57,12 +98,20 @@ file as reference.**
needs to take care to avoid having the strategy utilize data from the future.
Some common patterns for this are listed in the [Common Mistakes](#common-mistakes-when-developing-strategies) section of this document.
??? Hint "Lookahead and recursive analysis"
Freqtrade includes two helpful commands to help assess common lookahead (using future data) and
recursive bias (variance in indicator values) issues. Before running a strategy in dry or live more,
you should always use these commands first. Please check the relevant documentation for
[lookahead](lookahead-analysis.md) and [recursive](recursive-analysis.md) analysis.
### Dataframe
Freqtrade uses [pandas](https://pandas.pydata.org/) to store/provide the candlestick (OHLCV) data.
Pandas is a great library developed for processing large amounts of data.
Pandas is a great library developed for processing large amounts of data in tabular format.
Each row in a dataframe corresponds to one candle on a chart, with the latest candle always being the last in the dataframe (sorted by date).
Each row in a dataframe corresponds to one candle on a chart, with the latest complete candle always being the last in the dataframe (sorted by date).
If we were to look at the first few rows of the main dataframe using the pandas `head()` function, we would see:
```output
> dataframe.head()
@@ -74,20 +123,16 @@ Each row in a dataframe corresponds to one candle on a chart, with the latest ca
Pandas provides fast ways to calculate metrics. To benefit from this speed, it's advised to not use loops, but use vectorized methods instead.
Vectorized operations perform calculations across the whole range of data and are therefore, compared to looping through each row, a lot faster when calculating indicators.
As a dataframe is a table, simple python comparisons like the following will not work
A dataframe is a table where columns are not single values, but a series of data values. As such, simple python comparisons like the following will not work:
``` python
if dataframe['rsi'] > 30:
dataframe['enter_long'] = 1
```
The above section will fail with `The truth value of a Series is ambiguous. [...]`.
The above section will fail with `The truth value of a Series is ambiguous [...]`.
This must instead be written in a pandas-compatible way, so the operation is performed across the whole dataframe.
This must instead be written in a pandas-compatible way, so the operation is performed across the whole dataframe, i.e. `vectorisation`.
``` python
dataframe.loc[
@@ -97,13 +142,38 @@ This must instead be written in a pandas-compatible way, so the operation is per
With this section, you have a new column in your dataframe, which has `1` assigned whenever RSI is above 30.
Freqtrade uses this new column as an entry signal, where it is assumed that a trade will subsequently open on the next open candle.
Pandas provides fast ways to calculate metrics, i.e. "vectorisation". To benefit from this speed, it is advised to not use loops, but use vectorized methods instead.
Vectorized operations perform calculations across the whole range of data and are therefore, compared to looping through each row, a lot faster when calculating indicators.
??? Hint "Signals vs Trades"
- Signals are generated from indicators at candle close, and are intentions to enter a trade.
- Trades are orders that are executed (on the exchange in live mode) where a trade will then open as close to next candle open as possible.
!!! Warning "Trade order assumptions"
In backtesting, signals are generated on candle close. Trades are then initiated immeditely on next candle open.
In dry and live, this may be delayed due to all pair dataframes needing to be analysed first, then trade processing
for each of those pairs happens. This means that in dry/live you need to be mindful of having as low a computation
delay as possible, usually by running a low number of pairs and having a CPU with a good clock speed.
#### Why can't I see "real time" candle data?
Freqtrade does not store incomplete/unfinished candles in the dataframe.
The use of incomplete data for making strategy decisions is called "repainting" and you might see other platforms allow this.
Freqtrade does not. Only complete/finished candle data is available in the dataframe.
### Customize Indicators
Buy and sell signals need indicators. You can add more indicators by extending the list contained in the method `populate_indicators()` from your strategy file.
Entry and exit signals need indicators. You can add more indicators by extending the list contained in the method `populate_indicators()` from your strategy file.
You should only add the indicators used in either `populate_entry_trend()`, `populate_exit_trend()`, or to populate another indicator, otherwise performance may suffer.
It's important to always return the dataframe without removing/modifying the columns `"open", "high", "low", "close", "volume"`, otherwise these fields would contain something unexpected.
It's important to always return the dataframe from these three functions without removing/modifying the columns `"open", "high", "low", "close", "volume"`, otherwise these fields would contain something unexpected.
Additional technical libraries can be installed as necessary, or custom indicators may be written / invented by the strategy author.
### Strategy startup period
Most indicators have an instable startup period, in which they are either not available (NaN), or the calculation is incorrect. This can lead to inconsistencies, since Freqtrade does not know how long this instable period should be.
Some indicators have an unstable startup period in which there isn't enough candle data to calculate any values (NaN), or the calculation is incorrect. This can lead to inconsistencies, since Freqtrade does not know how long this unstable period is and uses whatever indicator values are in the dataframe.
To account for this, the strategy can be assigned the `startup_candle_count` attribute.
This should be set to the maximum number of candles that the strategy requires to calculate stable indicators. In the case where a user includes higher timeframes with informative pairs, the `startup_candle_count` does not necessarily change. The value is the maximum period (in candles) that any of the informatives timeframes need to compute stable indicators.
You can use [recursive-analysis](recursive-analysis.md) to check and find the correct `startup_candle_count` to be used.
You can use [recursive-analysis](recursive-analysis.md) to check and find the correct `startup_candle_count` to be used. When recursive analysis shows a variance of 0%, then you can be sure that you have enough startup candle data.
In this example strategy, this should be set to 400 (`startup_candle_count = 400`), since the minimum needed history for ema100 calculation to make sure the value is correct is 400 candles.
Assuming `startup_candle_count` is set to 400, backtesting knows it needs 400 candles to generate valid buy signals. It will load data from `20190101 - (400 * 5m)` - which is ~2018-12-30 11:40:00.
If this data is available, indicators will be calculated with this extended timerange. The instable startup period (up to 2019-01-01 00:00:00) will then be removed before starting backtesting.
Assuming `startup_candle_count` is set to 400, backtesting knows it needs 400 candles to generate valid entry signals. It will load data from `20190101 - (400 * 5m)` - which is ~2018-12-30 11:40:00.
!!! Note
If data for the startup period is not available, then the timerange will be adjusted to account for this startup period - so Backtesting would start at 2019-01-02 09:20:00.
If this data is available, indicators will be calculated with this extended timerange. The unstable startup period (up to 2019-01-01 00:00:00) will then be removed before backtesting is carried out.
!!! Note "Unavailable startup candle data"
If data for the startup period is not available, then the timerange will be adjusted to account for this startup period. In our example, backtesting would then start from 2019-01-02 09:20:00.
### Entry signal rules
Edit the method `populate_entry_trend()` in your strategy file to update your entry strategy.
It's important to always return the dataframe without removing/modifying the columns `"open", "high", "low", "close", "volume"`, otherwise these fields would contain something unexpected.
It's important to always return the dataframe without removing/modifying the columns `"open", "high", "low", "close", "volume"`, otherwise these fields would contain something unexpected. The strategy may then produce invalid values, or cease to work entirely.
This method will also define a new column, `"enter_long"` (`"enter_short"` for shorts), which needs to contain 1 for entries, and 0 for "no action". `enter_long` is a mandatory column that must be set even if the strategy is shorting only.
This method will also define a new column, `"enter_long"` (`"enter_short"` for shorts), which needs to contain `1` for entries, and `0` for "no action". `enter_long` is a mandatory column that must be set even if the strategy is shorting only.
You can name your entry signals by using the `"enter_tag"` column, which can help debug and assess your strategy later.
Sample from `user_data/strategies/sample_strategy.py`:
Buying requires sellers to buy from - therefore volume needs to be > 0 (`dataframe['volume'] > 0`) to make sure that the bot does not buy/sell in no-activity periods.
Buying requires sellers to buy from. Therefore volume needs to be > 0 (`dataframe['volume'] > 0`) to make sure that the bot does not buy/sell in no-activity periods.
### Exit signal rules
Edit the method `populate_exit_trend()` into your strategy file to update your exit strategy.
The exit-signal can be suppressed by setting `use_exit_signal` to false in the configuration or strategy.
`use_exit_signal` will not influence [signal collision rules](#colliding-signals) - which will still apply and can prevent entries.
It's important to always return the dataframe without removing/modifying the columns `"open", "high", "low", "close", "volume"`, otherwise these fields would contain something unexpected.
It's important to always return the dataframe without removing/modifying the columns `"open", "high", "low", "close", "volume"`, otherwise these fields would contain something unexpected. The strategy may then produce invalid values, or cease to work entirely.
This method will also define a new column, `"exit_long"` (`"exit_short"` for shorts), which needs to contain 1 for exits, and 0 for "no action".
This method will also define a new column, `"exit_long"` (`"exit_short"` for shorts), which needs to contain `1` for exits, and `0` for "no action".
You can name your exit signals by using the `"exit_tag"` column, which can help debug and assess your strategy later.
Sample from `user_data/strategies/sample_strategy.py`:
This dict defines the minimal Return On Investment (ROI) a trade should reach before exiting, independent from the exit signal.
The `minimal_roi` strategy variable defines the minimal Return On Investment (ROI) a trade should reach before exiting, independent from the exit signal.
It is of the following format, with the dict key (left side of the colon) being the minutes passed since the trade opened, and the value (right side of the colon) being the percentage.
It is of the following format, i.e. a python `dict`, with the dict key (left side of the colon) being the minutes passed since the trade opened, and the value (right side of the colon) being the percentage.
```python
minimal_roi = {
@@ -344,14 +432,19 @@ The above configuration would therefore mean:
The calculation does include fees.
#### Disabling minimal ROI
To disable ROI completely, set it to an empty dictionary:
```python
minimal_roi = {}
```
#### Using calculations in minimal ROI
To use times based on candle duration (timeframe), the following snippet can be handy.
This will allow you to change the timeframe for the strategy, and ROI times will still be set as candles (e.g. after 3 candles ...)
This will allow you to change the timeframe for the strategy, but the minimal ROI times will still be set as candles, e.g. after 3 candles.
``` python
from freqtrade.exchange import timeframe_to_minutes
@@ -367,6 +460,11 @@ class AwesomeStrategy(IStrategy):
}
```
??? info "Orders that don't fill immediately"
`minimal_roi` will take the `trade.open_date` as reference, which is the time the trade was initialized, i.e. when the first order for this trade was placed.
This will also hold true for limit orders that don't fill immediately (usually in combination with "off-spot" prices through `custom_entry_price()`), as well as for cases where the initial order price is replaced through `adjust_entry_price()`.
The time used will still be from the initial `trade.open_date` (when the initial order was first placed), not from the newly placed or adjusted order date.
### Stoploss
Setting a stoploss is highly recommended to protect your capital from strong moves against you.
@@ -381,33 +479,44 @@ For the full documentation on stoploss features, look at the dedicated [stoploss
### Timeframe
This is the set of candles the bot should download and use for the analysis.
This is the periodicity of candles the bot should use in the strategy.
Common values are `"1m"`, `"5m"`, `"15m"`, `"1h"`, however all values supported by your exchange should work.
Please note that the same entry/exit signals may work well with one timeframe, but not with the others.
Please note that the same entry/exit signals may work well with one timeframe, but not with others.
This setting is accessible within the strategy methods as the `self.timeframe` attribute.
### Can short
To use short signals in futures markets, you will have to let us know to do so by setting `can_short=True`.
To use short signals in futures markets, you will have to set `can_short = True`.
Strategies which enable this will fail to load on spot markets.
Disabling of this will have short signals ignored (also in futures markets).
If you have `1` values in the `enter_short` column to raise short signals, setting `can_short = False` (which is the default) will mean that these short signals are ignored, even if you have specified futures markets in your configuration.
### Metadata dict
The metadata-dict (available for `populate_entry_trend`, `populate_exit_trend`, `populate_indicators`) contains additional information.
Currently this is `pair`, which can be accessed using `metadata['pair']` - and will return a pair in the format `XRP/BTC`.
The `metadata` dict (available for `populate_entry_trend`, `populate_exit_trend`, `populate_indicators`) contains additional information.
Currently this is `pair`, which can be accessed using `metadata['pair']`, and will return a pair in the format `XRP/BTC` (or `XRP/BTC:BTC` for futures markets).
The Metadata-dict should not be modified and does not persist information across multiple calls.
Instead, have a look at the [Storing information](strategy-advanced.md#Storing-information) section.
The metadatadict should not be modified and does not persist information across multiple functions in your strategy.
Instead, please check the [Storing information](strategy-advanced.md#storing-information-persistent) section.
--8<-- "includes/strategy-imports.md"
## Strategy file loading
By default, freqtrade will attempt to load strategies from all `.py` files within `user_data/strategies`.
By default, freqtrade will attempt to load strategies from all `.py` files within the `userdir` (default `user_data/strategies`).
Assuming your strategy is called `AwesomeStrategy`, stored in the file `user_data/strategies/AwesomeStrategy.py`, then you can start freqtrade with `freqtrade trade --strategy AwesomeStrategy`.
Note that we're using the class-name, not the file name.
Assuming your strategy is called `AwesomeStrategy`, stored in the file `user_data/strategies/AwesomeStrategy.py`, then you can start freqtrade in dry (or live, depending on your configuration) mode with:
```bash
freqtrade trade --strategy AwesomeStrategy`
```
Note that we're using the class name, not the file name.
You can use `freqtrade list-strategies` to see a list of all strategies Freqtrade is able to load (all strategies in the correct folder).
It will also include a "status" field, highlighting potential problems.
@@ -419,9 +528,11 @@ It will also include a "status" field, highlighting potential problems.
### Get data for non-tradeable pairs
Data for additional, informative pairs (reference pairs) can be beneficial for some strategies.
Data for additional, informative pairs (reference pairs) can be beneficial for some strategies to see data on a wider timeframe.
OHLCV data for these pairs will be downloaded as part of the regular whitelist refresh process and is available via `DataProvider` just as other pairs (see below).
These parts will **not** be traded unless they are also specified in the pair whitelist, or have been selected by Dynamic Whitelisting.
These pairs will **not** be traded unless they are also specified in the pair whitelist, or have been selected by Dynamic Whitelisting, e.g. `VolumePairlist`.
The pairs need to be specified as tuples in the format `("pair", "timeframe")`, with pair as the first and timeframe as the second argument.
@@ -461,18 +572,24 @@ A full sample can be found [in the DataProvider section](#complete-data-provider
In most common case it is possible to easily define informative pairs by using a decorator. All decorated `populate_indicators_*` methods run in isolation,
not having access to data from other informative pairs, in the end all informative dataframes are merged and passed to main `populate_indicators()` method.
When hyperopting, use of hyperoptable parameter `.value` attribute is not supported. Please use `.range` attribute. See [optimizing an indicator parameter](hyperopt.md#optimizing-an-indicator-parameter)
for more information.
To easily define informative pairs, use the `@informative` decorator. All decorated `populate_indicators_*` methods run in isolation,
and do not have access to data from other informative pairs. However, all informative dataframes for each pair are merged and passed to main `populate_indicators()` method.
!!! Note
Do not use the `@informative` decorator if you need to use data from one informative pair when generating another informative pair. Instead, define informative pairs manually as described [in the DataProvider section](#complete-data-provider-sample).
When hyperopting, use of the hyperoptable parameter `.value` attribute is not supported. Please use the `.range` attribute. See [optimizing an indicator parameter](hyperopt.md#optimizing-an-indicator-parameter) for more information.
Do not use `@informative` decorator if you need to use data of one informative pair when generating another informative pair. Instead, define informative pairs
manually as described [in the DataProvider section](#complete-data-provider-sample).
!!! Note
Use string formatting when accessing informative dataframes of other pairs. This will allow easily changing stake currency in config without having to adjust strategy code.
@@ -584,18 +698,15 @@ for more information.
Alternatively column renaming may be used to remove stake currency from column names: `@informative('1h', 'BTC/{stake}', fmt='{base}_{column}_{timeframe}')`.
!!! Warning "Duplicate method names"
Methods tagged with `@informative()` decorator must always have unique names! Re-using same name (for example when copy-pasting already defined informative method)
will overwrite previously defined method and not produce any errors due to limitations of Python programming language. In such cases you will find that indicators
created in earlier-defined methods are not available in the dataframe. Carefully review method names and make sure they are unique!
Methods tagged with the `@informative()` decorator must always have unique names! Reusing the same name (for example when copy-pasting already defined informative methods) will overwrite previously defined methods and not produce any errors due to limitations of Python programming language. In such cases you will find that indicators created in methods higher up in the strategy file are not available in the dataframe. Carefully review method names and make sure they are unique!
### *merge_informative_pair()*
This method helps you merge an informative pair to a regular dataframe without lookahead bias.
It's there to help you merge the dataframe in a safe and consistent way.
This method helps you merge an informative pair to the regular main dataframe safely and consistently, without lookahead bias.
Options:
- Rename the columns for you to create unique columns
- Rename the columns to create unique columns
- Merge the dataframe without lookahead bias
- Forward-fill (optional)
@@ -646,20 +757,20 @@ All columns of the informative dataframe will be available on the returning data
```
!!! Warning "Informative timeframe < timeframe"
Using informative timeframes smaller than the dataframe timeframe is not recommended with this method, as it will not use any of the additional information this would provide.
To use the more detailed information properly, more advanced methods should be applied (which are out of scope for freqtrade documentation, as it'll depend on the respective need).
Using informative timeframes smaller than the main dataframe timeframe is not recommended with this method, as it will not use any of the additional information this would provide.
To use the more detailed information properly, more advanced methods should be applied (which are out of scope for this documentation).
## Additional data (DataProvider)
The strategy provides access to the `DataProvider`. This allows you to get additional data to use in your strategy.
All methods return `None` in case of failure (do not raise an exception).
All methods return `None` in case of failure, i.e. failures do not raise an exception.
Please always check the mode of operation to select the correct method to get data (samples see below).
Please always check the mode of operation to select the correct method to get data (see below for examples).
!!! Warning "Hyperopt"
Dataprovider is available during hyperopt, however it can only be used in `populate_indicators()` within a strategy.
It is not available in `populate_buy()` and `populate_sell()` methods, nor in `populate_indicators()`, if this method located in the hyperopt file.
!!! Warning "Hyperopt Limitations"
The DataProvider is available during hyperopt, however it can only be used in `populate_indicators()` **within a strategy**, not within a hyperopt class file.
It is also not available in `populate_entry_trend()` and `populate_exit_trend()` methods.
### Possible options for DataProvider
@@ -685,30 +796,31 @@ for pair, timeframe in self.dp.available_pairs:
### *current_whitelist()*
Imagine you've developed a strategy that trades the `5m` timeframe using signals generated from a `1d` timeframe on the top 10 volume pairs by volume.
Imagine you've developed a strategy that trades the `5m` timeframe using signals generated from a `1d` timeframe on the top 10 exchange pairs by volume.
The strategy might look something like this:
The strategy logic might look something like this:
*Scan through the top 10 pairs by volume using the `VolumePairList` every 5 minutes and use a 14 day RSI to buy and sell.*
*Scan through the top 10 pairs by volume using the `VolumePairList` every 5 minutes and use a 14 day RSI to enter and exit.*
Due to the limited available data, it's very difficult to resample `5m` candles into daily candles for use in a 14 day RSI. Most exchanges limit us to just 500-1000 candles which effectively gives us around 1.74 daily candles. We need 14 days at least!
Due to the limited available data, it's very difficult to resample `5m` candles into daily candles for use in a 14 day RSI. Most exchanges limit users to just 500-1000 candles which effectively gives us around 1.74 daily candles. We need 14 days at least!
Since we can't resample the data we will have to use an informative pair; and since the whitelist will be dynamic we don't know which pair(s) to use.
Since we can't resample the data we will have to use an informative pair, and since the whitelist will be dynamic we don't know which pair(s) to use! We have a problem!
This is where calling `self.dp.current_whitelist()` comes in handy.
This is where calling `self.dp.current_whitelist()` comes in handy to retrieve only those pairs in the whitelist.
```python
def informative_pairs(self):
# get access to all pairs available in whitelist.
pairs = self.dp.current_whitelist()
# Assign tf to each pair so they can be downloaded and cached for strategy.
# Assign timeframe to each pair so they can be downloaded and cached for strategy.
informative_pairs = [(pair, '1d') for pair in pairs]
return informative_pairs
```
??? Note "Plotting with current_whitelist"
Current whitelist is not supported for `plot-dataframe`, as this command is usually used by providing an explicit pairlist - and would therefore make the return values of this method misleading.
Current whitelist is not supported for `plot-dataframe`, as this command is usually used by providing an explicit pairlist and would therefore make the return values of this method misleading.
It's also not supported for FreqUI visualization in [webserver mode](utils.md#webserver-mode), as the configuration for webserver mode doesn't require a pairlist to be set.
### *get_pair_dataframe(pair, timeframe)*
@@ -749,7 +861,7 @@ if self.dp.runmode.value in ('live', 'dry_run'):
dataframe['best_ask'] = ob['asks'][0][0]
```
The orderbook structure is aligned with the order structure from [ccxt](https://github.com/ccxt/ccxt/wiki/Manual#order-book-structure), so the result will look as follows:
The orderbook structure is aligned with the order structure from [ccxt](https://github.com/ccxt/ccxt/wiki/Manual#order-book-structure), so the result will be formatted as follows:
``` js
{
@@ -767,10 +879,10 @@ The orderbook structure is aligned with the order structure from [ccxt](https://
}
```
Therefore, using `ob['bids'][0][0]` as demonstrated above will result in using the best bid price. `ob['bids'][0][1]` would look at the amount at this orderbook position.
Therefore, using `ob['bids'][0][0]` as demonstrated above will use the best bid price. `ob['bids'][0][1]` would look at the amount at this orderbook position.
!!! Warning "Warning about backtesting"
The order book is not part of the historic data which means backtesting and hyperopt will not work correctly if this method is used, as the method will return uptodate values.
The order book is not part of the historic data which means backtesting and hyperopt will not work correctly if this method is used, as the method will return up-to-date values.
### *ticker(pair)*
@@ -784,12 +896,12 @@ if self.dp.runmode.value in ('live', 'dry_run'):
!!! Warning
Although the ticker data structure is a part of the ccxt Unified Interface, the values returned by this method can
vary for different exchanges. For instance, many exchanges do not return `vwap` values, some exchanges
does not always fills in the `last` field (so it can be None), etc. So you need to carefully verify the ticker
vary for different exchanges. For instance, many exchanges do not return `vwap` values, and some exchanges
do not always fill in the `last` field (so it can be None), etc. So you need to carefully verify the ticker
data returned from the exchange and add appropriate error handling / defaults.
!!! Warning "Warning about backtesting"
This method will always return up-to-date values - so usage during backtesting / hyperopt without runmode checks will lead to wrong results.
This method will always return up-to-date / real-time values. As such, usage during backtesting / hyperopt without runmode checks will lead to wrong results, e.g. your whole dataframe will contain the same single value in all rows.
### Send Notification
@@ -808,7 +920,7 @@ Notifications will only be sent in trading modes (Live/Dry-run) - so this method
!!! Warning "Spamming"
You can spam yourself pretty good by setting `always_send=True` in this method. Use this with great care and only in conditions you know will not happen throughout a candle to avoid a message every 5 seconds.
### Complete Data-provider sample
### Complete DataProvider sample
```python
from freqtrade.strategy import IStrategy, merge_informative_pair
@@ -875,14 +987,14 @@ class SampleStrategy(IStrategy):
## Additional data (Wallets)
The strategy provides access to the `wallets` object. This contains the current balances on the exchange.
The strategy provides access to the `wallets` object. This contains the current balances of your wallets/accounts on the exchange.
!!! Note "Backtesting / Hyperopt"
Wallets behaves differently depending on the function it's called.
Wallets behaves differently depending on the function from which it is called.
Within `populate_*()` methods, it'll return the full wallet as configured.
Within [callbacks](strategy-callbacks.md), you'll get the wallet state corresponding to the actual simulated wallet at that point in the simulation process.
Please always check if `wallets` is available to avoid failures during backtesting.
Always check if `wallets` is available to avoid failures during backtesting.
``` python
if self.wallets:
@@ -901,15 +1013,15 @@ if self.wallets:
## Additional data (Trades)
A history of Trades can be retrieved in the strategy by querying the database.
A history of trades can be retrieved in the strategy by querying the database.
At the top of the file, import Trade.
At the top of the file, import the required object:
```python
from freqtrade.persistence import Trade
```
The following example queries for the current pair and trades from today, however other filters can easily be added.
The following example queries trades from today for the current pair (`metadata['pair']`). Other filters can easily be added.
@@ -927,7 +1039,9 @@ For a full list of available methods, please consult the [Trade object](trade-ob
## Prevent trades from happening for a specific pair
Freqtrade locks pairs automatically for the current candle (until that candle is over) when a pair is sold, preventing an immediate re-buy of that pair.
Freqtrade locks pairs automatically for the current candle (until that candle is over) when a pair exits, preventing an immediate re-entry of that pair.
This is to prevent "waterfalls" of many and frequent trades within a single candle.
Locked pairs will show the message `Pair <pair> is currently locked.`.
@@ -938,7 +1052,7 @@ Sometimes it may be desired to lock a pair after certain events happen (e.g. mul
Freqtrade has an easy method to do this from within the strategy, by calling `self.lock_pair(pair, until, [reason])`.
`until` must be a datetime object in the future, after which trading will be re-enabled for that pair, while `reason` is an optional string detailing why the pair was locked.
Locks can also be lifted manually, by calling `self.unlock_pair(pair)` or `self.unlock_reason(<reason>)` - providing reason the pair was locked with.
Locks can also be lifted manually, by calling `self.unlock_pair(pair)` or `self.unlock_reason(<reason>)`, providing the reason the pair was unlocked.
`self.unlock_reason(<reason>)` will unlock all pairs currently locked with the provided reason.
To verify if a pair is currently locked, use `self.is_pair_locked(pair)`.
@@ -947,7 +1061,7 @@ To verify if a pair is currently locked, use `self.is_pair_locked(pair)`.
Locked pairs will always be rounded up to the next candle. So assuming a `5m` timeframe, a lock with `until` set to 10:18 will lock the pair until the candle from 10:15-10:20 will be finished.
!!! Warning
Manually locking pairs is not available during backtesting, only locks via Protections are allowed.
Manually locking pairs is not available during backtesting. Only locks via Protections are allowed.
#### Pair locking example
@@ -957,7 +1071,7 @@ from datetime import timedelta, datetime, timezone
# Put the above lines a the top of the strategy file, next to all the other imports
# --------
# Within populate indicators (or populate_buy):
# Within populate indicators (or populate_entry_trend):
if self.config['runmode'].value in ('live', 'dry_run'):
# fetch closed trades for the last 2 days
trades = Trade.get_trades_proxy(
@@ -970,9 +1084,9 @@ if self.config['runmode'].value in ('live', 'dry_run'):
Printing more than a few rows is also possible (simply use `print(dataframe)` instead of `print(dataframe.tail())`), however not recommended, as that will be very verbose (~500 lines per pair every 5 seconds).
Printing more than a few rows is also possible by using `print(dataframe)` instead of `print(dataframe.tail())`. However this is not recommended, as can results in a lot of output (~500 lines per pair every 5 seconds).
## Common mistakes when developing strategies
### Peeking into the future while backtesting
### Looking into the future while backtesting
Backtesting analyzes the whole time-range at once for performance reasons. Because of this, strategy authors need to make sure that strategies do not look-ahead into the future.
This is a common pain-point, which can cause huge differences between backtesting and dry/live run methods, since they all use data which is not available during dry/live runs, so these strategies will perform well during backtesting, but will fail / perform badly in real conditions.
Backtesting analyzes the whole dataframe timerange at once for performance reasons. Because of this, strategy authors need to make sure that strategies do not lookahead into the future, i.e. using data that would not be available in dry or live mode.
The following lists some common patterns which should be avoided to prevent frustration:
This is a common pain-point, which can cause huge differences between backtesting and dry/live run methods. Strategies that look into the future will perform well during backtesting, often with incredible profits or winrates, but will fail or perform badly in real conditions.
- don't use `shift(-1)`. This uses data from the future, which is not available.
- don't use `.iloc[-1]` or any other absolute position in the dataframe, this will be different between dry-run and backtesting.
- don't use `dataframe['volume'].mean()`. This uses the full DataFrame for backtesting, including data from the future. Use `dataframe['volume'].rolling(<window>).mean()` instead
- don't use `.resample('1h')`. This uses the left border of the interval, so moves datafrom an hour to the start of the hour. Use `.resample('1h', label='right')` instead.
The following list contains some common patterns which should be avoided to prevent frustration:
- don't use `shift(-1)` or other negative values. This uses data from the future in backtesting, which is not available in dry or live modes.
- don't use `.iloc[-1]` or any other absolute position in the dataframe within `populate_` functions, as this will be different between dry-run and backtesting. Absolute `iloc` indexing is safe to use in callbacks however - see [Strategy Callbacks](strategy-callbacks.md).
- don't use functions that use all dataframe or column values, e.g. `dataframe['mean_volume'] = dataframe['volume'].mean()`. As backtesting uses the full dataframe, at any point in the dataframe, the `'mean_volume'` series would include data from the future. Use rolling() calculations instead, e.g. `dataframe['volume'].rolling(<window>).mean()`.
- don't use `.resample('1h')`. This uses the left border of the period interval, so moves data from an hour boundary to the start of the hour. Use `.resample('1h', label='right')` instead.
!!! Tip "Identifying problems"
You may also want to check the 2 helper commands [lookahead-analysis](lookahead-analysis.md) and [recursive-analysis](recursive-analysis.md), which can each help you figure out problems with your strategy in different ways.
Please treat them as what they are - helpers to identify most common problems. A negative result of each does not guarantee that there's none of the above errors included.
You should always use the two helper commands [lookahead-analysis](lookahead-analysis.md) and [recursive-analysis](recursive-analysis.md), which can each help you figure out problems with your strategy in different ways.
Please treat them as what they are - helpers to identify most common problems. A negative result of each does not guarantee that there are none of the above errors included.
### Colliding signals
When conflicting signals collide (e.g. both `'enter_long'` and `'exit_long'` are 1), freqtrade will do nothing and ignore the entry signal. This will avoid trades that enter, and exit immediately. Obviously, this can potentially lead to missed entries.
When conflicting signals collide (e.g. both `'enter_long'` and `'exit_long'` are set to `1`), freqtrade will do nothing and ignore the entry signal. This will avoid trades that enter, and exit immediately. Obviously, this can potentially lead to missed entries.
The following rules apply, and entry signals will be ignored if more than one of the 3 signals is set:
@@ -1023,11 +1138,11 @@ The following rules apply, and entry signals will be ignored if more than one of
## Further strategy ideas
To get additional Ideas for strategies, head over to the [strategy repository](https://github.com/freqtrade/freqtrade-strategies). Feel free to use them as they are - but results will depend on the current market situation, pairs used etc. - therefore please backtest the strategy for your exchange/desired pairs first, evaluate carefully, use at your own risk.
Feel free to use any of them as inspiration for your own strategies.
We're happy to accept Pull Requests containing new Strategies to that repo.
To get additional ideas for strategies, head over to the [strategy repository](https://github.com/freqtrade/freqtrade-strategies). Feel free to use them as examples, but results will depend on the current market situation, pairs used, etc. Therefore, these strategies should be considered only for learning purposes, not real world trading. Please backtest the strategy for your exchange/desired pairs first, then dry run to evaluate carefully, and use at your own risk.
## Next step
Feel free to use any of them as inspiration for your own strategies. We're happy to accept Pull Requests containing new strategies to the repository.
## Next steps
Now you have a perfect strategy you probably want to backtest it.
Your next step is to learn [How to use the Backtesting](backtesting.md).
Your next step is to learn [how to use backtesting](backtesting.md).
@@ -53,13 +53,14 @@ You can use bots in telegram groups by just adding them to the group. You can fi
}
```
For the Freqtrade configuration, you can then use the the full value (including `-` if it's there) as string:
For the Freqtrade configuration, you can then use the full value (including `-` if it's there) as string:
```json
"chat_id": "-1001332619709"
```
!!! Warning "Using telegram groups"
When using telegram groups, you're giving every member of the telegram group access to your freqtrade bot and to all commands possible via telegram. Please make sure that you can trust everyone in the telegram group to avoid unpleasent surprises.
When using telegram groups, you're giving every member of the telegram group access to your freqtrade bot and to all commands possible via telegram. Please make sure that you can trust everyone in the telegram group to avoid unpleasant surprises.
## Control telegram noise
@@ -93,9 +94,12 @@ Example configuration showing the different settings:
"trailing_stop_loss": "on",
"stop_loss": "on",
"stoploss_on_exchange": "on",
"custom_exit": "silent",
"partial_exit": "on"
"custom_exit": "silent", // custom_exit without specifying an exit reason
// "exit": "off", // Simplistic configuration to disable all exit messages
"exit_cancel": "on",
"exit_fill": "off",
"protection_trigger": "off",
@@ -108,16 +112,16 @@ Example configuration showing the different settings:
},
```
`entry` notifications are sent when the order is placed, while `entry_fill` notifications are sent when the order is filled on the exchange.
`exit` notifications are sent when the order is placed, while `exit_fill` notifications are sent when the order is filled on the exchange.
`*_fill` notifications are off by default and must be explicitly enabled.
`protection_trigger` notifications are sent when a protection triggers and `protection_trigger_global` notifications trigger when global protections are triggered.
`strategy_msg` - Receive notifications from the strategy, sent via `self.dp.send_msg()` from the strategy [more details](strategy-customization.md#send-notification).
`show_candle` - show candle values as part of entry/exit messages. Only possible values are `"ohlc"` or `"off"`.
`balance_dust_level` will define what the `/balance` command takes as "dust" - Currencies with a balance below this will be shown.
`reload` allows you to disable reload-buttons on selected messages.
* `entry` notifications are sent when the order is placed, while `entry_fill` notifications are sent when the order is filled on the exchange.
* `exit` notifications are sent when the order is placed, while `exit_fill` notifications are sent when the order is filled on the exchange.
Exit messages (`exit` and `exit_fill`) can be further controlled at individual exit reasons level, with the specific exit reason as the key. the default for all exit reasons is `on` - but can be configured via special `*` key - which will act as a wildcard for all exit reasons that are not explicitly defined.
* `*_fill` notifications are off by default and must be explicitly enabled.
* `protection_trigger` notifications are sent when a protection triggers and `protection_trigger_global` notifications trigger when global protections are triggered.
* `strategy_msg` - Receive notifications from the strategy, sent via `self.dp.send_msg()` from the strategy [more details](strategy-customization.md#send-notification).
* `show_candle` - show candle values as part of entry/exit messages. Only possible values are `"ohlc"` or `"off"`.
* `balance_dust_level` will define what the `/balance` command takes as "dust" - Currencies with a balance below this will be shown.
* `reload` allows you to disable reload-buttons on selected messages.
## Create a custom keyboard (command shortcut buttons)
@@ -175,11 +179,13 @@ official commands. You can ask at any moment for help with `/help`.
| `/status` | Lists all open trades
| `/status <trade_id>` | Lists one or more specific trade. Separate multiple <trade_id> with a blank space.
| `/status table` | List all open trades in a table format. Pending buy orders are marked with an asterisk (*) Pending sell orders are marked with a double asterisk (**)
| `/order <trade_id>` | Lists orders of one or more specific trade. Separate multiple <trade_id> with a blank space.
| `/trades [limit]` | List all recently closed trades in a table format.
| `/count` | Displays number of trades used and available
| `/locks` | Show currently locked pairs.
| `/unlock <pairorlock_id>` | Remove the lock for this pair (or for this lock id).
| `/marketdir [long | short | even | none]` | Updates the user managed variable that represents the current market direction. If no direction is provided, the currently set direction will be displayed.
| `/list_custom_data <trade_id> [key]` | List custom_data for Trade ID & Key combination. If no Key is supplied it will list all key-value pairs found for that Trade ID.
| **Modify Trade states** |
| `/forceexit <trade_id> | /fx <tradeid>` | Instantly exits the given trade (Ignoring `minimum_roi`).
| `/forceexit all | /fx all` | Instantly exits all open trades (Ignoring `minimum_roi`).
@@ -294,12 +300,12 @@ Return a summary of your profit/loss and performance.
The relative profit of `1.2%` is the average profit per trade.
The relative profit of `15.2 Σ%` is be based on the starting capital - so in this case, the starting capital was `0.00485701 * 1.152 = 0.00738 BTC`.
Starting capital is either taken from the `available_capital` setting, or calculated by using current wallet size - profits.
Profit Factor is calculated as gross profits / gross losses - and should serve as an overall metric for the strategy.
Expectancy corresponds to the average return per currency unit at risk, i.e. the winrate and the risk-reward ratio (the average gain of winning trades compared to the average loss of losing trades).
Expectancy Ratio is expected profit or loss of a subsequent trade based on the performance of all past trades.
Max drawdown corresponds to the backtesting metric `Absolute Drawdown (Account)` - calculated as `(Absolute Drawdown) / (DrawdownHigh + startingBalance)`.
Bot started date will refer to the date the bot was first started. For older bots, this will default to the first trade's open date.
**Starting capital(**) is either taken from the `available_capital` setting, or calculated by using current wallet size - profits.
**Profit Factor** is calculated as gross profits / gross losses - and should serve as an overall metric for the strategy.
**Expectancy** corresponds to the average return per currency unit at risk, i.e. the winrate and the risk-reward ratio (the average gain of winning trades compared to the average loss of losing trades).
**Expectancy Ratio** is expected profit or loss of a subsequent trade based on the performance of all past trades.
**Max drawdown** corresponds to the backtesting metric `Absolute Drawdown (Account)` - calculated as `(Absolute Drawdown) / (DrawdownHigh + startingBalance)`.
**Bot started date** will refer to the date the bot was first started. For older bots, this will default to the first trade's open date.
### /forceexit <trade_id>
@@ -343,7 +349,7 @@ Return the balance of all crypto-currency your have on the exchange.
> **Available:** 3.05890234
> **Balance:** 3.05890234
> **Pending:** 0.0
>
> **Currency:** CVC
> **Available:** 86.64180098
> **Balance:** 86.64180098
@@ -354,6 +360,7 @@ Return the balance of all crypto-currency your have on the exchange.
Per default `/daily` will return the 7 last days. The example below if for `/daily 3`:
> **Daily Profit over the last 3 days:**
```
Day (count) USDT USD Profit %
-------------- ------------ ---------- ----------
@@ -368,6 +375,7 @@ Per default `/weekly` will return the 8 last weeks, including the current week.
from Monday. The example below if for `/weekly 3`:
> **Weekly Profit over the last 3 weeks (starting from Monday):**
| `amount`| float | Amount in Asset / Base currency that is currently owned. Will be 0.0 until the initial order fills. |
| `open_date`| datetime | Timestamp when trade was opened **use `open_date_utc` instead** |
| `open_date_utc`| datetime | Timestamp when trade was opened - in UTC. |
| `close_date`| datetime | Timestamp when trade was closed **use `close_date_utc` instead** |
| `close_date_utc`| datetime | Timestamp when trade was closed - in UTC. |
| `close_profit`| float | Relative profit at the time of trade closure. `0.01` == 1% |
| `close_profit_abs`| float | Absolute profit (in stake currency) at the time of trade closure. |
| `leverage` | float | Leverage used for this trade - defaults to 1.0 in spot markets. |
| `enter_tag`| string | Tag provided on entry via the `enter_tag` column in the dataframe. |
| `is_short` | boolean | True for short trades, False otherwise. |
| `orders` | Order[] | List of order objects attached to this trade (includes both filled and cancelled orders). |
| `date_last_filled_utc` | datetime | Time of the last filled order. |
| `entry_side` | "buy" / "sell" | Order Side the trade was entered. |
| `exit_side` | "buy" / "sell" | Order Side that will result in a trade exit / position reduction. |
| `trade_direction` | "long" / "short" | Trade direction in text - long or short. |
| `nr_of_successful_entries` | int | Number of successful (filled) entry orders. |
| `nr_of_successful_exits` | int | Number of successful (filled) exit orders. |
## Class methods
@@ -126,24 +126,25 @@ An `Order` object will always be tied to it's corresponding [`Trade`](#trade-obj
### Order - Available attributes
an Order object is typically attached to a trade.
Most properties here can be None as they are dependant on the exchange response.
Most properties here can be None as they are dependent on the exchange response.
| Attribute | DataType | Description |
|------------|-------------|-------------|
`trade` | Trade | Trade object this order is attached to
`ft_pair` | string | Pair this order is for
`ft_is_open` | boolean | is the order filled?
`order_type` | string | Order type as defined on the exchange - usually market, limit or stoploss
`status` | string | Status as defined by ccxt. Usually open, closed, expired or canceled
`side` | string | Buy or Sell
`price` | float | Price the order was placed at
`average` | float | Average price the order filled at
`amount` | float | Amount in base currency
`filled` | float | Filled amount (in base currency)
`remaining` | float | Remaining amount
`cost` | float | Cost of the order - usually average * filled (*Exchange dependant on futures, may contain the cost with or without leverage and may be in contracts.*)
`stake_amount` | float | Stake amount used for this order. *Added in 2023.7.*
`order_date` | datetime | Order creation date **use `order_date_utc` instead**
`order_date_utc` | datetime | Order creation date (in UTC)
`order_fill_date` | datetime | Order fill date **use `order_fill_utc` instead**
`order_fill_date_utc` | datetime | Order fill date
| `trade` | Trade | Trade object this order is attached to |
| `ft_pair` | string | Pair this order is for |
| `ft_is_open` | boolean | is the order filled? |
| `order_type` | string | Order type as defined on the exchange - usually market, limit or stoploss |
| `status` | string | Status as defined by ccxt. Usually open, closed, expired or canceled |
| `side` | string | Buy or Sell |
| `price` | float | Price the order was placed at |
| `average` | float | Average price the order filled at |
| `amount` | float | Amount in base currency |
| `filled` | float | Filled amount (in base currency) |
| `remaining` | float | Remaining amount |
| `cost` | float | Cost of the order - usually average * filled (*Exchange dependent on futures, may contain the cost with or without leverage and may be in contracts.*) |
| `stake_amount` | float | Stake amount used for this order. *Added in 2023.7.* |
| `stake_amount_filled` | float | Filled Stake amount used for this order. *Added in 2024.11.* |
| `order_date` | datetime | Order creation date **use `order_date_utc` instead** |
| `order_date_utc` | datetime | Order creation date (in UTC) |
| `order_fill_date` | datetime | Order fill date **use `order_fill_utc` instead** |
| `order_fill_date_utc` | datetime | Order fill date |
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
