this avoids false positives - but could cause false-negatives
if the problem is in a pricing callback.
`--allow-limit-orders` can re-allow limit orders to test for this scenario.
part of #12168
Precision for ta-lib 0.6.x was increased
but is lower by 0001 than what was in the freqtrade ta-lib fix
https://github.com/TA-Lib/ta-lib/blob/main/src/ta_func/ta_utility.h#L257
This won't be a problem for any normal usdt pair.
shitcoins on some exchanges (e.g gate) with 12 zeros before the first significant digit wouldn't have worked today, either.
Modify the original three duplicate functions (_profit_short, _profit_long, _profit), and add _profit_handler and _format_profit_message.
Refactor telegram.py and rpc.py.
Sorry for the duplicate functions yesterday, I was a bit rushed.
Both pytest and ruff have passed.
# Added `/profit_long` and `/profit_short` Commands
Users can now use commands like:
- `/profit_long [<n>]`
- `/profit_short [<n>]`
- `/profit [<n>]`
---
## Key Changes Implemented
### `freqtrade/rpc/telegram.py`:
- The `_profit` command handler has been updated to robustly parse `long` or `short` as optional arguments.
- **Translation:** The `_profit` command handler has been improved to reliably interpret `long` or `short` as optional parameters.
- The determined direction is passed to the RPC layer.
- **Translation:** The direction determined (either `long` or `short`) is passed to the RPC layer.
- The `/help` command documentation is updated.
- **Translation:** The documentation for the `/help` command has been updated accordingly.
---
### `freqtrade/rpc/rpc.py`:
- The `_rpc_trade_statistics` method now accepts a direction parameter.
- **Translation:** The `_rpc_trade_statistics` method has been updated to accept a `direction` parameter.
- The method has been refactored into a main function and a `_process_trade_stats` helper function to reduce complexity and improve readability.
- **Translation:** The method has been refactored into a main function and a helper function, `_process_trade_stats`, to reduce complexity and improve readability.
- The database query filter is dynamically modified to include a condition on `Trade.is_short` when a direction is provided.
- **Translation:** The database query filter dynamically adjusts to include a condition on `Trade.is_short` when a direction is specified.
---
### `tests/rpc/test_rpc_telegram.py`:
- Existing tests for `_profit` have been updated to match the new message format.
- **Translation:** Existing tests for the `_profit` function have been updated to match the new message format.
- New test cases have been added to specifically validate the `long` and `short` filtering functionality.
- **Translation:** New test cases have been added to specifically validate the filtering functionality for `long` and `short` trades.
---
## Testing
- All local `pytest` tests pass successfully.
- **Translation:** All local `pytest` tests have passed successfully.
- All `ruff` linter checks pass.
- **Translation:** All `ruff` code checks have passed.
- As I do not have a full local deployment, I am relying on the CI pipeline for final validation.
- **Translation:** Since I don't have a complete local deployment, I am relying on the CI pipeline for final validation.
---
This time, only a little AI was used :)
Except for the translation.
This commit enhances the /profit Telegram command to allow filtering by trade direction.
- The `_profit` handler in `telegram.py` now parses 'long'/'short' arguments and passes the direction to the RPC layer.
- The `_rpc_trade_statistics` method in `rpc.py` is updated to filter trades based on the provided direction. It has also been refactored for lower complexity.
- The `/help` command documentation is updated to reflect the new functionality.
- Corresponding unit tests in `test_rpc_telegram.py` are updated and extended to cover the new cases.
We'll for now issue a warning about this - and use the "current" tier
This way, gaps in tier data (between maxNotional and the next
minNotional) no longer cause an operational exception.
closes#11923
The bug happened since it just checked the length of the list itself, not what it represents. in this case .*/USDT could be any amount of pairs
when the user sets a max_open_trades of let's say 3 then the pairs only have 3 trade slots of whatever amount of pairs it really has and thereby creating a bottleneck.
This just sets the max_open_trades to -1 without even checking it, letting freqtrade itself handle the amount of trades allowed at a given time.
AI generated strategies and configs cause a lot of noise.
we should be clear that people shall read the documentation first
(this will also allow us to point people at this within the issue).
I got the following error on Pi 2 (using Debian Bookworm armhf arch)
```
...
running bdist_wheel
running build
running build_py
creating build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/lock.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/backend_ctypes.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/_imp_emulation.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/model.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/ffiplatform.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/setuptools_ext.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/error.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/vengine_gen.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/api.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/__init__.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/recompiler.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/cffi_opcode.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/pkgconfig.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/verifier.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/cparser.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/_shimmed_dist_utils.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/commontypes.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/vengine_cpy.py -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/_cffi_include.h -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/parse_c_type.h -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/_embedding.h -> build/lib.linux-armv7l-cpython-311/cffi
copying src/cffi/_cffi_errors.h -> build/lib.linux-armv7l-cpython-311/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-armv7l-cpython-311/src/c
arm-linux-gnueabihf-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/freqtrade/.venv/include -I/usr/include/python3.11 -c src/c/_cffi_backend.c -o build/temp.linux-armv7l-cpython-311/src/c/_cffi_backend.o
src/c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
15 | #include <ffi.h>
| ^~~~~~~
compilation terminated.
error: command '/usr/bin/arm-linux-gnueabihf-gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for cffi
Failed to build cffi
ERROR: Failed to build installable wheels for some pyproject.toml based projects (cffi)
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Failed installing dependencies
```
It can be Easily solved by installing libffi-dev before hand
There are tests failing when using delayed and wrap_non_picklable_objects as decorator.
until I'll find a solution to run generate_optimizer standalone for analyze_per_epoch=True
with warnings.catch_warnings():
from optuna.exceptions import ExperimentalWarning
warnings.filterwarnings("ignore", category=FutureWarning)
this should be when importing sampler
This version supports numpy2 - while still remaining on ta-lib C of 0.4
This will avoid huge update problems, as the underlying library doesn't need to be updated
freqtrade/optimize/space/optunaspaces.py:39: error: Argument 1 to "__init__" of "IntDistribution" has incompatible type "int | float"; expected "int" [arg-type]
freqtrade/optimize/space/optunaspaces.py:39: error: Argument 2 to "__init__" of "IntDistribution" has incompatible type "int | float"; expected "int" [arg-type]
remove all references for ExtraTreesRegressor and skopt.space
@@ -9,8 +9,10 @@ Issues labeled [good first issue](https://github.com/freqtrade/freqtrade/labels/
Few pointers for contributions:
- Create your PR against the `develop` branch, not `stable`.
-New features need to contain unit tests, must conform to PEP8 (max-line-length = 100) and should be documented with the introduction PR.
-PR's can be declared as `[WIP]` - which signify Work in Progress Pull Requests (which are not finished).
-Stick to english in both commit messages, PR descriptions and code comments and variable names.
-New features need to contain unit tests, must pass CI (run pre-commit and pytest to get an early feedback) and should be documented with the introduction PR.
- PR's can be declared as draft - signaling Work in Progress for Pull Requests (which are not finished). We'll still aim to provide feedback on draft PR's in a timely manner.
- If you're using AI for your PR, please both mention it in the PR description and do a thorough review of the generated code. The final responsibility for the code with the PR author, not with the AI.
If you are unsure, discuss the feature on our [discord server](https://discord.gg/p7nuUNVfP7) or in a [issue](https://github.com/freqtrade/freqtrade/issues) before a Pull Request.
### 2. Test if your code corresponds to our style guide
#### Run Ruff
We receive a lot of code that fails preliminary CI checks.
To help with that, we encourage contributors to install the git pre-commit hook that will let you know immediately when you try to commit code that fails these checks.
You can manually run pre-commit with `pre-commit run -a` - or install the git hook with `pre-commit install` to have it run automatically on each commit.
Running `pre-commit run -a` will run all checks, including `ruff`, `mypy` and `codespell` (among others).
#### Additional styles applied
- Have docstrings on all public methods
- Use double-quotes for docstrings
- Multiline docstrings should be indented to the level of the first quote
- Doc-strings should follow the reST format (`:param xxx: ...`, `:return: ...`, `:raises KeyError: ...`)
#### Manually run the individual checks
The following sections describe how to run the individual checks that are running as part of the pre-commit hook.
##### Run ruff
Check your code with ruff to ensure that it follows the style guide.
```bash
ruff check .
ruff format .
```
We receive a lot of code that fails the `ruff` checks.
To help with that, we encourage you to install the git pre-commit
hook that will warn you when you try to commit code that fails these checks.
##### Run mypy
you can manually run pre-commit with `pre-commit run -a`.
##### Additional styles applied
* Have docstrings on all public methods
* Use double-quotes for docstrings
* Multiline docstrings should be indented to the level of the first quote
* Doc-strings should follow the reST format (`:param xxx: ...`, `:return: ...`, `:raises KeyError: ... `)
### 3. Test if all type-hints are correct
#### Run mypy
Check your code with mypy to ensure that it follows the type-hinting rules.
``` bash
mypy freqtrade
```
### 4. Ensure formatting is correct
#### Run ruff
``` bash
ruff format .
```
## (Core)-Committer Guide
### Process: Pull Requests
@@ -118,18 +120,18 @@ Exceptions:
- Ensure cross-platform compatibility for every change that's accepted. Windows, Mac & Linux.
- Ensure no malicious code is introduced into the core code.
- Create issues for any major changes and enhancements that you wish to make. Discuss things transparently and get community feedback.
- Keep feature versions as small as possible, preferably one new feature per version.
- Keep feature PR's as small as possible, preferably one new feature per PR.
- Be welcoming to newcomers and encourage diverse new contributors from all backgrounds. See the Python Community Code of Conduct (https://www.python.org/psf/codeofconduct/).
### Becoming a Committer
Contributors may be given commit privileges. Preference will be given to those with:
1. Past contributions to Freqtrade and other related open-source projects. Contributions to Freqtrade include both code (both accepted and pending) and friendly participation in the issue tracker and Pull request reviews. Both quantity and quality are considered.
1. Past contributions to Freqtrade and other related opensource projects. Contributions to Freqtrade include both code (both accepted and pending) and friendly participation in the issue tracker and Pull request reviews. Both quantity and quality are considered.
1. A coding style that the other core committers find simple, minimal, and clean.
1. Access to resources for cross-platform development and testing.
1. Time to devote to the project regularly.
Being a Committer does not grant write permission on `develop` or `stable` for security reasons (Users trust Freqtrade with their Exchange API keys).
Being a Committer does not automatically grant write permission on `develop` or `stable` for security reasons (Users trust Freqtrade with their Exchange API keys).
After being Committer for some time, a Committer may be named Core Committer and given full repository access.
Freqtrade is a free and open source crypto trading bot written in Python. It is designed to support all major exchanges and be controlled via Telegram or webUI. It contains backtesting, plotting and money management tools as well as strategy optimization by machine learning.
@@ -28,8 +27,9 @@ hesitate to read the source code and understand the mechanism of this bot.
Please read the [exchange specific notes](docs/exchanges.md) to learn about eventual, special configurations needed for each exchange.
- [X] [Binance](https://www.binance.com/)
- [X] [Bitmart](https://bitmart.com/)
- [X] [BingX](https://bingx.com/invite/0EM9RX)
- [X] [Bitget](https://www.bitget.com/)
- [X] [Bitmart](https://bitmart.com/)
- [X] [Bybit](https://bybit.com/)
- [X] [Gate.io](https://www.gate.io/ref/6266643)
- [X] [HTX](https://www.htx.com/)
@@ -42,6 +42,7 @@ Please read the [exchange specific notes](docs/exchanges.md) to learn about even
### Supported Futures Exchanges (experimental)
- [X] [Binance](https://www.binance.com/)
- [X] [Bitget](https://www.bitget.com/)
- [X] [Gate.io](https://www.gate.io/ref/6266643)
- [X] [Hyperliquid](https://hyperliquid.xyz/) (A decentralized exchange, or DEX)
- [X] [OKX](https://okx.com/)
@@ -64,13 +65,12 @@ Please find the complete documentation on the [freqtrade website](https://www.fr
## Features
- [x]**Based on Python 3.10+**: For botting on any operating system - Windows, macOS and Linux.
- [x]**Based on Python 3.11+**: For botting on any operating system - Windows, macOS and Linux.
- [x]**Persistence**: Persistence is achieved through sqlite.
- [x]**Dry-run**: Run the bot without paying money.
- [x]**Backtesting**: Run a simulation of your buy/sell strategy.
- [x]**Strategy Optimization by machine learning**: Use machine learning to optimize your buy/sell strategy parameters with real exchange data.
- [X]**Adaptive prediction modeling**: Build a smart strategy with FreqAI that self-trains to the market via adaptive machine learning methods. [Learn more](https://www.freqtrade.io/en/stable/freqai/)
- [x]**Edge position sizing** Calculate your win rate, risk reward ratio, the best stoploss and adjust your position size before taking a position for each specific market. [Learn more](https://www.freqtrade.io/en/stable/edge/).
- [x]**Whitelist crypto-currencies**: Select which crypto-currency you want to trade or use dynamic whitelists.
- [x]**Blacklist crypto-currencies**: Select which crypto-currency you want to avoid.
- [x]**Builtin WebUI**: Builtin web UI to manage your bot.
@@ -112,7 +112,6 @@ positional arguments:
backtesting-show Show past Backtest results
backtesting-analysis
Backtest Analysis module.
edge Edge module.
hyperopt Hyperopt module.
hyperopt-list List Hyperopt results
hyperopt-show Show details of Hyperopt results
@@ -148,6 +147,8 @@ Telegram is not mandatory. However, this is a great way to control your bot. Mor
-`/stopentry`: Stop entering new trades.
-`/status <trade_id>|[table]`: Lists all or specific open trades.
-`/profit [<n>]`: Lists cumulative profit from all finished trades, over the last n days.
-`/profit_long [<n>]`: Lists cumulative profit from all finished long trades, over the last n days.
-`/profit_short [<n>]`: Lists cumulative profit from all finished short trades, over the last n days.
-`/forceexit <trade_id>|all`: Instantly exits the given trade (Ignoring `minimum_roi`).
-`/fx <trade_id>|all`: Alias to `/forceexit`
-`/performance`: Show performance of each finished trade grouped by pair
@@ -156,6 +157,7 @@ Telegram is not mandatory. However, this is a great way to control your bot. Mor
-`/help`: Show help message.
-`/version`: Show version.
## Development branches
The project is currently setup in two main branches:
@@ -221,7 +223,7 @@ To run this bot we recommend you a cloud instance with a minimum of:
"description":"Trading fee percentage. Can help to simulate slippage in backtesting",
"type":"number",
@@ -258,10 +268,73 @@
"day",
"week",
"month",
"year"
"year",
"weekday"
]
}
},
"backtest_cache":{
"description":"Load a cached backtest result no older than specified age.",
"type":"string",
"enum":[
"none",
"day",
"week",
"month"
]
},
"hyperopt_path":{
"description":"Specify additional lookup path for Hyperopt Loss functions.",
"type":"string"
},
"epochs":{
"description":"Number of training epochs for Hyperopt.",
"type":"integer",
"minimum":1
},
"early_stop":{
"description":"Early stop hyperopt if no improvement after <epochs>. Set to 0 to disable.",
"type":"integer",
"minimum":0
},
"spaces":{
"description":"Hyperopt parameter spaces to optimize. Default is the default set andincludes all spaces except for 'trailing', 'protection', and 'trades'.",
"type":"array",
"items":{
"type":"string"
},
"default":[
"default"
]
},
"analyze_per_epoch":{
"description":"Perform analysis after each epoch in Hyperopt.",
"type":"boolean"
},
"print_all":{
"description":"Print all hyperopt trials, not just the best ones.",
"type":"boolean",
"default":false
},
"hyperopt_jobs":{
"description":"The number of concurrently running jobs for hyperoptimization (hyperopt worker processes). If -1 (default), all CPUs are used, for -2, all CPUs but one are used, etc. If 1 is given, no parallel computing is used.",
"type":"integer",
"default":-1
},
"hyperopt_random_state":{
"description":"Random state for hyperopt trials.",
"type":"integer",
"minimum":0
},
"hyperopt_min_trades":{
"description":"Minimum number of trades per epoch for hyperopt.",
"type":"integer",
"minimum":0
},
"hyperopt_loss":{
"description":"The class name of the hyperopt loss function class (IHyperOptLoss). Different functions can generate completely different results, since the target for optimization is different. Built-in Hyperopt-loss-functions are: ShortTradeDurHyperOptLoss, OnlyProfitHyperOptLoss, SharpeHyperOptLoss, SharpeHyperOptLossDaily, SortinoHyperOptLoss, SortinoHyperOptLossDaily, CalmarHyperOptLoss, MaxDrawDownHyperOptLoss, MaxDrawDownRelativeHyperOptLoss, MaxDrawDownPerPairHyperOptLoss, ProfitDrawDownHyperOptLoss, MultiMetricHyperOptLoss",
"type":"string"
},
"bot_name":{
"description":"Name of the trading bot. Passed via API to a client.",
"type":"string"
@@ -538,10 +611,6 @@
"description":"Exchange configuration.",
"$ref":"#/definitions/exchange"
},
"edge":{
"description":"Edge configuration.",
"$ref":"#/definitions/edge"
},
"log_config":{
"description":"Logging configuration.",
"$ref":"#/definitions/logging"
@@ -566,6 +635,7 @@
"pairlists":{
"description":"Configuration for pairlists.",
"type":"array",
"minItems":1,
"items":{
"type":"object",
"properties":{
@@ -580,6 +650,7 @@
"RemotePairList",
"MarketCapPairList",
"AgeFilter",
"DelistFilter",
"FullTradesFilter",
"OffsetFilter",
"PerformanceFilter",
@@ -610,11 +681,11 @@
"type":"string"
},
"chat_id":{
"description":"Telegram chat or group ID",
"description":"Telegram chat or group ID. Recommended to be set via environment variable FREQTRADE__TELEGRAM__CHAT_ID",
"type":"string"
},
"topic_id":{
"description":"Telegram topic ID - only applicable for group chats",
"description":"Telegram topic ID - only applicable for group chats. Recommended to be set via environment variable FREQTRADE__TELEGRAM__TOPIC_ID",
"type":"string"
},
"authorized_users":{
@@ -773,9 +844,11 @@
"type":"object",
"properties":{
"enabled":{
"description":"Enable webhook notifications.",
"type":"boolean"
},
"url":{
"description":"Webhook URL. Recommended to be set via environment variable FREQTRADE__WEBHOOK__URL",
"type":"string"
},
"format":{
@@ -853,6 +926,7 @@
"type":"boolean"
},
"webhook_url":{
"description":"Discord webhook URL. Recommended to be set via environment variable FREQTRADE__DISCORD__WEBHOOK_URL",
"type":"string"
},
"exit_fill":{
@@ -1168,28 +1242,35 @@
"description":"Name of the exchange.",
"type":"string"
},
"enable_ws":{
"description":"Enable WebSocket connections to the exchange.",
"type":"boolean",
"default":true
},
"key":{
"description":"API key for the exchange.",
"description":"API key for the exchange. Recommended to be set via environment variable FREQTRADE__EXCHANGE__KEY",
"type":"string",
"default":""
},
"secret":{
"description":"API secret for the exchange.",
"description":"API secret for the exchange. Recommended to be set via environment variable FREQTRADE__EXCHANGE__SECRET",
"type":"string",
"default":""
},
"password":{
"description":"Password for the exchange, if required.",
"description":"Password for the exchange, if required. Recommended to be set via environment variable FREQTRADE__EXCHANGE__PASSWORD",
"type":"string",
"default":""
},
"uid":{
"description":"User ID for the exchange, if required.",
"description":"User ID for the exchange, if required. Recommended to be set via environment variable FREQTRADE__EXCHANGE__UID",
"type":"string"
},
"account_id":{
"description":"Account ID for the exchange, if required. Recommended to be set via environment variable FREQTRADE__EXCHANGE__ACCOUNT_ID",
"type":"string"
},
"wallet_address":{
"description":"Wallet address for the exchange, if required. Usually used by DEX exchanges. Recommended to be set via environment variable FREQTRADE__EXCHANGE__WALLET_ADDRESS",
"type":"string"
},
"private_key":{
"description":"Private key for the exchange, if required. Usually used by DEX exchanges. Recommended to be set via environment variable FREQTRADE__EXCHANGE__PRIVATE_KEY",
"type":"string"
},
"pair_whitelist":{
@@ -1213,6 +1294,11 @@
"type":"boolean",
"default":false
},
"enable_ws":{
"description":"Enable WebSocket connections to the exchange.",
"description":"CCXT asynchronous configuration settings.Usually ccxt_config should be used instead.",
"type":"object"
},
"ccxt_sync_config":{
"description":"CCXT synchronous configuration settings. Usually ccxt_config should be used instead.",
"type":"object"
}
},
@@ -1240,52 +1330,6 @@
"name"
]
},
"edge":{
"type":"object",
"properties":{
"enabled":{
"type":"boolean"
},
"process_throttle_secs":{
"type":"integer",
"minimum":600
},
"calculate_since_number_of_days":{
"type":"integer"
},
"allowed_risk":{
"type":"number"
},
"stoploss_range_min":{
"type":"number"
},
"stoploss_range_max":{
"type":"number"
},
"stoploss_range_step":{
"type":"number"
},
"minimum_winrate":{
"type":"number"
},
"minimum_expectancy":{
"type":"number"
},
"min_trade_number":{
"type":"number"
},
"max_trade_duration_minute":{
"type":"integer"
},
"remove_pumps":{
"type":"boolean"
}
},
"required":[
"process_throttle_secs",
"allowed_risk"
]
},
"logging":{
"type":"object",
"properties":{
@@ -1480,6 +1524,11 @@
"type":"boolean",
"default":false
},
"override_exchange_check":{
"description":"Override the exchange check to force FreqAI to use exchanges that may not have enough historic data. Turn this to True if you know your FreqAI model and strategy do not require historical data.",
"type":"boolean",
"default":false
},
"feature_parameters":{
"description":"The parameters used to engineer the feature set",
@@ -46,29 +46,32 @@ ranging from the simplest (0) to the most detailed per pair, per buy and per sel
More options are available by running with the `-h` option.
### Using export-filename
### Using backtest-filename
Normally, `backtesting-analysis` uses the latest backtest results, but if you wanted to go
back to a previous backtest output, you need to supply the `--export-filename` option.
You can supply the same parameter to `backtest-analysis` with the name of the final backtest
output file. This allows you to keep historical versions of backtest results and re-analyse
them at a later date:
By default, `backtesting-analysis` processes the most recent backtest results in the `user_data/backtest_results` directory.
If you want to analyze results from an earlier backtest, use the `--backtest-filename` option to specify the desired file. This lets you revisit and re-analyze historical backtest outputs at any time by providing the filename of the relevant backtest result:
Possible values are either one of "GP", "RF", "ET", "GBRT" (Details can be found in the [scikit-optimize documentation](https://scikit-optimize.github.io/)), or "an instance of a class that inherits from `RegressorMixin` (from sklearn) and where the `predict` method has an optional `return_std` argument, which returns `std(Y | x)` along with `E[Y | x]`".
Possible values are either one of "NSGAIISampler", "TPESampler", "GPSampler", "CmaEsSampler", "NSGAIIISampler", "QMCSampler" (Details can be found in the [optuna-samplers documentation](https://optuna.readthedocs.io/en/stable/reference/samplers/index.html)), or "an instance of a class that inherits from `optuna.samplers.BaseSampler`".
Some research will be necessary to find additional Regressors.
Example for `ExtraTreesRegressor` ("ET") with additional parameters:
# Corresponds to "ET" - but allows additional parameters.
return ExtraTreesRegressor(n_estimators=100)
```
The `dimensions` parameter is the list of `skopt.space.Dimension` objects corresponding to the parameters to be optimized. It can be used to create isotropic kernels for the `skopt.learning.GaussianProcessRegressor` estimator. Here's an example:
Some research will be necessary to find additional Samplers (from optunahub) for example.
!!! Note
While custom estimators can be provided, it's up to you as User to do research on possible parameters and analyze / understand which ones should be used.
If you're unsure about this, best use one of the Defaults (`"ET"` has proven to be the most versatile) without further parameters.
If you're unsure about this, best use one of the Defaults (`"NSGAIIISampler"` has proven to be the most versatile) without further parameters.
Freqtrade will automatically create the tables necessary upon startup.
@@ -315,7 +315,6 @@ $RepeatedMsgReduction on
The syslog address can be either a Unix domain socket (socket filename) or a UDP socket specification, consisting of IP address and UDP port, separated by the `:` character.
So, the following are the examples of possible addresses:
* `"address": "/dev/log"` -- log to syslog (rsyslog) using the `/dev/log` socket, suitable for most systems.
@@ -323,20 +322,18 @@ So, the following are the examples of possible addresses:
* `"address": "localhost:514"` -- log to local syslog using UDP socket, if it listens on port 514.
* `"address": "<ip>:514"` -- log to remote syslog at IP address and port 514. This may be used on Windows for remote logging to an external syslog server.
??? Info "Deprecated - configure syslog via command line"
`--logfile syslog:<syslog_address>` -- send log messages to `syslog` service using the `<syslog_address>` as the syslog address.
`--logfile syslog:<syslog_address>` -- send log messages to `syslog` service using the `<syslog_address>` as the syslog address.
The syslog address can be either a Unix domain socket (socket filename) or a UDP socket specification, consisting of IP address and UDP port, separated by the `:` character.
The syslog address can be either a Unix domain socket (socket filename) or a UDP socket specification, consisting of IP address and UDP port, separated by the `:` character.
So, the following are the examples of possible usages:
So, the following are the examples of possible usages:
* `--logfile syslog:/dev/log` -- log to syslog (rsyslog) using the `/dev/log` socket, suitable for most systems.
* `--logfile syslog` -- same as above, the shortcut for `/dev/log`.
* `--logfile syslog:/var/run/syslog` -- log to syslog (rsyslog) using the `/var/run/syslog` socket. Use this on MacOS.
* `--logfile syslog:localhost:514` -- log to local syslog using UDP socket, if it listens on port 514.
* `--logfile syslog:<ip>:514` -- log to remote syslog at IP address and port 514. This may be used on Windows for remote logging to an external syslog server.
* `--logfile syslog:/dev/log` -- log to syslog (rsyslog) using the `/dev/log` socket, suitable for most systems.
* `--logfile syslog` -- same as above, the shortcut for `/dev/log`.
* `--logfile syslog:/var/run/syslog` -- log to syslog (rsyslog) using the `/var/run/syslog` socket. Use this on MacOS.
* `--logfile syslog:localhost:514` -- log to local syslog using UDP socket, if it listens on port 514.
* `--logfile syslog:<ip>:514` -- log to remote syslog at IP address and port 514. This may be used on Windows for remote logging to an external syslog server.
@@ -5,6 +5,8 @@ This page explains how to validate your strategy performance by using Backtestin
Backtesting requires historic data to be available.
To learn how to get data for the pairs and exchange you're interested in, head over to the [Data Downloading](data-download.md) section of the documentation.
Backtesting is also available in [webserver mode](freq-ui.md#backtesting), which allows you to run backtests via the web interface.
## Backtesting command reference
--8<--"commands/backtesting.md"
@@ -103,12 +105,14 @@ Only use this if you're sure you'll not want to plot or analyze your results fur
The output will show a table containing the realized absolute Profit (in stake currency) for the given timeperiod, as well as wins, draws and losses that materialized (closed) on this day. Below that there will be a second table for the summarized values of weeks indicated by the date of the closing Sunday. The same would apply to a monthly breakdown indicated by the last day of the month.
The output will display tables containing the realized absolute profit (in stake currency) for the selected period, along with additional statistics such as number of trades, profit factor, and distribution of wins, draws, and losses that materialized (closed) on this period.
### Backtest result caching
@@ -435,15 +471,19 @@ To save time, by default backtest will reuse a cached result from within the las
To further analyze your backtest results, freqtrade will export the trades to file by default.
You can then load the trades to perform further analysis as shown in the [data analysis](strategy_analysis_example.md#load-backtest-results-to-pandas-dataframe) backtesting section.
Also, you can use freqtrade in [webserver mode](freq-ui.md#backtesting) to visualize the backtest results in a web interface.
This mode also allows you to load existing backtest results, so you can analyze them without running the backtest again.
For this mode - `--notes"<notes>"` can be used to add notes to the backtest results, which will be shown in the web interface.
### Backtest output file
The output file freqtrade produces is a zip file containing the following files:
- The backtest report in json format
- the market change data in feather format
- a copy of the strategy file
- a copy of the strategy parameters (if a parameter file was used)
- a sanitized copy of the config file
- The market change data in feather format
- A copy of the strategy file
- A copy of the strategy parameters (if a parameter file was used)
- A sanitized copy of the config file
This will ensure results are reproducible - under the assumption that the same data is available.
@@ -461,7 +501,7 @@ Since backtesting lacks some detailed information about what happens within a ca
- Exit-signal is favored over Stoploss, because exit-signals are assumed to trigger on candle's open
- ROI
- Exits are compared to high - but the ROI value is used (e.g. ROI = 2%, high=5% - so the exit will be at 2%)
- Exits are never "below the candle", so a ROI of 2% may result in a exit at 2.4% if low was at 2.4% profit
- Exits are never "below the candle", so a ROI of 2% may result in an exit at 2.4% if low was at 2.4% profit
- ROI entries which came into effect on the triggering candle (e.g. `120: 0.02` for 1h candles, from `60: 0.05`) will use the candle's open as exit rate
- Force-exits caused by `<N>=-1` ROI entries use low as exit value, unless N falls on the candle open (e.g. `120: -1` for 1h candles)
- Stoploss exits happen exactly at stoploss price, even if low was lower, but the loss will be `2 * fees` higher than the stoploss price
This will load 1h data (the main timeframe) as well as 5m data (detail timeframe) for the selected timerange.
The strategy will be analyzed with the 1h timeframe.
Candles where activity may take place (there's an active signal, the pair is in a trade) are evaluated at the 5m timeframe.
Candles where activity may take place (there's an active signal, the pair is in a trade) are evaluated at the 5m timeframe.
This will allow for a more accurate simulation of intra-candle movements - and can lead to different results, especially on higher timeframes.
Entries will generally still happen at the main candle's open, however freed trade slots may be freed earlier (if the exit signal is triggered on the 5m candle), which can then be used for a new trade of a different pair.
@@ -590,5 +630,5 @@ Detailed output for all strategies one after the other will be available, so mak
## Next step
Great, your strategy is profitable. What if the bot can give your the optimal parameters to use for your strategy?
Great, your strategy is profitable. What if the bot can give you the optimal parameters to use for your strategy?
Your next step is to learn [how to find optimal parameters with Hyperopt](hyperopt.md)
@@ -180,7 +180,7 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `minimal_roi` | **Required.** Set the threshold as ratio the bot will use to exit a trade. [More information below](#understand-minimal_roi). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Dict
| `stoploss` | **Required.** Value as ratio of the stoploss used by the bot. More details in the [stoploss documentation](stoploss.md). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Float (as ratio)
| `trailing_stop` | Enables trailing stoploss (based on `stoploss` in either configuration or strategy file). More details in the [stoploss documentation](stoploss.md#trailing-stop-loss). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Boolean
| `trailing_stop_positive` | Changes stoploss once profit has been reached. More details in the [stoploss documentation](stoploss.md#trailing-stop-loss-custom-positive-loss). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Float
| `trailing_stop_positive` | Changes stoploss once profit has been reached. More details in the [stoploss documentation](stoploss.md#trailing-stop-loss-different-positive-loss). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Float
| `trailing_stop_positive_offset` | Offset on when to apply `trailing_stop_positive`. Percentage value which should be positive. More details in the [stoploss documentation](stoploss.md#trailing-stop-loss-only-once-the-trade-has-reached-a-certain-offset). [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `0.0` (no offset).* <br> **Datatype:** Float
| `trailing_only_offset_is_reached` | Only apply trailing stoploss when the offset is reached. [stoploss documentation](stoploss.md). [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `false`.* <br> **Datatype:** Boolean
| `fee` | Fee used during backtesting / dry-runs. Should normally not be configured, which has freqtrade fall back to the exchange default fee. Set as ratio (e.g. 0.001 = 0.1%). Fee is applied twice for each trade, once when buying, once when selling. <br> **Datatype:** Float (as ratio)
@@ -234,7 +234,6 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `exchange.only_from_ccxt` | Prevent data-download from data.binance.vision. Leaving this as false can greatly speed up downloads, but may be problematic if the site is not available.<br>*Defaults to `false`*<br> **Datatype:** Boolean
| `experimental.block_bad_exchanges` | Block exchanges known to not work with freqtrade. Leave on default unless you want to test if that exchange works now. <br>*Defaults to `true`.* <br> **Datatype:** Boolean
| | **Plugins**
| `edge.*` | Please refer to [edge configuration document](edge.md) for detailed explanation of all possible configuration options.
| `pairlists` | Define one or more pairlists to be used. [More information](plugins.md#pairlists-and-pairlist-handlers). <br>*Defaults to `StaticPairList`.* <br> **Datatype:** List of Dicts
| | **Telegram**
| `telegram.enabled` | Enable the usage of Telegram. <br> **Datatype:** Boolean
@@ -567,14 +566,12 @@ Configuration:
### Understand order_time_in_force
The `order_time_in_force` configuration parameter defines the policy by which the order
is executed on the exchange. Three commonly used time in force are:
The `order_time_in_force` configuration parameter defines the policy by which the order is executed on the exchange.
Commonly used time in force are:
**GTC (Good Till Canceled):**
This is most of the time the default time in force. It means the order will remain
on exchange till it is cancelled by the user. It can be fully or partially fulfilled.
If partially fulfilled, the remaining will stay on the exchange till cancelled.
This is most of the time the default time in force. It means the order will remain on exchange till it is cancelled by the user. It can be fully or partially fulfilled. If partially fulfilled, the remaining will stay on the exchange till cancelled.
**FOK (Fill Or Kill):**
@@ -582,19 +579,22 @@ It means if the order is not executed immediately AND fully then it is cancelled
**IOC (Immediate Or Canceled):**
It is the same as FOK (above) except it can be partially fulfilled. The remaining part
is automatically cancelled by the exchange.
It is the same as FOK (above) except it can be partially fulfilled. The remaining part is automatically cancelled by the exchange.
Not necessarily recommended, as this can lead to partial fills below the minimum trade size.
**PO (Post only):**
Post only order. The order is either placed as a maker order, or it is canceled.
This means the order must be placed on orderbook for at least time in an unfilled state.
Please check the [Exchange documentation](exchanges.md) for supported time in force values for your exchange.
#### time_in_force config
The `order_time_in_force` parameter contains a dict with entry and exit time in force policy values.
This can be set in the configuration file or in the strategy.
Values set in the configuration file overwrites values set in the strategy.
Values set in the configuration file overwrite values from in the strategy, following the regular [precedence rules](#configuration-option-prevalence).
The possible values are: `GTC` (default), `FOK` or `IOC`.
@@ -606,9 +606,9 @@ The possible values are: `GTC` (default), `FOK` or `IOC`.
```
!!! Warning
This is ongoing work. For now, it is supported only for binance, gate and kucoin.
Please don't change the default value unless you know what you are doing and have researched the impact of using different values for your particular exchange.
### Fiat conversion
Freqtrade uses the Coingecko API to convert the coin value to it's corresponding fiat value for the Telegram reports.
@@ -672,7 +672,7 @@ Should you experience problems you suspect are caused by websockets, you can dis
}
```
Should you be required to use a proxy, please refer to the [proxy section](#using-proxy-with-freqtrade) for more information.
Should you be required to use a proxy, please refer to the [proxy section](#using-a-proxy-with-freqtrade) for more information.
!!! Info "Rollout"
We're implementing this out slowly, ensuring stability of your bots.
@@ -304,6 +304,13 @@ The `IProtection` parent class provides a helper method for this in `calculate_l
Most exchanges supported by CCXT should work out of the box.
If you need to implement a specific exchange class, these are found in the `freqtrade/exchange` source folder. You'll also need to add the import to `freqtrade/exchange/__init__.py` to make the loading logic aware of the new exchange.
We recommend looking at existing exchange implementations to get an idea of what might be required.
!!! Warning
Implementing and testing an exchange can be a lot of trial and error, so please bear this in mind.
You should also have some development experience, as this is not a beginner task.
To quickly test the public endpoints of an exchange, add a configuration for your exchange to `tests/exchange_online/conftest.py` and run these tests with `pytest --longrun tests/exchange_online/test_ccxt_compat.py`.
Completing these tests successfully a good basis point (it's a requirement, actually), however these won't guarantee correct exchange functioning, as this only tests public endpoints, but no private endpoint (like generate order or similar).
The `Edge Positioning` module uses probability to calculate your win rate and risk reward ratio. It will use these statistics to control your strategy trade entry points, position size and, stoploss.
!!! Danger "Deprecated functionality"
`Edge positioning` (or short Edge) is currently in maintenance mode only (we keep existing functionality alive) and should be considered as deprecated.
It will currently not receive new features until either someone stepped forward to take up ownership of that module - or we'll decide to remove edge from freqtrade.
!!! Warning
When using `Edge positioning` with a dynamic whitelist (VolumePairList), make sure to also use `AgeFilter` and set it to at least `calculate_since_number_of_days` to avoid problems with missing data.
!!! Note
`Edge Positioning` only considers *its own* buy/sell/stoploss signals. It ignores the stoploss, trailing stoploss, and ROI settings in the strategy configuration file.
`Edge Positioning` improves the performance of some trading strategies and *decreases* the performance of others.
## Introduction
Trading strategies are not perfect. They are frameworks that are susceptible to the market and its indicators. Because the market is not at all predictable, sometimes a strategy will win and sometimes the same strategy will lose.
To obtain an edge in the market, a strategy has to make more money than it loses. Making money in trading is not only about *how often* the strategy makes or loses money.
!!! tip "It doesn't matter how often, but how much!"
A bad strategy might make 1 penny in *ten* transactions but lose 1 dollar in *one* transaction. If one only checks the number of winning trades, it would be misleading to think that the strategy is actually making a profit.
The Edge Positioning module seeks to improve a strategy's winning probability and the money that the strategy will make *on the long run*.
We raise the following question[^1]:
!!! Question "Which trade is a better option?"
a) A trade with 80% of chance of losing 100\$ and 20% chance of winning 200\$<br/>
b) A trade with 100% of chance of losing 30\$
???+ Info "Answer"
The expected value of *a)* is smaller than the expected value of *b)*.<br/>
Hence, *b*) represents a smaller loss in the long run.<br/>
However, the answer is: *it depends*
Another way to look at it is to ask a similar question:
!!! Question "Which trade is a better option?"
a) A trade with 80% of chance of winning 100\$ and 20% chance of losing 200\$<br/>
b) A trade with 100% of chance of winning 30\$
Edge positioning tries to answer the hard questions about risk/reward and position size automatically, seeking to minimizes the chances of losing of a given strategy.
### Trading, winning and losing
Let's call $o$ the return of a single transaction $o$ where $o \in \mathbb{R}$. The collection $O = \{o_1, o_2, ..., o_N\}$ is the set of all returns of transactions made during a trading session. We say that $N$ is the cardinality of $O$, or, in lay terms, it is the number of transactions made in a trading session.
!!! Example
In a session where a strategy made three transactions we can say that $O = \{3.5, -1, 15\}$. That means that $N = 3$ and $o_1 = 3.5$, $o_2 = -1$, $o_3 = 15$.
A winning trade is a trade where a strategy *made* money. Making money means that the strategy closed the position in a value that returned a profit, after all deducted fees. Formally, a winning trade will have a return $o_i > 0$. Similarly, a losing trade will have a return $o_j \leq 0$. With that, we can discover the set of all winning trades, $T_{win}$, as follows:
$$ T_{win} = \{ o \in O | o > 0 \} $$
Similarly, we can discover the set of losing trades $T_{lose}$ as follows:
$$ T_{lose} = \{o \in O | o \leq 0\} $$
!!! Example
In a section where a strategy made four transactions $O = \{3.5, -1, 15, 0\}$:<br>
$T_{win} = \{3.5, 15\}$<br>
$T_{lose} = \{-1, 0\}$<br>
### Win Rate and Lose Rate
The win rate $W$ is the proportion of winning trades with respect to all the trades made by a strategy. We use the following function to compute the win rate:
$$W = \frac{|T_{win}|}{N}$$
Where $W$ is the win rate, $N$ is the number of trades and, $T_{win}$ is the set of all trades where the strategy made money.
Similarly, we can compute the rate of losing trades:
$$
L = \frac{|T_{lose}|}{N}
$$
Where $L$ is the lose rate, $N$ is the amount of trades made and, $T_{lose}$ is the set of all trades where the strategy lost money. Note that the above formula is the same as calculating $L = 1 – W$ or $W = 1 – L$
### Risk Reward Ratio
Risk Reward Ratio ($R$) is a formula used to measure the expected gains of a given investment against the risk of loss. It is basically what you potentially win divided by what you potentially lose. Formally:
$$ R = \frac{\text{potential_profit}}{\text{potential_loss}} $$
???+ Example "Worked example of $R$ calculation"
Let's say that you think that the price of *stonecoin* today is 10.0\$. You believe that, because they will start mining stonecoin, it will go up to 15.0\$ tomorrow. There is the risk that the stone is too hard, and the GPUs can't mine it, so the price might go to 0\$ tomorrow. You are planning to invest 100\$, which will give you 10 shares (100 / 10).
R &= \frac{\text{potential_profit}}{\text{potential_loss}}\\
&= \frac{50}{15}\\
&= 3.33
\end{aligned}$<br>
What it effectively means is that the strategy have the potential to make 3.33\$ for each 1\$ invested.
On a long horizon, that is, on many trades, we can calculate the risk reward by dividing the strategy' average profit on winning trades by the strategy' average loss on losing trades. We can calculate the average profit, $\mu_{win}$, as follows:
By combining the Win Rate $W$ and the Risk Reward ratio $R$ to create an expectancy ratio $E$. A expectance ratio is the expected return of the investment made in a trade. We can compute the value of $E$ as follows:
$$E = R * W - L$$
!!! Example "Calculating $E$"
Let's say that a strategy has a win rate $W = 0.28$ and a risk reward ratio $R = 5$. What this means is that the strategy is expected to make 5 times the investment around on 28% of the trades it makes. Working out the example:<br>
$E = R * W - L = 5 * 0.28 - 0.72 = 0.68$
<br>
The expectancy worked out in the example above means that, on average, this strategy' trades will return 1.68 times the size of its losses. Said another way, the strategy makes 1.68\$ for every 1\$ it loses, on average.
This is important for two reasons: First, it may seem obvious, but you know right away that you have a positive return. Second, you now have a number you can compare to other candidate systems to make decisions about which ones you employ.
It is important to remember that any system with an expectancy greater than 0 is profitable using past data. The key is finding one that will be profitable in the future.
You can also use this value to evaluate the effectiveness of modifications to this system.
!!! Note
It's important to keep in mind that Edge is testing your expectancy using historical data, there's no guarantee that you will have a similar edge in the future. It's still vital to do this testing in order to build confidence in your methodology but be wary of "curve-fitting" your approach to the historical data as things are unlikely to play out the exact same way for future trades.
## How does it work?
Edge combines dynamic stoploss, dynamic positions, and whitelist generation into one isolated module which is then applied to the trading strategy. If enabled in config, Edge will go through historical data with a range of stoplosses in order to find buy and sell/stoploss signals. It then calculates win rate and expectancy over *N* trades for each stoploss. Here is an example:
The goal here is to find the best stoploss for the strategy in order to have the maximum expectancy. In the above example stoploss at $3%$ leads to the maximum expectancy according to historical data.
Edge module then forces stoploss value it evaluated to your strategy dynamically.
### Position size
Edge dictates the amount at stake for each trade to the bot according to the following factors:
- Allowed capital at risk
- Stoploss
Allowed capital at risk is calculated as follows:
```
Allowed capital at risk = (Capital available_percentage) X (Allowed risk per trade)
```
Stoploss is calculated as described above with respect to historical data.
The position size is calculated as follows:
```
Position size = (Allowed capital at risk) / Stoploss
```
Example:
Let's say the stake currency is **ETH** and there is $10$ **ETH** on the wallet. The capital available percentage is $50%$ and the allowed risk per trade is $1\%$. Thus, the available capital for trading is $10 * 0.5 = 5$ **ETH** and the allowed capital at risk would be $5 * 0.01 = 0.05$ **ETH**.
-**Trade 1:** The strategy detects a new buy signal in the **XLM/ETH** market. `Edge Positioning` calculates a stoploss of $2\%$ and a position of $0.05 / 0.02 = 2.5$ **ETH**. The bot takes a position of $2.5$ **ETH** in the **XLM/ETH** market.
-**Trade 2:** The strategy detects a buy signal on the **BTC/ETH** market while **Trade 1** is still open. `Edge Positioning` calculates the stoploss of $4\%$ on this market. Thus, **Trade 2** position size is $0.05 / 0.04 = 1.25$ **ETH**.
!!! Tip "Available Capital $\neq$ Available in wallet"
The available capital for trading didn't change in **Trade 2** even with **Trade 1** still open. The available capital **is not** the free amount in the wallet.
-**Trade 3:** The strategy detects a buy signal in the **ADA/ETH** market. `Edge Positioning` calculates a stoploss of $1\%$ and a position of $0.05 / 0.01 = 5$ **ETH**. Since **Trade 1** has $2.5$ **ETH** blocked and **Trade 2** has $1.25$ **ETH** blocked, there is only $5 - 1.25 - 2.5 = 1.25$ **ETH** available. Hence, the position size of **Trade 3** is $1.25$ **ETH**.
!!! Tip "Available Capital Updates"
The available capital does not change before a position is sold. After a trade is closed the Available Capital goes up if the trade was profitable or goes down if the trade was a loss.
- The strategy detects a sell signal in the **XLM/ETH** market. The bot exits **Trade 1** for a profit of $1$ **ETH**. The total capital in the wallet becomes $11$ **ETH** and the available capital for trading becomes $5.5$ **ETH**.
-**Trade 4** The strategy detects a new buy signal int the **XLM/ETH** market. `Edge Positioning` calculates the stoploss of $2\%$, and the position size of $0.055 / 0.02 = 2.75$ **ETH**.
## Edge command reference
--8<--"commands/edge.md"
## Configurations
Edgemodulehasfollowingconfigurationoptions:
|Parameter|Description|
|------------|-------------|
|`enabled`|Iftrue,thenEdgewillrunperiodically.<br>*Defaults to `false`.* <br>**Datatype:** Boolean
| `process_throttle_secs` | How often should Edge run in seconds. <br>*Defaults to `3600` (once per hour).* <br>**Datatype:** Integer
| `calculate_since_number_of_days` | Number of days of data against which Edge calculates Win Rate, Risk Reward and Expectancy. <br>**Note** that it downloads historical data so increasing this number would lead to slowing down the bot. <br>*Defaults to `7`.* <br>**Datatype:** Integer
| `allowed_risk` | Ratio of allowed risk per trade. <br>*Defaults to `0.01` (1%)).* <br>**Datatype:** Float
| `stoploss_range_min` | Minimum stoploss. <br>*Defaults to `-0.01`.* <br>**Datatype:** Float
| `stoploss_range_max` | Maximum stoploss. <br>*Defaults to `-0.10`.* <br>**Datatype:** Float
| `stoploss_range_step` | As an example if this is set to -0.01 then Edge will test the strategy for `[-0.01, -0,02, -0,03 ..., -0.09, -0.10]` ranges. <br>**Note** than having a smaller step means having a bigger range which could lead to slow calculation. <br> If you set this parameter to -0.001, you then slow down the Edge calculation by a factor of 10. <br>*Defaults to `-0.001`.* <br>**Datatype:** Float
| `minimum_winrate` | It filters out pairs which don't have at least minimum_winrate. <br>This comes handy if you want to be conservative and don't comprise win rate in favour of risk reward ratio. <br>*Defaults to `0.60`.* <br>**Datatype:** Float
| `minimum_expectancy` | It filters out pairs which have the expectancy lower than this number. <br>Having an expectancy of 0.20 means if you put 10\$ on a trade you expect a 12\$ return. <br>*Defaults to `0.20`.* <br>**Datatype:** Float
| `min_trade_number` | When calculating *W*, *R* and *E* (expectancy) against historical data, you always want to have a minimum number of trades. The more this number is the more Edge is reliable. <br>Having a win rate of 100% on a single trade doesn't mean anything at all. But having a win rate of 70% over past 100 trades means clearly something. <br>*Defaults to `10` (it is highly recommended not to decrease this number).* <br>**Datatype:** Integer
| `max_trade_duration_minute` | Edge will filter out trades with long duration. If a trade is profitable after 1 month, it is hard to evaluate the strategy based on it. But if most of trades are profitable and they have maximum duration of 30 minutes, then it is clearly a good sign.<br>**NOTICE:** While configuring this value, you should take into consideration your timeframe. As an example filtering out trades having duration less than one day for a strategy which has 4h interval does not make sense. Default value is set assuming your strategy interval is relatively small (1m or 5m, etc.).<br>*Defaults to `1440` (one day).* <br>**Datatype:** Integer
| `remove_pumps` | Edge will remove sudden pumps in a given market while going through historical data. However, given that pumps happen very often in crypto markets, we recommend you keep this off.<br>*Defaults to `false`.* <br>**Datatype:** Boolean
## Running Edge independently
You can run Edge independently in order to see in details the result. Here is an example:
Edge produced the above table by comparing `calculate_since_number_of_days` to `minimum_expectancy` to find `min_trade_number` historical information based on the config file. The timerange Edge uses for its comparisons can be further limited by using the `--timerange` switch.
In live and dry-run modes, after the `process_throttle_secs` has passed, Edge will again process `calculate_since_number_of_days` against `minimum_expectancy` to find `min_trade_number`. If no `min_trade_number` is found, the bot will return "whitelist empty". Depending on the trade strategy being deployed, "whitelist empty" may be return much of the time - or *all* of the time. The use of Edge may also cause trading to occur in bursts, though this is rare.
If you encounter "whitelist empty" a lot, condsider tuning `calculate_since_number_of_days`, `minimum_expectancy` and `min_trade_number` to align to the trading frequency of your strategy.
### Update cached pairs with the latest data
Edge requires historic data the same way as backtesting does.
Please refer to the [Data Downloading](data-download.md) section of the documentation for details.
Doing `--timerange=-20190901` will get all available data until September 1st (excluding September 1st 2019).
The full timerange specification:
* Use tickframes till 2018/01/31: `--timerange=-20180131`
* Use tickframes since 2018/01/31: `--timerange=20180131-`
* Use tickframes since 2018/01/31 till 2018/03/01 : `--timerange=20180131-20180301`
* Use tickframes between POSIX timestamps 1527595200 1527618600: `--timerange=1527595200-1527618600`
[^1]: Question extracted from MIT Opencourseware S096 - Mathematics with applications in Finance: https://ocw.mit.edu/courses/mathematics/18-s096-topics-in-mathematics-with-applications-in-finance-fall-2013/
Kucoin supports [time_in_force](configuration.md#understand-order_time_in_force) with settings "GTC" (good till cancelled), "FOK" (full-or-cancel) and "IOC" (immediate-or-cancel) settings.
!!! Tip "Stoploss on Exchange"
Kucoin supports `stoploss_on_exchange` and can use both stop-loss-market and stop-loss-limit orders. It provides great advantages, so we recommend to benefit from it.
@@ -271,7 +275,9 @@ Using the wrong exchange will result in the error "OKX Error 50119: API key does
## Gate.io
!!! Tip "Stoploss on Exchange"
Gate.io supports `stoploss_on_exchange` and uses `stop-loss-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange..
Gate.io supports `stoploss_on_exchange` and uses `stop-loss-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
Gate.io supports [time_in_force](configuration.md#understand-order_time_in_force) with settings "GTC" (good till cancelled), and "IOC" (immediate-or-cancel) settings.
Gate.io allows the use of `POINT` to pay for fees. As this is not a tradable currency (no regular market available), automatic fee calculations will fail (and default to a fee of 0).
The configuration parameter `exchange.unknown_fee_rate` can be used to specify the exchange rate between Point and the stake currency. Obviously, changing the stake-currency will also require changes to this value.
@@ -286,9 +292,22 @@ Without these permissions, the bot will not start correctly and show errors like
## Bybit
Futures trading on bybit is currently supported for USDT markets, and will use isolated futures mode.
!!! Tip "Stoploss on Exchange"
Bybit (futures only) supports `stoploss_on_exchange` and uses `stop-loss-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
On futures, Bybit supports both `stop-limit` as well as `stop-market` orders. You can use either `"limit"` or `"market"` in the `order_types.stoploss` configuration setting to decide which type to use.
On startup, freqtrade will set the position mode to "One-way Mode" for the whole (sub)account. This avoids making this call over and over again (slowing down bot operations), but means that changes to this setting may result in exceptions and errors.
Bybit supports [time_in_force](configuration.md#understand-order_time_in_force) with settings "GTC" (good till cancelled), "FOK" (full-or-cancel), "IOC" (immediate-or-cancel) and "PO" (Post only) settings.
!!! Warning "Unified accounts"
Freqtrade assumes accounts to be dedicated to the bot.
We therefore recommend the usage of one subaccount per bot. This is especially important when using unified accounts.
Other configurations (multiple bots on one account, manual non-bot trades on the bot account) are not supported and may lead to unexpected behavior.
### Bybit Futures
Futures trading on bybit is supported for isolated futures mode.
On startup, freqtrade will set the position mode to "One-way Mode" for the whole (sub)account. This avoids making this call over and over again (slowing down bot operations), but means that manual changes to this setting may result in exceptions and errors.
As bybit doesn't provide funding rate history, the dry-run calculation is used for live trades as well.
@@ -300,15 +319,6 @@ API Keys for live futures trading must have the following permissions:
We do strongly recommend to limit all API keys to the IP you're going to use it from.
!!! Warning "Unified accounts"
Freqtrade assumes accounts to be dedicated to the bot.
We therefore recommend the usage of one subaccount per bot. This is especially important when using unified accounts.
Other configurations (multiple bots on one account, manual non-bot trades on the bot account) are not supported and may lead to unexpected behavior.
!!! Tip "Stoploss on Exchange"
Bybit (futures only) supports `stoploss_on_exchange` and uses `stop-loss-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
On futures, Bybit supports both `stop-limit` as well as `stop-market` orders. You can use either `"limit"` or `"market"` in the `order_types.stoploss` configuration setting to decide which type to use.
## Bitmart
@@ -328,6 +338,32 @@ It's therefore required to pass the UID as well.
!!! Warning "Necessary Verification"
Bitmart requires Verification Lvl2 to successfully trade on the spot market through the API - even though trading via UI works just fine with just Lvl1 verification.
## Bitget
Bitget requires a passphrase for each api key, you will therefore need to add this key into the configuration so your exchange section looks as follows:
```json
"exchange": {
"name": "bitget",
"key": "your_exchange_key",
"secret": "your_exchange_secret",
"password": "your_exchange_api_key_password",
// ...
}
```
Bitget supports [time_in_force](configuration.md#understand-order_time_in_force) with settings "GTC" (good till cancelled), "FOK" (full-or-cancel), "IOC" (immediate-or-cancel) and "PO" (Post only) settings.
!!! Tip "Stoploss on Exchange"
Bitget supports `stoploss_on_exchange` and can use both stop-loss-market and stop-loss-limit orders. It provides great advantages, so we recommend to benefit from it.
You can use either `"limit"` or `"market"` in the `order_types.stoploss` configuration setting to decide which type of stoploss shall be used.
### Bitget Futures
Futures trading on bitget is supported for isolated futures mode.
On startup, freqtrade will set the position mode to "One-way Mode" for the whole (sub)account. This avoids making this call over and over again (slowing down bot operations), but means that manual changes to this setting may result in exceptions and errors.
## Hyperliquid
!!! Tip "Stoploss on Exchange"
@@ -339,13 +375,13 @@ This needs to be configured like this:
```json
"exchange": {
"name": "hyperliquid",
"walletAddress": "your_eth_wallet_address",
"walletAddress": "your_eth_wallet_address", // This should NOT be your API Wallet Address!
"privateKey": "your_api_private_key",
// ...
}
```
* walletAddress in hex format: `0x<40hexcharacters>` - Can be easily copied from your wallet - and should be your wallet address, not your API Wallet Address.
* walletAddress in hex format: `0x<40hexcharacters>` - Can be easily copied from your wallet - and should be your main wallet address, not your API Wallet Address.
* privateKey in hex format: `0x<64hexcharacters>` - Use the key the API Wallet shows on creation.
Hyperliquid handles deposits and withdrawals on the Arbitrum One chain, a Layer 2 scaling solution built on top of Ethereum. Hyperliquid uses USDC as quote / collateral. The process of depositing USDC on Hyperliquid requires a couple of steps, see [how to start trading](https://hyperliquid.gitbook.io/hyperliquid-docs/onboarding/how-to-start-trading) for details on what steps are needed.
@@ -363,10 +399,54 @@ Hyperliquid handles deposits and withdrawals on the Arbitrum One chain, a Layer
* Create a different software wallet, only transfer the funds you want to trade with to that wallet, and use that wallet to trade on Hyperliquid.
* If you have funds you don't want to use for trading (after making a profit for example), transfer them back to your hardware wallet.
### Hyperliquid Vault / Subaccount
Hyperliquid allows you to create either a vault or a subaccount.
To use these with Freqtrade, you will need to use the following configuration pattern:
``` json
"exchange": {
"name": "hyperliquid",
"walletAddress": "your_master_wallet_address", // Your master wallet address (not the API wallet address and not the vault/subaccount address).
"privateKey": "your_api_private_key", // API wallet private key (see https://app.hyperliquid.xyz/API). You'll only need the private key.
"ccxt_config": {
"options": {
"vaultAddress": "your_vault_address", // Optional, only if you want to use a vault ...
"subAccountAddress": "your_subaccount_address" // OR optional, only if you want to use a subaccount
}
},
// ...
}
```
Your balance and trades will now be used from your vault / subaccount - and no longer from your main account.
!!! Note
You can only use either a vault or a subaccount - not both at the same time.
### Historic Hyperliquid data
The Hyperliquid API does not provide historic data beyond the single call to fetch current data, so downloading data is not possible, as the downloaded data would not constitute proper historic data.
## Bitvavo
If your account is required to use an operatorId, you can set it in the configuration file as follows:
``` json
"exchange": {
"name": "bitvavo",
"key": "",
"secret": "",
"ccxt_config": {
"options": {
"operatorId": "123567"
}
},
}
```
Bitvavo expects the `operatorId` to be an integer.
## All exchanges
Should you experience constant errors with Nonce (like `InvalidNonce`), it is best to regenerate the API keys. Resetting Nonce is difficult and it's usually easier to regenerate the API keys.
@@ -411,3 +491,5 @@ For example, to test the order type `FOK` with Kraken, and modify candle limit t
!!! Warning
Please make sure to fully understand the impacts of these settings before modifying them.
Using `_ft_has_params` overrides may lead to unexpected behavior, and may even break your bot.
We will not be able to provide support for issues caused by custom settings in `_ft_has_params`.
@@ -102,6 +102,14 @@ You can use "current" market data by using the [dataprovider](strategy-customiza
You can use the `/stopentry` command in Telegram to prevent future trade entry, followed by `/forceexit all` (sell all open trades).
### I sold the bot's capital and now there's errors in the log
Freqtrade assumes that the trades it opens are managed only though the bot.
If you happen to (accidentally) sell the bot's capital, freqtrade will try to recover by trying to re-find on-exchange orders.
This is a best-effort approach, and will not work in all cases, especially when using order types that are not supported by freqtrade (OCO, iceberg, etc.), or when working with older trades (where the exchange no longer provides full order information).
The exact limits will vary between exchanges - with the details usually being documented in the exchange's API documentation.
### I want to run multiple bots on the same machine
Please look at the [advanced setup documentation Page](advanced-setup.md#running-multiple-instances-of-freqtrade).
@@ -151,6 +159,14 @@ This warning can point to one of the below problems:
* Barely traded pair -> Check the pair on the exchange webpage, look at the timeframe your strategy uses. If the pair does not have any volume in some candles (usually visualized with a "volume 0" bar, and a "_" as candle), this pair did not have any trades in this timeframe. These pairs should ideally be avoided, as they can cause problems with order-filling.
* API problem -> API returns wrong data (this only here for completeness, and should not happen with supported exchanges).
### I get the message "Couldn't reuse watch for xxx" in the log
This is an informational message that the bot tried to use candles from the websocket, but the exchange didn't provide the right information.
This can happen if there was an interruption to the websocket connection - or if the pair didn't have any trades happen in the timeframe you are using.
Freqtrade will handle this gracefully by falling back to the REST api.
While this makes the iteration slightly slower (due to the REST Api call) - it will not cause any problems to the bot's operation.
### I'm getting the "Exchange XXX does not support market orders." message and cannot run my strategy
As the message says, your exchange does not support market orders and you have one of the [order types](configuration.md/#understand-order_types) set to "market". Your strategy was probably written with other exchanges in mind and sets "market" orders for "stoploss" orders, which is correct and preferable for most of the exchanges supporting market orders (but not for Gate.io).
@@ -219,10 +235,7 @@ On Windows, the `--logfile` option is also supported by Freqtrade and you can us
First of all, most indicator libraries don't have GPU support - as such, there would be little benefit for indicator calculations.
The GPU improvements would only apply to pandas-native calculations - or ones written by yourself.
For hyperopt, freqtrade is using scikit-optimize, which is built on top of scikit-learn.
Their statement about GPU support is [pretty clear](https://scikit-learn.org/stable/faq.html#will-you-add-gpu-support).
GPU's also are only good at crunching numbers (floating point operations).
GPU's are only good at crunching numbers (floating point operations).
For hyperopt, we need both number-crunching (find next parameters) and running python code (running backtesting).
As such, GPU's are not too well suited for most parts of hyperopt.
@@ -271,20 +284,6 @@ Example: 4% profit 650 times vs 0,3% profit a trade 10000 times in a year. If we
### Edge implements interesting approach for controlling position size, is there any theory behind it?
The Edge module is mostly a result of brainstorming of [@mishaker](https://github.com/mishaker) and [@creslinux](https://github.com/creslinux) freqtrade team members.
You can find further info on expectancy, win rate, risk management and position size in the following sources:
@@ -4,7 +4,7 @@ Freqtrade provides a builtin webserver, which can serve [FreqUI](https://github.
By default, the UI is automatically installed as part of the installation (script, docker).
freqUI can also be manually installed by using the `freqtrade install-ui` command.
This same command can also be used to update freqUI to new new releases.
This same command can also be used to update freqUI to new releases.
Once the bot is started in trade / dry-run mode (with `freqtrade trade`) - the UI will be available under the configured API port (by default `http://127.0.0.1:8080`).
@@ -70,7 +70,16 @@ Things you can change (among others):
when freqtrade is started in [webserver mode](utils.md#webserver-mode) (freqtrade started with `freqtrade webserver`), the webserver will start in a special mode allowing for additional features, for example:
* Downloading data
* Testing pairlists
* [Backtesting strategies](#backtesting)
* ... to be expanded
### Backtesting
When freqtrade is started in [webserver mode](utils.md#webserver-mode) (freqtrade started with `freqtrade webserver`), the backtesting view becomes available.
This view allows you to backtest strategies and visualize the results.
@@ -181,7 +181,7 @@ You can ask for each of the defined features to be included also for informative
In total, the number of features the user of the presented example strategy has created is: length of `include_timeframes`* no. features in `feature_engineering_expand_*()` * length of `include_corr_pairlist` * no. `include_shifted_candles` * length of `indicator_periods_candles`
$= 3 * 3 * 3 * 2 * 2 = 108$.
!!! note "Learn more about creative feature engineering"
!!! note "Learn more about creative feature engineering"
Check out our [medium article](https://emergentmethods.medium.com/freqai-from-price-to-prediction-6fadac18b665) geared toward helping users learn how to creatively engineer features.
### Gain finer control over `feature_engineering_*` functions with `metadata`
@@ -310,7 +310,7 @@ class MyCoolTransform(BaseTransform):
If you have created your own custom `IFreqaiModel` with a custom `train()`/`predict()` function, *and* you still rely on `data_cleaning_train/predict()`, then you will need to migrate to the new pipeline. If your model does *not* rely on `data_cleaning_train/predict()`, then you do not need to worry about this migration.
More details about the migration can be found [here](strategy_migration.md#freqai---new-data-pipeline).
More details about the migration can be found [here](strategy_migration.md#freqai-new-data-pipeline).
@@ -79,7 +79,7 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| `model_type` | Model string from stable_baselines3 or SBcontrib. Available strings include: `'TRPO', 'ARS', 'RecurrentPPO', 'MaskablePPO', 'PPO', 'A2C', 'DQN'`. User should ensure that `model_training_parameters` match those available to the corresponding stable_baselines3 model by visiting their documentation. [PPO doc](https://stable-baselines3.readthedocs.io/en/master/modules/ppo.html) (external website) <br> **Datatype:** string.
| `policy_type` | One of the available policy types from stable_baselines3 <br> **Datatype:** string.
| `max_training_drawdown_pct` | The maximum drawdown that the agent is allowed to experience during training. <br> **Datatype:** float. <br> Default: 0.8
| `cpu_count` | Number of threads/cpus to dedicate to the Reinforcement Learning training process (depending on if `ReinforcementLearning_multiproc` is selected or not). Recommended to leave this untouched, by default, this value is set to the total number of physical cores minus 1. <br> **Datatype:** int.
| `cpu_count` | Number of threads/cpus to dedicate to the Reinforcement Learning training process (depending on if `ReinforcementLearner_multiproc` is selected or not). Recommended to leave this untouched, by default, this value is set to the total number of physical cores minus 1. <br> **Datatype:** int.
| `model_reward_parameters` | Parameters used inside the customizable `calculate_reward()` function in `ReinforcementLearner.py` <br> **Datatype:** int.
| `add_state_info` | Tell FreqAI to include state information in the feature set for training and inferencing. The current state variables include trade duration, current profit, trade position. This is only available in dry/live runs, and is automatically switched to false for backtesting. <br> **Datatype:** bool. <br> Default: `False`.
| `net_arch` | Network architecture which is well described in [`stable_baselines3` doc](https://stable-baselines3.readthedocs.io/en/master/guide/custom_policy.html#examples). In summary: `[<sharedlayers>, dict(vf=[<non-sharedvaluenetworklayers>], pi=[<non-sharedpolicynetworklayers>])]`. By default this is set to `[128, 128]`, which defines 2 shared hidden layers with 128 units each.
FreqAI is a software designed to automate a variety of tasks associated with training a predictive machine learning model to generate market forecasts given a set of input signals. In general, FreqAI aims to be a sandbox for easily deploying robust machine learning libraries on real-time data ([details](#freqai-position-in-open-source-machine-learning-landscape)).
!!! Note
FreqAI is, and always will be, a not-for-profit, open-source project. FreqAI does *not* have a crypto token, FreqAI does *not* sell signals, and FreqAI does not have a domain besides the present [freqtrade documentation](https://www.freqtrade.io/en/latest/freqai/).
FreqAI is, and always will be, a not-for-profit, opensource project. FreqAI does *not* have a crypto token, FreqAI does *not* sell signals, and FreqAI does not have a domain besides the present [freqtrade documentation](https://www.freqtrade.io/en/latest/freqai/).
If you are using docker, a dedicated tag with FreqAI dependencies is available as `:freqai`. As such - you can replace the image line in your docker compose file with `image: freqtradeorg/freqtrade:stable_freqai`. This image contains the regular FreqAI dependencies. Similar to native installs, Catboost will not be available on ARM based devices. If you would like to use PyTorch or Reinforcement learning, you should use the torch or RL tags, `image: freqtradeorg/freqtrade:stable_freqaitorch`, `image: freqtradeorg/freqtrade:stable_freqairl`.
!!! note "docker-compose-freqai.yml"
We do provide an explicit docker-compose file for this in `docker/docker-compose-freqai.yml` - which can be used via `docker compose -f docker/docker-compose-freqai.yml run ...` - or can be copied to replace the original docker file. This docker-compose file also contains a (disabled) section to enable GPU resources within docker containers. This obviously assumes the system has GPU resources available.
We do provide an explicit docker-compose file for this in `docker/docker-compose-freqai.yml` - which can be used via `docker compose -f docker/docker-compose-freqai.yml run ...` - or can be copied to replace the original docker file. This docker-compose file also contains a (disabled) section to enable GPU resources within docker containers. This obviously assumes the system has GPU resources available.
### FreqAI position in open-source machine learning landscape
### FreqAI position in opensource machine learning landscape
Forecasting chaotic time-series based systems, such as equity/cryptocurrency markets, requires a broad set of tools geared toward testing a wide range of hypotheses. Fortunately, a recent maturation of robust machine learning libraries (e.g. `scikit-learn`) has opened up a wide range of research possibilities. Scientists from a diverse range of fields can now easily prototype their studies on an abundance of established machine learning algorithms. Similarly, these user-friendly libraries enable "citizen scientists" to use their basic Python skills for data exploration. However, leveraging these machine learning libraries on historical and live chaotic data sources can be logistically difficult and expensive. Additionally, robust data collection, storage, and handling presents a disparate challenge. [`FreqAI`](#freqai) aims to provide a generalized and extensible open-sourced framework geared toward live deployments of adaptive modeling for market forecasting. The `FreqAI` framework is effectively a sandbox for the rich world of open-source machine learning libraries. Inside the `FreqAI` sandbox, users find they can combine a wide variety of third-party libraries to test creative hypotheses on a free live 24/7 chaotic data source - cryptocurrency exchange data.
Forecasting chaotic time-series based systems, such as equity/cryptocurrency markets, requires a broad set of tools geared toward testing a wide range of hypotheses. Fortunately, a recent maturation of robust machine learning libraries (e.g. `scikit-learn`) has opened up a wide range of research possibilities. Scientists from a diverse range of fields can now easily prototype their studies on an abundance of established machine learning algorithms. Similarly, these user-friendly libraries enable "citizen scientists" to use their basic Python skills for data exploration. However, leveraging these machine learning libraries on historical and live chaotic data sources can be logistically difficult and expensive. Additionally, robust data collection, storage, and handling presents a disparate challenge. [`FreqAI`](#freqai) aims to provide a generalized and extensible open-sourced framework geared toward live deployments of adaptive modeling for market forecasting. The `FreqAI` framework is effectively a sandbox for the rich world of opensource machine learning libraries. Inside the `FreqAI` sandbox, users find they can combine a wide variety of third-party libraries to test creative hypotheses on a free live 24/7 chaotic data source - cryptocurrency exchange data.
This page explains how to tune your strategy by finding the optimal
parameters, a process called hyperparameter optimization. The bot uses algorithms included in the `scikit-optimize` package to accomplish this.
parameters, a process called hyperparameter optimization. The bot uses algorithms included in the `optuna` package to accomplish this.
The search will burn all your CPU cores, make your laptop sound like a fighter jet and still take a long time.
In general, the search for best parameters starts with a few random combinations (see [below](#reproducible-results) for more details) and then uses Bayesian search with a ML regressor algorithm (currently ExtraTreesRegressor) to quickly find a combination of parameters in the search hyperspace that minimizes the value of the [loss function](#loss-functions).
In general, the search for best parameters starts with a few random combinations (see [below](#reproducible-results) for more details) and then uses one of optuna's sampler algorithms (currently NSGAIIISampler) to quickly find a combination of parameters in the search hyperspace that minimizes the value of the [loss function](#loss-functions).
Hyperopt requires historic data to be available, just as backtesting does (hyperopt runs backtesting many times with different parameters).
To learn how to get data for the pairs and exchange you're interested in, head over to the [Data Downloading](data-download.md) section of the documentation.
@@ -46,10 +46,17 @@ Depending on the space you want to optimize, only some of the below are required
@@ -79,15 +86,15 @@ Based on the loss function result, hyperopt will determine the next set of param
### Configure your Guards and Triggers
There are two places you need to change in your strategy file to add a new buy hyperopt for testing:
There are two places you need to change in your strategy file to add a new hyperopt parameter for optimization:
* Define the parameters at the class level hyperopt shall be optimizing.
* Within `populate_entry_trend()` - use defined parameter values instead of raw constants.
There you have two different types of indicators: 1. `guards` and 2. `triggers`.
1. Guards are conditions like "never buy if ADX < 10", or never buy if current price is over EMA10.
2. Triggers are ones that actually trigger buy in specific moment, like "buy when EMA5 crosses over EMA10" or "buy when close price touches lower Bollinger band".
1. Guards are conditions like "never enter if ADX < 10", or never enter if current price is over EMA10.
2. Triggers are ones that actually trigger entry in specific moment, like "enter when EMA5 crosses over EMA10" or "enter when close price touches lower Bollinger band".
!!! Hint "Guards and Triggers"
Technically, there is no difference between Guards and Triggers.
@@ -160,9 +167,11 @@ We use these to either enable or disable the ADX and RSI guards.
The last one we call `trigger` and use it to decide which buy trigger we want to use.
!!! Note "Parameter space assignment"
Parameters must either be assigned to a variable named `buy_*` or `sell_*` - or contain `space='buy'` | `space='sell'` to be assigned to a space correctly.
If no parameter is available for a space, you'll receive the error that no space was found when running hyperopt.
- Parameters must either be assigned to a variable named `buy_*`, `sell_*`, `enter_*` or `exit_*` or `protection_*` - or contain have a space assigned explicitly via parameter (`space='buy'`, `space='sell'`, `space='protection'`).
- Parameters with conflicting assignments (e.g. `buy_adx = IntParameter(4, 24, default=14, space='sell')`) will use the explicit space assignment.
- If no parameter is available for a space, you'll receive the error that no space was found when running hyperopt.
Parameters with unclear space (e.g. `adx_period = IntParameter(4, 24, default=14)` - no explicit nor implicit space) will not be detected and will therefore be ignored.
Spaces can also be custom named (e.g. `space='my_custom_space'`), with the only limitation that the space name cannot be `all`, `default` - and must result in a valid python identifier.
So let's write the buy strategy using these values:
The `-e` option will set how many evaluations hyperopt will do. Since hyperopt uses Bayesian search, running too many epochs at once may not produce greater results. Experience has shown that best results are usually not improving much after 500-1000 epochs.
The `--early-stop` option will set after how many epochs with no improvements hyperopt will stop. A good value is 20-30% of the total epochs. Any value greater than 0 and lower than 20 it will be replaced by 20. Early stop is by default disabled (`--early-stop=0`)
Doing multiple runs (executions) with a few 1000 epochs and different random state will most likely produce different results.
The `--spaces all` option determines that all possible parameters should be optimized. Possibilities are listed below.
* `roi`: just optimize the minimal profit table for your strategy
* `stoploss`: search for the best stoploss value
* `trailing`: search for the best trailing stop values
* `trades`: search for the best max open trades values
* `protection`: search for the best protection parameters (read the [protections section](#optimizing-protections) on how to properly define these)
* `default`: `all` except `trailing` and `protection`
* `default`: `all` except `trailing`, `trades` and `protection`
* `custom_space_name`: any custom space used by any parameter in your strategy
* space-separated list of any of the above values for example `--spaces roi stoploss`
The default Hyperopt Search Space, used when no `--space` command line option is specified, does not include the `trailing` hyperspace. We recommend you to run optimization for the `trailing` hyperspace separately, when the best parameters for other hyperspaces were found, validated and pasted into your custom strategy.
@@ -4,7 +4,7 @@ Pairlist Handlers define the list of pairs (pairlist) that the bot should trade.
In your configuration, you can use Static Pairlist (defined by the [`StaticPairList`](#static-pair-list) Pairlist Handler) and Dynamic Pairlist (defined by the [`VolumePairList`](#volume-pair-list) and [`PercentChangePairList`](#percent-change-pair-list) Pairlist Handlers).
Additionally, [`AgeFilter`](#agefilter), [`PrecisionFilter`](#precisionfilter), [`PriceFilter`](#pricefilter), [`ShuffleFilter`](#shufflefilter), [`SpreadFilter`](#spreadfilter) and [`VolatilityFilter`](#volatilityfilter) act as Pairlist Filters, removing certain pairs and/or moving their positions in the pairlist.
Additionally, [`AgeFilter`](#agefilter), [`DelistFilter`](#delistfilter), [`PrecisionFilter`](#precisionfilter), [`PriceFilter`](#pricefilter), [`ShuffleFilter`](#shufflefilter), [`SpreadFilter`](#spreadfilter) and [`VolatilityFilter`](#volatilityfilter) act as Pairlist Filters, removing certain pairs and/or moving their positions in the pairlist.
If multiple Pairlist Handlers are used, they are chained and a combination of all Pairlist Handlers forms the resulting pairlist the bot uses for trading and backtesting. Pairlist Handlers are executed in the sequence they are configured. You can define either `StaticPairList`, `VolumePairList`, `ProducerPairList`, `RemotePairList`, `MarketCapPairList` or `PercentChangePairList` as the starting Pairlist Handler.
@@ -27,6 +27,7 @@ You may also use something like `.*DOWN/BTC` or `.*UP/BTC` to exclude leveraged
* [`RemotePairList`](#remotepairlist)
* [`MarketCapPairList`](#marketcappairlist)
* [`AgeFilter`](#agefilter)
* [`DelistFilter`](#delistfilter)
* [`FullTradesFilter`](#fulltradesfilter)
* [`OffsetFilter`](#offsetfilter)
* [`PerformanceFilter`](#performancefilter)
@@ -38,7 +39,7 @@ You may also use something like `.*DOWN/BTC` or `.*UP/BTC` to exclude leveraged
* [`VolatilityFilter`](#volatilityfilter)
!!! Tip "Testing pairlists"
Pairlist configurations can be quite tricky to get right. Best use the [`test-pairlist`](utils.md#test-pairlist) utility sub-command to test your configuration quickly.
Pairlist configurations can be quite tricky to get right. Best use freqUI in [webserver mode](freq-ui.md#webserver-mode) or the [`test-pairlist`](utils.md#test-pairlist) utility sub-command to test your Pairlist configuration quickly.
#### Static Pair List
@@ -180,7 +181,7 @@ More sophisticated approach can be used, by using `lookback_timeframe` for candl
* `refresh_period`: Defines the interval (in seconds) at which the pairlist will be refreshed. The default is 1800 seconds (30 minutes).
* `lookback_days`: Number of days to look back. When `lookback_days` is selected, the `lookback_timeframe` is defaulted to 1 day.
* `lookback_timeframe`: Timeframe to use for the lookback period.
* `lookback_period`: Number of periods to look back at.
* `lookback_period`: Number of periods to look back at.
When PercentChangePairList is used after other Pairlist Handlers, it will operate on the outputs of those handlers. If it is the leading Pairlist Handler, it will select pairs from all available markets with the specified stake currency.
@@ -270,7 +271,6 @@ You can limit the length of the pairlist with the optional parameter `number_ass
],
```
!!! Tip "Combining pairlists"
This pairlist can be combined with all other pairlists and filters for further pairlist reduction, and can also act as an "additional" pairlist, on top of already defined pairs.
`ProducerPairList` can also be used multiple times in sequence, combining the pairs from multiple producers.
@@ -312,7 +312,7 @@ The `pairlist_url` option specifies the URL of the remote server where the pairl
The `save_to_file` option, when provided with a valid filename, saves the processed pairlist to that file in JSON format. This option is optional, and by default, the pairlist is not saved to a file.
??? Example "Multi bot with shared pairlist example"
`save_to_file` can be used to save the pairlist to a file with Bot1:
```json
@@ -367,7 +367,7 @@ The optional `bearer_token` will be included in the requests Authorization Heade
#### MarketCapPairList
`MarketCapPairList` employs sorting/filtering of pairs by their marketcap rank based of CoinGecko. The returned pairlist will be sorted based of their marketcap ranks.
`MarketCapPairList` employs sorting/filtering of pairs by their marketcap rank based of CoinGecko. The returned pairlist will be sorted based of their marketcap ranks if used in whitelist `mode`.
```json
"pairlists": [
@@ -376,19 +376,26 @@ The optional `bearer_token` will be included in the requests Authorization Heade
"number_assets": 20,
"max_rank": 50,
"refresh_period": 86400,
"mode": "whitelist",
"categories": ["layer-1"]
}
]
```
`number_assets` defines the maximum number of pairs returned by the pairlist. `max_rank` will determine the maximum rank used in creating/filtering the pairlist. It's expected that some coins within the top `max_rank` marketcap will not be included in the resulting pairlist since not all pairs will have active trading pairs in your preferred market/stake/exchange combination.
`number_assets` defines the maximum number of pairs returned by the pairlist if used in whitelist `mode`. In blacklist `mode`, this setting will be ignored.
`max_rank` will determine the maximum rank used in creating/filtering the pairlist. It's expected that some coins within the top `max_rank` marketcap will not be included in the resulting pairlist since not all pairs will have active trading pairs in your preferred market/stake/exchange combination.
While using a `max_rank` bigger than 250 is supported, it's not recommended, as it'll cause multiple API calls to CoinGecko, which can lead to rate limit issues.
The `refresh_period` setting defines the interval (in seconds) at which the marketcap rank data will be refreshed. The default is 86,400 seconds (1 day). The pairlist cache (`refresh_period`) applies to both generating pairlists (when in the first position in the list) and filtering instances (when not in the first position in the list).
The `mode` setting defines whether the plugin will filters in (whitelist `mode`) or filters out (blacklist `mode`) top marketcap ranked coins. By default, the plugin will be in whitelist mode.
The `categories` setting specifies the [coingecko categories](https://www.coingecko.com/en/categories) from which to select coins from. The default is an empty list `[]`, meaning no category filtering is applied.
If an incorrect category string is chosen, the plugin will print the available categories from CoinGecko and fail. The category should be the ID of the category, for example, for `https://www.coingecko.com/en/categories/layer-1`, the category ID would be `layer-1`. You can pass multiple categories such as `["layer-1", "meme-token"]` to select from several categories.
Coins like 1000PEPE/USDT or KPEPE/USDT:USDT are detected on a best effort basis, with the prefixes `1000` and `K` being used to identify them.
!!! Warning "Many categories"
Each added category corresponds to one API call to CoinGecko. The more categories you add, the longer the pairlist generation will take, potentially causing rate limit issues.
@@ -405,6 +412,16 @@ be caught out buying before the pair has finished dropping in price.
This filter allows freqtrade to ignore pairs until they have been listed for at least `min_days_listed` days and listed before `max_days_listed`.
#### DelistFilter
Removes pairs that will be delisted on the exchange maximum `max_days_from_now` days from now (defaults to `0` which remove all future delisted pairs no matter how far from now). Currently this filter only supports following exchanges:
!!! Note "Available exchanges"
Delist filter is available on Bybit Futures, Bitget Futures and Binance, where Binance Futures will work for both dry and live modes, while Binance Spot is limited to live mode (for technical reasons).
!!! Warning "Backtesting"
`DelistFilter` does not support backtesting mode.
#### FullTradesFilter
Shrink whitelist to consist only in-trade pairs when the trade slots are full (when `max_open_trades` isn't being set to `-1` in the config).
@@ -436,7 +453,7 @@ Example to remove the first 10 pairs from the pairlist, and takes the next 20 (t
```
!!! Warning
When `OffsetFilter` is used to split a larger pairlist among multiple bots in combination with `VolumeFilter`
When `OffsetFilter` is used to split a larger pairlist among multiple bots in combination with `VolumeFilter`
it can not be guaranteed that pairs won't overlap due to slightly different refresh intervals for the
`VolumeFilter`.
@@ -599,7 +616,7 @@ Adding `"sort_direction": "asc"` or `"sort_direction": "desc"` enables sorting m
### Full example of Pairlist Handlers
The below example blacklists `BNB/BTC`, uses `VolumePairList` with `20` assets, sorting pairs by `quoteVolume` and applies [`PrecisionFilter`](#precisionfilter) and [`PriceFilter`](#pricefilter), filtering all assets where 1 price unit is > 1%. Then the [`SpreadFilter`](#spreadfilter) and [`VolatilityFilter`](#volatilityfilter) is applied and pairs are finally shuffled with the random seed set to some predefined value.
The below example blacklists `BNB/BTC`, uses `VolumePairList` with `20` assets, sorting pairs by `quoteVolume`, then filter future delisted pairs using [`DelistFilter`](#delistfilter) and [`AgeFilter`](#agefilter) to remove pairs that are listed less than 10 days ago. After that [`PrecisionFilter`](#precisionfilter) and [`PriceFilter`](#pricefilter) are applied, filtering all assets where 1 price unit is > 1%. Then the [`SpreadFilter`](#spreadfilter) and [`VolatilityFilter`](#volatilityfilter) are applied and pairs are finally shuffled with the random seed set to some predefined value.
```json
"exchange": {
@@ -612,6 +629,10 @@ The below example blacklists `BNB/BTC`, uses `VolumePairList` with `20` assets,
Freqtrade allows your strategy to implement different exit logic using signal-based or callback-based functions.
This section aims to compare each different function, helping you to choose the one that best fits your needs.
* **`populate_exit_trend()`** - Vectorized signal-based exit logic using indicators in the main dataframe
✅ **Use** to define exit signals based on indicators or other data that can be calculated in a vectorized manner.
🚫 **Don't use** to customize exit conditions for each individual trade, or if trade data is necessary to make an exit decision.
* **`custom_exit()`** - Custom exit logic that will fully exit a trade immediately, called for every open trade at every bot loop iteration until a trade is closed.
✅ **Use** to specify exit conditions for each individual trade (including any additional adjusted orders using `adjust_trade_position()`), or if trade data is necessary to make an exit decision, e.g. using profit data to exit.
🚫 **Don't use** when you want to exit using vectorised indicator-based data (use a `populate_exit_trend()` signal instead), or as a proxy for `custom_stoploss()`, and be aware that rate-based exits in backtesting can be inaccurate.
* **`custom_stoploss()`** - Custom trailing stoploss, called for every open trade every iteration until a trade is closed. The value returned here is also used for [stoploss on exchange](stoploss.md#stop-loss-on-exchangefreqtrade).
✅ **Use** to customize the stoploss logic to set a dynamic stoploss based on trade data or other conditions.
🚫 **Don't use** to exit a trade immediately based on a specific condition. Use `custom_exit()` for that purpose.
* **`custom_roi()`** - Custom ROI, called for every open trade every iteration until a trade is closed.
✅ **Use** to specify a minimum ROI threshold ("take-profit") to exit a trade at this ROI level at some point within the trade duration, based on profit or other conditions.
🚫 **Don't use** to exit a trade immediately based on a specific condition. Use `custom_exit()`.
🚫 **Don't use** for static ROI. Use `minimal_roi`.
<!-- Place this tag where you want the button to render. -->
<aclass="github-button"href="https://github.com/freqtrade/freqtrade"data-icon="octicon-star"data-size="large"aria-label="Star freqtrade/freqtrade on GitHub">Star</a>
<aclass="github-button"href="https://github.com/freqtrade/freqtrade/fork"data-icon="octicon-repo-forked"data-size="large"aria-label="Fork freqtrade/freqtrade on GitHub">Fork</a>
<aclass="github-button"href="https://github.com/freqtrade/freqtrade/archive/stable.zip"data-icon="octicon-cloud-download"data-size="large"aria-label="Download freqtrade/freqtrade on GitHub">Download</a>
@@ -31,7 +30,6 @@ Freqtrade is a free and open source crypto trading bot written in Python. It is
- Optimize: Find the best parameters for your strategy using hyperoptimization which employs machine learning methods. You can optimize buy, sell, take profit (ROI), stop-loss and trailing stop-loss parameters for your strategy.
- Select markets: Create your static list or use an automatic one based on top traded volumes and/or prices (not available during backtesting). You can also explicitly blacklist markets you don't want to trade.
- Run: Test your strategy with simulated money (Dry-Run mode) or deploy it with real money (Live-Trade mode).
- Run using Edge (optional module): The concept is to find the best historical [trade expectancy](edge.md#expectancy) by markets based on variation of the stop-loss and then allow/reject markets to trade. The sizing of the trade is based on a risk of a percentage of your capital.
- Control/Monitor: Use Telegram or a WebUI (start/stop the bot, show profit/loss, daily summary, current open trades results, etc.).
- Analyze: Further analysis can be performed on either Backtesting data or Freqtrade trading history (SQL database), including automated standard plots, and methods to load the data into [interactive environments](data-analysis.md).
@@ -41,6 +39,7 @@ Please read the [exchange specific notes](exchanges.md) to learn about eventual,
- [X] [Binance](https://www.binance.com/)
- [X] [BingX](https://bingx.com/invite/0EM9RX)
- [X] [Bitget](https://www.bitget.com/)
- [X] [Bitmart](https://bitmart.com/)
- [X] [Bybit](https://bybit.com/)
- [X] [Gate.io](https://www.gate.io/ref/6266643)
@@ -54,6 +53,7 @@ Please read the [exchange specific notes](exchanges.md) to learn about eventual,
### Supported Futures Exchanges (experimental)
- [X] [Binance](https://www.binance.com/)
- [X] [Bitget](https://www.bitget.com/)
- [X] [Bybit](https://bybit.com/)
- [X] [Gate.io](https://www.gate.io/ref/6266643)
- [X] [Hyperliquid](https://hyperliquid.xyz/) (A decentralized exchange, or DEX)
@@ -88,7 +88,7 @@ To run this bot we recommend you a linux cloud instance with a minimum of:
@@ -24,7 +24,7 @@ The easiest way to install and run Freqtrade is to clone the bot Github reposito
The `stable` branch contains the code of the last release (done usually once per month on an approximately one week old snapshot of the `develop` branch to prevent packaging bugs, so potentially it's more stable).
!!! Note
Python3.10 or higher and the corresponding `pip` are assumed to be available. The install-script will warn you and stop if that's not the case. `git` is also needed to clone the Freqtrade repository.
Either [uv](https://docs.astral.sh/uv/), or Python3.11 or higher and the corresponding `pip` are assumed to be available. The install-script will warn you and stop if that's not the case. `git` is also needed to clone the Freqtrade repository.
Also, python headers (`python<yourversion>-dev` / `python<yourversion>-devel`) must be available for the installation to complete successfully.
!!! Warning "Up-to-date clock"
@@ -42,11 +42,10 @@ These requirements apply to both [Script Installation](#script-installation) and
@@ -17,7 +17,7 @@ If you already have an existing strategy, please read the [strategy migration gu
## Shorting
Shorting is not possible when trading with [`trading_mode`](#leverage-trading-modes) set to `spot`. To short trade, `trading_mode` must be set to `margin`(currently unavailable) or [`futures`](#futures), with [`margin_mode`](#margin-mode) set to `cross`(currently unavailable) or [`isolated`](#isolated-margin-mode)
Shorting is not possible when trading with [`trading_mode`](#leverage-trading-modes) set to `spot`. To short trade, `trading_mode` must be set to `margin`(currently unavailable) or [`futures`](#futures), with [`margin_mode`](#margin-mode) set to [`cross`](#cross-margin-mode) or [`isolated`](#isolated-margin-mode)
For a strategy to short, the strategy class must set the class variable `can_short = True`
@@ -72,7 +72,7 @@ A futures pair will therefore have the naming of `base/quote:settle` (e.g. `ETH/
On top of `trading_mode` - you will also have to configure your `margin_mode`.
While freqtrade currently only supports one margin mode, this will change, and by configuring it now you're all set for future updates.
The possible values are: `isolated`, or `cross`(*currently unavailable*).
The possible values are: `isolated`, or `cross`.
#### Isolated margin mode
@@ -92,6 +92,11 @@ One account is used to share collateral between markets (trading pairs). Margin
Please read the [exchange specific notes](exchanges.md) for exchanges that support this mode and how they differ.
!!! Warning "Increased risk of liquidation"
Cross margin mode increases the risk of full account liquidation, as all trades share the same collateral.
A loss on one trade can affect the liquidation price of other trades.
Also, cross-position influence may not be fully simulated in dry-run or backtesting mode.
## Set leverage to use
Different strategies and risk profiles will require different levels of leverage.
**These are not biased and can safely be ignored.**
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
