CTYPTO/freqtrade
1
0
Fork 0
mirror of https://github.com/freqtrade/freqtrade.git synced 2025-12-02 01:53:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'develop' into dependabot/pip/develop/pydantic-2.1.1

Browse Source
This commit is contained in:
Matthias
2023-08-18 10:18:53 +02:00
parent 33d3c4f7d5 d4bc736eb1
commit ee11dae82a
85 changed files with 1700 additions and 1139 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.github/workflows/ci.yml vendored
View File

@@ -461,7 +461,7 @@ jobs:
python setup.py sdist bdist_wheel
- name: Publish to PyPI (Test)
uses: pypa/gh-action-pypi-publish@v1.8.8
uses: pypa/gh-action-pypi-publish@v1.8.10
if: (github.event_name == 'release')
with:
user: __token__
@@ -469,7 +469,7 @@ jobs:
repository_url: https://test.pypi.org/legacy/
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@v1.8.8
uses: pypa/gh-action-pypi-publish@v1.8.10
if: (github.event_name == 'release')
with:
user: __token__

2
.pre-commit-config.yaml
View File

@@ -8,7 +8,7 @@ repos:
# stages: [push]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: "v1.3.0"
rev: "v1.5.0"
hooks:
- id: mypy
exclude: build_helpers

5
build_helpers/install_ta-lib.sh
View File

@@ -8,8 +8,9 @@ if [ -n "$2" ] || [ ! -f "${INSTALL_LOC}/lib/libta_lib.a" ]; then
tar zxvf ta-lib-0.4.0-src.tar.gz
cd ta-lib \
&& sed -i.bak "s|0.00000001|0.000000000000000001 |g" src/ta_func/ta_utility.h \
&& curl 'https://raw.githubusercontent.com/gcc-mirror/gcc/master/config.guess' -o config.guess \
&& curl 'https://raw.githubusercontent.com/gcc-mirror/gcc/master/config.sub' -o config.sub \
&& echo "Downloading gcc config.guess and config.sub" \
&& curl -s 'https://raw.githubusercontent.com/gcc-mirror/gcc/master/config.guess' -o config.guess \
&& curl -s 'https://raw.githubusercontent.com/gcc-mirror/gcc/master/config.sub' -o config.sub \
&& ./configure --prefix=${INSTALL_LOC}/ \
&& make
if [ $? -ne 0 ]; then

1
config_examples/config_full.example.json
View File

@@ -89,7 +89,6 @@
],
"exchange": {
"name": "binance",
"sandbox": false,
"key": "your_exchange_key",
"secret": "your_exchange_secret",
"password": "",

BIN
docs/assets/pycharm_debug.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 48 KiB

2
docs/bot-usage.md
View File

@@ -3,7 +3,7 @@
This page explains the different parameters of the bot and how to run it.
!!! Note
If you've used `setup.sh`, don't forget to activate your virtual environment (`source .env/bin/activate`) before running freqtrade commands.
If you've used `setup.sh`, don't forget to activate your virtual environment (`source .venv/bin/activate`) before running freqtrade commands.
!!! Warning "Up-to-date clock"
The clock on the system running the bot must be accurate, synchronized to a NTP server frequently enough to avoid problems with communication to the exchanges.

1
docs/configuration.md
View File

@@ -188,7 +188,6 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `max_entry_position_adjustment` | Maximum additional order(s) for each open trade on top of the first entry Order. Set it to `-1` for unlimited additional orders. [More information here](strategy-callbacks.md#adjust-trade-position). <br> [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `-1`.*<br> **Datatype:** Positive Integer or -1
| | **Exchange**
| `exchange.name` | **Required.** Name of the exchange class to use. [List below](#user-content-what-values-for-exchangename). <br> **Datatype:** String
| `exchange.sandbox` | Use the 'sandbox' version of the exchange, where the exchange provides a sandbox for risk-free integration. See [here](sandbox-testing.md) in more details.<br> **Datatype:** Boolean
| `exchange.key` | API key to use for the exchange. Only required when you are in production mode.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
| `exchange.secret` | API secret to use for the exchange. Only required when you are in production mode.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
| `exchange.password` | API password to use for the exchange. Only required when you are in production mode and for exchanges that use password for API requests.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String

2
docs/data-analysis.md
View File

@@ -27,7 +27,7 @@ For this to work, first activate your virtual environment and run the following
``` bash
# Activate virtual environment
source .env/bin/activate
source .venv/bin/activate
pip install ipykernel
ipython kernel install --user --name=freqtrade

15
docs/developer.md
View File

@@ -77,7 +77,7 @@ def test_method_to_test(caplog):
### Debug configuration
To debug freqtrade, we recommend VSCode with the following launch configuration (located in `.vscode/launch.json`).
To debug freqtrade, we recommend VSCode (with the Python extension) with the following launch configuration (located in `.vscode/launch.json`).
Details will obviously vary between setups - but this should work to get you started.
``` json
@@ -102,6 +102,19 @@ This method can also be used to debug a strategy, by setting the breakpoints wit
A similar setup can also be taken for Pycharm - using `freqtrade` as module name, and setting the command line arguments as "parameters".
??? Tip "Correct venv usage"
When using a virtual environment (which you should), make sure that your Editor is using the correct virtual environment to avoid problems or "unknown import" errors.
#### Vscode
You can select the correct environment in VSCode with the command "Python: Select Interpreter" - which will show you environments the extension detected.
If your environment has not been detected, you can also pick a path manually.
#### Pycharm
In pycharm, you can select the appropriate Environment in the "Run/Debug Configurations" window.
![Pycharm debug configuration](assets/pycharm_debug.png)
!!! Note "Startup directory"
This assumes that you have the repository checked out, and the editor is started at the repository root level (so setup.py is at the top level of your repository).

31
docs/docker_quickstart.md
View File

@@ -14,6 +14,9 @@ Start by downloading and installing Docker / Docker Desktop for your platform:
Freqtrade documentation assumes the use of Docker desktop (or the docker compose plugin).
While the docker-compose standalone installation still works, it will require changing all `docker compose` commands from `docker compose` to `docker-compose` to work (e.g. `docker compose up -d` will become `docker-compose up -d`).
??? Warning "Docker on windows"
If you just installed docker on a windows system, make sure to reboot your system, otherwise you might encounter unexplainable Problems related to network connectivity to docker containers.
## Freqtrade with docker
Freqtrade provides an official Docker image on [Dockerhub](https://hub.docker.com/r/freqtradeorg/freqtrade/), as well as a [docker compose file](https://github.com/freqtrade/freqtrade/blob/stable/docker-compose.yml) ready for usage.
@@ -78,7 +81,7 @@ If you've selected to enable FreqUI in the `new-config` step, you will have freq
You can now access the UI by typing localhost:8080 in your browser.
??? Note "UI Access on a remote servers"
??? Note "UI Access on a remote server"
If you're running on a VPS, you should consider using either a ssh tunnel, or setup a VPN (openVPN, wireguard) to connect to your bot.
This will ensure that freqUI is not directly exposed to the internet, which is not recommended for security reasons (freqUI does not support https out of the box).
Setup of these tools is not part of this tutorial, however many good tutorials can be found on the internet.
@@ -128,7 +131,7 @@ All freqtrade arguments will be available by running `docker compose run --rm fr
!!! Note "`docker compose run --rm`"
Including `--rm` will remove the container after completion, and is highly recommended for all modes except trading mode (running with `freqtrade trade` command).
??? Note "Using docker without docker"
??? Note "Using docker without docker compose"
"`docker compose run --rm`" will require a compose file to be provided.
Some freqtrade commands that don't require authentication such as `list-pairs` can be run with "`docker run --rm`" instead.
For example `docker run --rm freqtradeorg/freqtrade:stable list-pairs --exchange binance --quote BTC --print-json`.
@@ -172,7 +175,7 @@ You can then run `docker compose build --pull` to build the docker image, and ru
### Plotting with docker
Commands `freqtrade plot-profit` and `freqtrade plot-dataframe` ([Documentation](plotting.md)) are available by changing the image to `*_plot` in your docker-compose.yml file.
Commands `freqtrade plot-profit` and `freqtrade plot-dataframe` ([Documentation](plotting.md)) are available by changing the image to `*_plot` in your `docker-compose.yml` file.
You can then use these commands as follows:
``` bash
@@ -203,16 +206,20 @@ docker compose -f docker/docker-compose-jupyter.yml build --no-cache
### Docker on Windows
* Error: `"Timestamp for this request is outside of the recvWindow."`
* The market api requests require a synchronized clock but the time in the docker container shifts a bit over time into the past.
To fix this issue temporarily you need to run `wsl --shutdown` and restart docker again (a popup on windows 10 will ask you to do so).
A permanent solution is either to host the docker container on a linux host or restart the wsl from time to time with the scheduler.
* Error: `"Timestamp for this request is outside of the recvWindow."`
The market api requests require a synchronized clock but the time in the docker container shifts a bit over time into the past.
To fix this issue temporarily you need to run `wsl --shutdown` and restart docker again (a popup on windows 10 will ask you to do so).
A permanent solution is either to host the docker container on a linux host or restart the wsl from time to time with the scheduler.
``` bash
taskkill /IM "Docker Desktop.exe" /F
wsl --shutdown
start "" "C:\Program Files\Docker\Docker\Docker Desktop.exe"
```
``` bash
taskkill /IM "Docker Desktop.exe" /F
wsl --shutdown
start "" "C:\Program Files\Docker\Docker\Docker Desktop.exe"
```
* Cannot connect to the API (Windows)
If you're on windows and just installed Docker (desktop), make sure to reboot your System. Docker can have problems with network connectivity without a restart.
You should obviously also make sure to have your [settings](#accessing-the-ui) accordingly.
!!! Warning
Due to the above, we do not recommend the usage of docker on windows for production setups, but only for experimentation, datadownload and backtesting.

12
docs/faq.md
View File

@@ -20,7 +20,7 @@ Futures trading is supported for selected exchanges. Please refer to the [docume
* When you work with your strategy & hyperopt file you should use a proper code editor like VSCode or PyCharm. A good code editor will provide syntax highlighting as well as line numbers, making it easy to find syntax errors (most likely pointed out by Freqtrade during startup).
## Freqtrade common issues
## Freqtrade common questions
### Can freqtrade open multiple positions on the same pair in parallel?
@@ -36,7 +36,7 @@ Running the bot with `freqtrade trade --config config.json` shows the output `fr
This could be caused by the following reasons:
* The virtual environment is not active.
* Run `source .env/bin/activate` to activate the virtual environment.
* Run `source .venv/bin/activate` to activate the virtual environment.
* The installation did not complete successfully.
* Please check the [Installation documentation](installation.md).
@@ -78,6 +78,14 @@ Where possible (e.g. on binance), the use of the exchange's dedicated fee curren
On binance, it's sufficient to have BNB in your account, and have "Pay fees in BNB" enabled in your profile. Your BNB balance will slowly decline (as it's used to pay fees) - but you'll no longer encounter dust (Freqtrade will include the fees in the profit calculations).
Other exchanges don't offer such possibilities, where it's simply something you'll have to accept or move to a different exchange.
### I deposited more funds to the exchange, but my bot doesn't recognize this
Freqtrade will update the exchange balance when necessary (Before placing an order).
RPC calls (Telegram's `/balance`, API calls to `/balance`) can trigger an update at max. once per hour.
If `adjust_trade_position` is enabled (and the bot has open trades eligible for position adjustments) - then the wallets will be refreshed once per hour.
To force an immediate update, you can use `/reload_config` - which will restart the bot.
### I want to use incomplete candles
Freqtrade will not provide incomplete candles to strategies. Using incomplete candles will lead to repainting and consequently to strategies with "ghost" buys, which are impossible to both backtest, and verify after they happened.

12
docs/freqai-parameter-table.md
View File

@@ -100,12 +100,12 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
#### trainer_kwargs
| Parameter | Description |
|------------|-------------|
| | **Model training parameters within the `freqai.model_training_parameters.model_kwargs` sub dictionary**
| `max_iters` | The number of training iterations to run. iteration here refers to the number of times we call self.optimizer.step(). used to calculate n_epochs. <br> **Datatype:** int. <br> Default: `100`.
| `batch_size` | The size of the batches to use during training.. <br> **Datatype:** int. <br> Default: `64`.
| `max_n_eval_batches` | The maximum number batches to use for evaluation.. <br> **Datatype:** int, optional. <br> Default: `None`.
| Parameter | Description |
|--------------|-------------|
| | **Model training parameters within the `freqai.model_training_parameters.model_kwargs` sub dictionary**
| `n_epochs` | The `n_epochs` parameter is a crucial setting in the PyTorch training loop that determines the number of times the entire training dataset will be used to update the model's parameters. An epoch represents one full pass through the entire training dataset. Overrides `n_steps`. Either `n_epochs` or `n_steps` must be set. <br><br> **Datatype:** int. optional. <br> Default: `10`.
| `n_steps` | An alternative way of setting `n_epochs` - the number of training iterations to run. Iteration here refer to the number of times we call `optimizer.step()`. Ignored if `n_epochs` is set. A simplified version of the function: <br><br> n_epochs = n_steps / (n_obs / batch_size) <br><br> The motivation here is that `n_steps` is easier to optimize and keep stable across different n_obs - the number of data points. <br> <br> **Datatype:** int. optional. <br> Default: `None`.
| `batch_size` | The size of the batches to use during training. <br><br> **Datatype:** int. <br> Default: `64`.
### Additional parameters

6
docs/freqai-reinforcement-learning.md
View File

@@ -20,7 +20,7 @@ With the current framework, we aim to expose the training environment via the co
We envision the majority of users focusing their effort on creative design of the `calculate_reward()` function [details here](#creating-a-custom-reward-function), while leaving the rest of the environment untouched. Other users may not touch the environment at all, and they will only play with the configuration settings and the powerful feature engineering that already exists in FreqAI. Meanwhile, we enable advanced users to create their own model classes entirely.
The framework is built on stable_baselines3 (torch) and OpenAI gym for the base environment class. But generally speaking, the model class is well isolated. Thus, the addition of competing libraries can be easily integrated into the existing framework. For the environment, it is inheriting from `gym.env` which means that it is necessary to write an entirely new environment in order to switch to a different library.
The framework is built on stable_baselines3 (torch) and OpenAI gym for the base environment class. But generally speaking, the model class is well isolated. Thus, the addition of competing libraries can be easily integrated into the existing framework. For the environment, it is inheriting from `gym.Env` which means that it is necessary to write an entirely new environment in order to switch to a different library.
### Important considerations
@@ -173,7 +173,7 @@ class MyCoolRLModel(ReinforcementLearner):
"""
class MyRLEnv(Base5ActionRLEnv):
"""
User made custom environment. This class inherits from BaseEnvironment and gym.env.
User made custom environment. This class inherits from BaseEnvironment and gym.Env.
Users can override any functions from those parent classes. Here is an example
of a user customized `calculate_reward()` function.
@@ -254,7 +254,7 @@ FreqAI also provides a built in episodic summary logger called `self.tensorboard
```python
class MyRLEnv(Base5ActionRLEnv):
"""
User made custom environment. This class inherits from BaseEnvironment and gym.env.
User made custom environment. This class inherits from BaseEnvironment and gym.Env.
Users can override any functions from those parent classes. Here is an example
of a user customized `calculate_reward()` function.
"""

2
docs/hyperopt.md
View File

@@ -31,7 +31,7 @@ The docker-image includes hyperopt dependencies, no further action needed.
### Easy installation script (setup.sh) / Manual installation
```bash
source .env/bin/activate
source .venv/bin/activate
pip install -r requirements-hyperopt.txt
```

18
docs/installation.md
View File

@@ -143,11 +143,11 @@ If you are on Debian, Ubuntu or MacOS, freqtrade provides the script to install
### Activate your virtual environment
Each time you open a new terminal, you must run `source .env/bin/activate` to activate your virtual environment.
Each time you open a new terminal, you must run `source .venv/bin/activate` to activate your virtual environment.
```bash
# then activate your .env
source ./.env/bin/activate
# activate virtual environment
source ./.venv/bin/activate
```
### Congratulations
@@ -172,7 +172,7 @@ With this option, the script will install the bot and most dependencies:
You will need to have git and python3.8+ installed beforehand for this to work.
* Mandatory software as: `ta-lib`
* Setup your virtualenv under `.env/`
* Setup your virtualenv under `.venv/`
This option is a combination of installation tasks and `--reset`
@@ -225,11 +225,11 @@ rm -rf ./ta-lib*
You will run freqtrade in separated `virtual environment`
```bash
# create virtualenv in directory /freqtrade/.env
python3 -m venv .env
# create virtualenv in directory /freqtrade/.venv
python3 -m venv .venv
# run virtualenv
source .env/bin/activate
source .venv/bin/activate
```
#### Install python dependencies
@@ -411,8 +411,8 @@ If you used (1)`Script` or (2)`Manual` installation, you need to run the bot in
# if:
bash: freqtrade: command not found
# then activate your .env
source ./.env/bin/activate
# then activate your virtual environment
source ./.venv/bin/activate
```
### MacOS installation error

5
docs/lookahead-analysis.md
View File

@@ -21,7 +21,10 @@ It also supports the lookahead-analysis of freqai strategies.
- `--cache` is forced to "none".
- `--max-open-trades` is forced to be at least equal to the number of pairs.
- `--dry-run-wallet` is forced to be basically infinite.
- `--dry-run-wallet` is forced to be basically infinite (1 billion).
- `--stake-amount` is forced to be a static 10000 (10k).
Those are set to avoid users accidentally generating false positives.
## Lookahead-analysis command reference

121
docs/sandbox-testing.md
View File

@@ -1,121 +0,0 @@
# Sandbox API testing
Some exchanges provide sandboxes or testbeds for risk-free testing, while running the bot against a real exchange.
With some configuration, freqtrade (in combination with ccxt) provides access to these.
This document is an overview to configure Freqtrade to be used with sandboxes.
This can be useful to developers and trader alike.
!!! Warning
Sandboxes usually have very low volume, and either a very wide spread, or no orders available at all.
Therefore, sandboxes will usually not do a good job of showing you how a strategy would work in real trading.
## Exchanges known to have a sandbox / testnet
* [binance](https://testnet.binance.vision/)
* [coinbasepro](https://public.sandbox.pro.coinbase.com)
* [gemini](https://exchange.sandbox.gemini.com/)
* [huobipro](https://www.testnet.huobi.pro/)
* [kucoin](https://sandbox.kucoin.com/)
* [phemex](https://testnet.phemex.com/)
!!! Note
We did not test correct functioning of all of the above testnets. Please report your experiences with each sandbox.
---
## Configure a Sandbox account
When testing your API connectivity, make sure to use the appropriate sandbox / testnet URL.
In general, you should follow these steps to enable an exchange's sandbox:
* Figure out if an exchange has a sandbox (most likely by using google or the exchange's support documents)
* Create a sandbox account (often the sandbox-account requires separate registration)
* [Add some test assets to account](#add-test-funds)
* Create API keys
### Add test funds
Usually, sandbox exchanges allow depositing funds directly via web-interface.
You should make sure to have a realistic amount of funds available to your test-account, so results are representable of your real account funds.
!!! Warning
Test exchanges will **NEVER** require your real credit card or banking details!
## Configure freqtrade to use a exchange's sandbox
### Sandbox URLs
Freqtrade makes use of CCXT which in turn provides a list of URLs to Freqtrade.
These include `['test']` and `['api']`.
* `[Test]` if available will point to an Exchanges sandbox.
* `[Api]` normally used, and resolves to live API target on the exchange.
To make use of sandbox / test add "sandbox": true, to your config.json
```json
"exchange": {
"name": "coinbasepro",
"sandbox": true,
"key": "5wowfxemogxeowo;heiohgmd",
"secret": "/ZMH1P62rCVmwefewrgcewX8nh4gob+lywxfwfxwwfxwfNsH1ySgvWCUR/w==",
"password": "1bkjfkhfhfu6sr",
"outdated_offset": 5
"pair_whitelist": [
"BTC/USD"
]
},
"datadir": "user_data/data/coinbasepro_sandbox"
```
Also the following information:
* api-key (created for the sandbox webpage)
* api-secret (noted earlier)
* password (the passphrase - noted earlier)
!!! Tip "Different data directory"
We also recommend to set `datadir` to something identifying downloaded data as sandbox data, to avoid having sandbox data mixed with data from the real exchange.
This can be done by adding the `"datadir"` key to the configuration.
Now, whenever you use this configuration, your data directory will be set to this directory.
---
## You should now be ready to test your sandbox
Ensure Freqtrade logs show the sandbox URL, and trades made are shown in sandbox. Also make sure to select a pair which shows at least some decent value (which very often is BTC/<somestablecoin>).
## Common problems with sandbox exchanges
Sandbox exchange instances often have very low volume, which can cause some problems which usually are not seen on a real exchange instance.
### Old Candles problem
Since Sandboxes often have low volume, candles can be quite old and show no volume.
To disable the error "Outdated history for pair ...", best increase the parameter `"outdated_offset"` to a number that seems realistic for the sandbox you're using.
### Unfilled orders
Sandboxes often have very low volumes - which means that many trades can go unfilled, or can go unfilled for a very long time.
To mitigate this, you can try to match the first order on the opposite orderbook side using the following configuration:
``` jsonc
"order_types": {
"entry": "limit",
"exit": "limit"
// ...
},
"entry_pricing": {
"price_side": "other",
// ...
},
"exit_pricing":{
"price_side": "other",
// ...
},
```
The configuration is similar to the suggested configuration for market orders - however by using limit-orders you can avoid moving the price too much, and you can set the worst price you might get.

2
docs/utils.md
View File

@@ -967,7 +967,7 @@ Print trades with id 2 and 3 as json
freqtrade show-trades --db-url sqlite:///tradesv3.sqlite --trade-ids 2 3 --print-json
```
### Strategy-Updater
## Strategy-Updater
Updates listed strategies or all strategies within the strategies folder to be v3 compliant.
If the command runs without --strategy-list then all strategies inside the strategies folder will be converted.

4
docs/windows_installation.md
View File

@@ -31,8 +31,8 @@ Other versions must be downloaded from the above link.
``` powershell
cd \path\freqtrade
python -m venv .env
.env\Scripts\activate.ps1
python -m venv .venv
.venv\Scripts\activate.ps1
# optionally install ta-lib from wheel
# Eventually adjust the below filename to match the downloaded wheel
pip install --find-links build_helpers\ TA-Lib -U

2
freqtrade/commands/build_config_commands.py
View File

@@ -10,7 +10,7 @@ from freqtrade.configuration.directory_operations import chown_user_directory
from freqtrade.constants import UNLIMITED_STAKE_AMOUNT
from freqtrade.exceptions import OperationalException
from freqtrade.exchange import MAP_EXCHANGE_CHILDCLASS, available_exchanges
from freqtrade.misc import render_template
from freqtrade.util import render_template
logger = logging.getLogger(__name__)

7
freqtrade/commands/deploy_commands.py
View File

@@ -10,7 +10,7 @@ from freqtrade.configuration.directory_operations import copy_sample_files, crea
from freqtrade.constants import USERPATH_STRATEGIES
from freqtrade.enums import RunMode
from freqtrade.exceptions import OperationalException
from freqtrade.misc import render_template, render_template_with_fallback
from freqtrade.util import render_template, render_template_with_fallback
logger = logging.getLogger(__name__)
@@ -35,6 +35,10 @@ def deploy_new_strategy(strategy_name: str, strategy_path: Path, subtemplate: st
Deploy new strategy from template to strategy_path
"""
fallback = 'full'
attributes = render_template_with_fallback(
templatefile=f"strategy_subtemplates/strategy_attributes_{subtemplate}.j2",
templatefallbackfile=f"strategy_subtemplates/strategy_attributes_{fallback}.j2",
)
indicators = render_template_with_fallback(
templatefile=f"strategy_subtemplates/indicators_{subtemplate}.j2",
templatefallbackfile=f"strategy_subtemplates/indicators_{fallback}.j2",
@@ -58,6 +62,7 @@ def deploy_new_strategy(strategy_name: str, strategy_path: Path, subtemplate: st
strategy_text = render_template(templatefile='base_strategy.py.j2',
arguments={"strategy": strategy_name,
"attributes": attributes,
"indicators": indicators,
"buy_trend": buy_trend,
"sell_trend": sell_trend,

5
freqtrade/commands/webserver_commands.py
View File

@@ -7,9 +7,10 @@ def start_webserver(args: Dict[str, Any]) -> None:
"""
Main entry point for webserver mode
"""
from freqtrade.configuration import Configuration
from freqtrade.configuration import setup_utils_configuration
from freqtrade.rpc.api_server import ApiServer
# Initialize configuration
config = Configuration(args, RunMode.WEBSERVER).get_config()
config = setup_utils_configuration(args, RunMode.WEBSERVER)
ApiServer(config, standalone=True)

2
freqtrade/configuration/config_validation.py
View File

@@ -51,6 +51,8 @@ def validate_config_schema(conf: Dict[str, Any], preliminary: bool = False) -> D
conf_schema['required'] = constants.SCHEMA_BACKTEST_REQUIRED
else:
conf_schema['required'] = constants.SCHEMA_BACKTEST_REQUIRED_FINAL
elif conf.get('runmode', RunMode.OTHER) == RunMode.WEBSERVER:
conf_schema['required'] = constants.SCHEMA_MINIMAL_WEBSERVER
else:
conf_schema['required'] = constants.SCHEMA_MINIMAL_REQUIRED
try:

4
freqtrade/constants.py
View File

@@ -461,7 +461,6 @@ CONF_SCHEMA = {
'type': 'object',
'properties': {
'name': {'type': 'string'},
'sandbox': {'type': 'boolean', 'default': False},
'key': {'type': 'string', 'default': ''},
'secret': {'type': 'string', 'default': ''},
'password': {'type': 'string', 'default': ''},
@@ -668,6 +667,9 @@ SCHEMA_MINIMAL_REQUIRED = [
'dataformat_ohlcv',
'dataformat_trades',
]
SCHEMA_MINIMAL_WEBSERVER = SCHEMA_MINIMAL_REQUIRED + [
'api_server',
]
CANCEL_REASON = {
"TIMEOUT": "cancelled due to timeout",

33
freqtrade/data/btanalysis.py
View File

@@ -12,7 +12,7 @@ import pandas as pd
from freqtrade.constants import LAST_BT_RESULT_FN, IntOrInf
from freqtrade.exceptions import OperationalException
from freqtrade.misc import json_load
from freqtrade.misc import file_dump_json, json_load
from freqtrade.optimize.backtest_caching import get_backtest_metadata_filename
from freqtrade.persistence import LocalTrade, Trade, init_db
from freqtrade.types import BacktestHistoryEntryType, BacktestResultType
@@ -175,6 +175,21 @@ def _get_backtest_files(dirname: Path) -> List[Path]:
return list(reversed(sorted(dirname.glob('backtest-result-*-[0-9][0-9].json'))))
def get_backtest_result(filename: Path) -> List[BacktestHistoryEntryType]:
"""
Get backtest result read from metadata file
"""
return [
{
'filename': filename.stem,
'strategy': s,
'notes': v.get('notes', ''),
'run_id': v['run_id'],
'backtest_start_time': v['backtest_start_time'],
} for s, v in load_backtest_metadata(filename).items()
]
def get_backtest_resultlist(dirname: Path) -> List[BacktestHistoryEntryType]:
"""
Get list of backtest results read from metadata files
@@ -184,6 +199,7 @@ def get_backtest_resultlist(dirname: Path) -> List[BacktestHistoryEntryType]:
'filename': filename.stem,
'strategy': s,
'run_id': v['run_id'],
'notes': v.get('notes', ''),
'backtest_start_time': v['backtest_start_time'],
}
for filename in _get_backtest_files(dirname)
@@ -203,6 +219,21 @@ def delete_backtest_result(file_abs: Path):
file_abs_meta.unlink()
def update_backtest_metadata(filename: Path, strategy: str, content: Dict[str, Any]):
"""
Updates backtest metadata file with new content.
:raises: ValueError if metadata file does not exist, or strategy is not in this file.
"""
metadata = load_backtest_metadata(filename)
if not metadata:
raise ValueError("File does not exist.")
if strategy not in metadata:
raise ValueError("Strategy not in metadata.")
metadata[strategy].update(content)
# Write data again.
file_dump_json(get_backtest_metadata_filename(filename), metadata)
def find_existing_backtest_stats(dirname: Union[Path, str], run_ids: Dict[str, str],
min_backtest_date: Optional[datetime] = None) -> Dict[str, Any]:
"""

23
freqtrade/data/dataprovider.py
View File

@@ -17,7 +17,7 @@ from freqtrade.constants import (FULL_DATAFRAME_THRESHOLD, Config, ListPairsWith
from freqtrade.data.history import load_pair_history
from freqtrade.enums import CandleType, RPCMessageType, RunMode
from freqtrade.exceptions import ExchangeError, OperationalException
from freqtrade.exchange import Exchange, timeframe_to_seconds
from freqtrade.exchange import Exchange, timeframe_to_prev_date, timeframe_to_seconds
from freqtrade.exchange.types import OrderBook
from freqtrade.misc import append_candles_to_dataframe
from freqtrade.rpc import RPCManager
@@ -46,6 +46,8 @@ class DataProvider:
self.__rpc = rpc
self.__cached_pairs: Dict[PairWithTimeframe, Tuple[DataFrame, datetime]] = {}
self.__slice_index: Optional[int] = None
self.__slice_date: Optional[datetime] = None
self.__cached_pairs_backtesting: Dict[PairWithTimeframe, DataFrame] = {}
self.__producer_pairs_df: Dict[str,
Dict[PairWithTimeframe, Tuple[DataFrame, datetime]]] = {}
@@ -64,10 +66,19 @@ class DataProvider:
def _set_dataframe_max_index(self, limit_index: int):
"""
Limit analyzed dataframe to max specified index.
Only relevant in backtesting.
:param limit_index: dataframe index.
"""
self.__slice_index = limit_index
def _set_dataframe_max_date(self, limit_date: datetime):
"""
Limit infomrative dataframe to max specified index.
Only relevant in backtesting.
:param limit_date: "current date"
"""
self.__slice_date = limit_date
def _set_cached_df(
self,
pair: str,
@@ -284,7 +295,7 @@ class DataProvider:
def historic_ohlcv(
self,
pair: str,
timeframe: Optional[str] = None,
timeframe: str,
candle_type: str = ''
) -> DataFrame:
"""
@@ -307,7 +318,7 @@ class DataProvider:
timerange.subtract_start(tf_seconds * startup_candles)
self.__cached_pairs_backtesting[saved_pair] = load_pair_history(
pair=pair,
timeframe=timeframe or self._config['timeframe'],
timeframe=timeframe,
datadir=self._config['datadir'],
timerange=timerange,
data_format=self._config['dataformat_ohlcv'],
@@ -354,7 +365,13 @@ class DataProvider:
data = self.ohlcv(pair=pair, timeframe=timeframe, candle_type=candle_type)
else:
# Get historical OHLCV data (cached on disk).
timeframe = timeframe or self._config['timeframe']
data = self.historic_ohlcv(pair=pair, timeframe=timeframe, candle_type=candle_type)
# Cut date to timeframe-specific date.
# This is necessary to prevent lookahead bias in callbacks through informative pairs.
if self.__slice_date:
cutoff_date = timeframe_to_prev_date(timeframe, self.__slice_date)
data = data.loc[data['date'] < cutoff_date]
if len(data) == 0:
logger.warning(f"No data found for ({pair}, {timeframe}, {candle_type}).")
return data

918
freqtrade/exchange/binance_leverage_tiers.json
View File

File diff suppressed because it is too large Load Diff

33
freqtrade/exchange/bybit.py
View File

@@ -7,10 +7,10 @@ import ccxt
from freqtrade.constants import BuySell
from freqtrade.enums import MarginMode, PriceType, TradingMode
from freqtrade.enums.candletype import CandleType
from freqtrade.exceptions import DDosProtection, OperationalException, TemporaryError
from freqtrade.exchange import Exchange
from freqtrade.exchange.common import retrier
from freqtrade.exchange.exchange_utils import timeframe_to_msecs
logger = logging.getLogger(__name__)
@@ -27,7 +27,7 @@ class Bybit(Exchange):
"""
_ft_has: Dict = {
"ohlcv_candle_limit": 200,
"ohlcv_candle_limit": 1000,
"ohlcv_has_history": True,
}
_ft_has_futures: Dict = {
@@ -91,28 +91,13 @@ class Bybit(Exchange):
except ccxt.BaseError as e:
raise OperationalException(e) from e
async def _fetch_funding_rate_history(
self,
pair: str,
timeframe: str,
limit: int,
since_ms: Optional[int] = None,
) -> List[List]:
"""
Fetch funding rate history
Necessary workaround until https://github.com/ccxt/ccxt/issues/15990 is fixed.
"""
params = {}
if since_ms:
until = since_ms + (timeframe_to_msecs(timeframe) * self._ft_has['ohlcv_candle_limit'])
params.update({'until': until})
# Funding rate
data = await self._api_async.fetch_funding_rate_history(
pair, since=since_ms,
params=params)
# Convert funding rate to candle pattern
data = [[x['timestamp'], x['fundingRate'], 0, 0, 0, 0] for x in data]
return data
def ohlcv_candle_limit(
self, timeframe: str, candle_type: CandleType, since_ms: Optional[int] = None) -> int:
if candle_type in (CandleType.FUNDING_RATE):
return 200
return super().ohlcv_candle_limit(timeframe, candle_type, since_ms)
def _lev_prep(self, pair: str, leverage: float, side: BuySell, accept_fail: bool = False):
if self.trading_mode != TradingMode.SPOT:

16
freqtrade/exchange/exchange.py
View File

@@ -263,8 +263,6 @@ class Exchange:
except ccxt.BaseError as e:
raise OperationalException(f"Initialization of ccxt failed. Reason: {e}") from e
self.set_sandbox(api, exchange_config, name)
return api
@property
@@ -465,16 +463,6 @@ class Exchange:
return amount_to_contract_precision(amount, self.get_precision_amount(pair),
self.precisionMode, contract_size)
def set_sandbox(self, api: ccxt.Exchange, exchange_config: dict, name: str) -> None:
if exchange_config.get('sandbox'):
if api.urls.get('test'):
api.urls['api'] = api.urls['test']
logger.info("Enabled Sandbox API on %s", name)
else:
logger.warning(
f"No Sandbox URL in CCXT for {name}, exiting. Please check your config.json")
raise OperationalException(f'Exchange {name} does not provide a sandbox api')
def _load_async_markets(self, reload: bool = False) -> None:
try:
if self._api_async:
@@ -580,7 +568,7 @@ class Exchange:
for pair in [f"{curr_1}/{curr_2}", f"{curr_2}/{curr_1}"]:
if pair in self.markets and self.markets[pair].get('active'):
return pair
raise ExchangeError(f"Could not combine {curr_1} and {curr_2} to get a valid pair.")
raise ValueError(f"Could not combine {curr_1} and {curr_2} to get a valid pair.")
def validate_timeframes(self, timeframe: Optional[str]) -> None:
"""
@@ -1876,7 +1864,7 @@ class Exchange:
tick = self.fetch_ticker(comb)
fee_to_quote_rate = safe_value_fallback2(tick, tick, 'last', 'ask')
except ExchangeError:
except (ValueError, ExchangeError):
fee_to_quote_rate = self._config['exchange'].get('unknown_fee_rate', None)
if not fee_to_quote_rate:
return None

2
freqtrade/freqai/data_drawer.py
View File

@@ -375,7 +375,7 @@ class FreqaiDataDrawer:
num_keep = self.freqai_info["purge_old_models"]
if not num_keep:
return
elif type(num_keep) == bool:
elif isinstance(num_keep, bool):
num_keep = 2
model_folders = [x for x in self.full_path.iterdir() if x.is_dir()]

4
freqtrade/freqai/prediction_models/PyTorchMLPClassifier.py
View File

@@ -26,9 +26,9 @@ class PyTorchMLPClassifier(BasePyTorchClassifier):
"model_training_parameters" : {
"learning_rate": 3e-4,
"trainer_kwargs": {
"max_iters": 5000,
"n_steps": 5000,
"batch_size": 64,
"max_n_eval_batches": null,
"n_epochs": null,
},
"model_kwargs": {
"hidden_dim": 512,

4
freqtrade/freqai/prediction_models/PyTorchMLPRegressor.py
View File

@@ -27,9 +27,9 @@ class PyTorchMLPRegressor(BasePyTorchRegressor):
"model_training_parameters" : {
"learning_rate": 3e-4,
"trainer_kwargs": {
"max_iters": 5000,
"n_steps": 5000,
"batch_size": 64,
"max_n_eval_batches": null,
"n_epochs": null,
},
"model_kwargs": {
"hidden_dim": 512,

4
freqtrade/freqai/prediction_models/PyTorchTransformerRegressor.py
View File

@@ -30,9 +30,9 @@ class PyTorchTransformerRegressor(BasePyTorchRegressor):
"model_training_parameters" : {
"learning_rate": 3e-4,
"trainer_kwargs": {
"max_iters": 5000,
"n_steps": 5000,
"batch_size": 64,
"max_n_eval_batches": null
"n_epochs": null
},
"model_kwargs": {
"hidden_dim": 512,

30
freqtrade/freqai/torch/PyTorchDataConvertor.py
View File

@@ -1,5 +1,4 @@
from abc import ABC, abstractmethod
from typing import Optional
import pandas as pd
import torch
@@ -12,14 +11,14 @@ class PyTorchDataConvertor(ABC):
"""
@abstractmethod
def convert_x(self, df: pd.DataFrame, device: Optional[str] = None) -> torch.Tensor:
def convert_x(self, df: pd.DataFrame, device: str) -> torch.Tensor:
"""
:param df: "*_features" dataframe.
:param device: The device to use for training (e.g. 'cpu', 'cuda').
"""
@abstractmethod
def convert_y(self, df: pd.DataFrame, device: Optional[str] = None) -> torch.Tensor:
def convert_y(self, df: pd.DataFrame, device: str) -> torch.Tensor:
"""
:param df: "*_labels" dataframe.
:param device: The device to use for training (e.g. 'cpu', 'cuda').
@@ -33,8 +32,8 @@ class DefaultPyTorchDataConvertor(PyTorchDataConvertor):
def __init__(
self,
target_tensor_type: Optional[torch.dtype] = None,
squeeze_target_tensor: bool = False
target_tensor_type: torch.dtype = torch.float32,
squeeze_target_tensor: bool = False,
):
"""
:param target_tensor_type: type of target tensor, for classification use
@@ -45,23 +44,14 @@ class DefaultPyTorchDataConvertor(PyTorchDataConvertor):
self._target_tensor_type = target_tensor_type
self._squeeze_target_tensor = squeeze_target_tensor
def convert_x(self, df: pd.DataFrame, device: Optional[str] = None) -> torch.Tensor:
x = torch.from_numpy(df.values).float()
if device:
x = x.to(device)
def convert_x(self, df: pd.DataFrame, device: str) -> torch.Tensor:
numpy_arrays = df.values
x = torch.tensor(numpy_arrays, device=device, dtype=torch.float32)
return x
def convert_y(self, df: pd.DataFrame, device: Optional[str] = None) -> torch.Tensor:
y = torch.from_numpy(df.values)
if self._target_tensor_type:
y = y.to(self._target_tensor_type)
def convert_y(self, df: pd.DataFrame, device: str) -> torch.Tensor:
numpy_arrays = df.values
y = torch.tensor(numpy_arrays, device=device, dtype=self._target_tensor_type)
if self._squeeze_target_tensor:
y = y.squeeze()
if device:
y = y.to(device)
return y

83
freqtrade/freqai/torch/PyTorchModelTrainer.py
View File

@@ -1,5 +1,4 @@
import logging
import math
from pathlib import Path
from typing import Any, Dict, List, Optional
@@ -40,23 +39,27 @@ class PyTorchModelTrainer(PyTorchTrainerInterface):
state_dict and model_meta_data saved by self.save() method.
:param model_meta_data: Additional metadata about the model (optional).
:param data_convertor: convertor from pd.DataFrame to torch.tensor.
:param max_iters: The number of training iterations to run.
iteration here refers to the number of times we call
self.optimizer.step(). used to calculate n_epochs.
:param n_steps: used to calculate n_epochs. The number of training iterations to run.
iteration here refers to the number of times optimizer.step() is called.
ignored if n_epochs is set.
:param n_epochs: The maximum number batches to use for evaluation.
:param batch_size: The size of the batches to use during training.
:param max_n_eval_batches: The maximum number batches to use for evaluation.
"""
self.model = model
self.optimizer = optimizer
self.criterion = criterion
self.model_meta_data = model_meta_data
self.device = device
self.max_iters: int = kwargs.get("max_iters", 100)
self.n_epochs: Optional[int] = kwargs.get("n_epochs", 10)
self.n_steps: Optional[int] = kwargs.get("n_steps", None)
if self.n_steps is None and not self.n_epochs:
raise Exception("Either `n_steps` or `n_epochs` should be set.")
self.batch_size: int = kwargs.get("batch_size", 64)
self.max_n_eval_batches: Optional[int] = kwargs.get("max_n_eval_batches", None)
self.data_convertor = data_convertor
self.window_size: int = window_size
self.tb_logger = tb_logger
self.test_batch_counter = 0
def fit(self, data_dictionary: Dict[str, pd.DataFrame], splits: List[str]):
"""
@@ -72,55 +75,46 @@ class PyTorchModelTrainer(PyTorchTrainerInterface):
backpropagation.
- Updates the model's parameters using an optimizer.
"""
data_loaders_dictionary = self.create_data_loaders_dictionary(data_dictionary, splits)
epochs = self.calc_n_epochs(
n_obs=len(data_dictionary["train_features"]),
batch_size=self.batch_size,
n_iters=self.max_iters
)
self.model.train()
for epoch in range(1, epochs + 1):
for i, batch_data in enumerate(data_loaders_dictionary["train"]):
data_loaders_dictionary = self.create_data_loaders_dictionary(data_dictionary, splits)
n_obs = len(data_dictionary["train_features"])
n_epochs = self.n_epochs or self.calc_n_epochs(n_obs=n_obs)
batch_counter = 0
for _ in range(n_epochs):
for _, batch_data in enumerate(data_loaders_dictionary["train"]):
xb, yb = batch_data
xb.to(self.device)
yb.to(self.device)
xb = xb.to(self.device)
yb = yb.to(self.device)
yb_pred = self.model(xb)
loss = self.criterion(yb_pred, yb)
self.optimizer.zero_grad(set_to_none=True)
loss.backward()
self.optimizer.step()
self.tb_logger.log_scalar("train_loss", loss.item(), i)
self.tb_logger.log_scalar("train_loss", loss.item(), batch_counter)
batch_counter += 1
# evaluation
if "test" in splits:
self.estimate_loss(
data_loaders_dictionary,
self.max_n_eval_batches,
"test"
)
self.estimate_loss(data_loaders_dictionary, "test")
@torch.no_grad()
def estimate_loss(
self,
data_loader_dictionary: Dict[str, DataLoader],
max_n_eval_batches: Optional[int],
split: str,
) -> None:
self.model.eval()
n_batches = 0
for i, batch_data in enumerate(data_loader_dictionary[split]):
if max_n_eval_batches and i > max_n_eval_batches:
n_batches += 1
break
for _, batch_data in enumerate(data_loader_dictionary[split]):
xb, yb = batch_data
xb.to(self.device)
yb.to(self.device)
xb = xb.to(self.device)
yb = yb.to(self.device)
yb_pred = self.model(xb)
loss = self.criterion(yb_pred, yb)
self.tb_logger.log_scalar(f"{split}_loss", loss.item(), i)
self.tb_logger.log_scalar(f"{split}_loss", loss.item(), self.test_batch_counter)
self.test_batch_counter += 1
self.model.train()
@@ -148,31 +142,30 @@ class PyTorchModelTrainer(PyTorchTrainerInterface):
return data_loader_dictionary
@staticmethod
def calc_n_epochs(n_obs: int, batch_size: int, n_iters: int) -> int:
def calc_n_epochs(self, n_obs: int) -> int:
"""
Calculates the number of epochs required to reach the maximum number
of iterations specified in the model training parameters.
the motivation here is that `max_iters` is easier to optimize and keep stable,
the motivation here is that `n_steps` is easier to optimize and keep stable,
across different n_obs - the number of data points.
"""
assert isinstance(self.n_steps, int), "Either `n_steps` or `n_epochs` should be set."
n_batches = n_obs // self.batch_size
n_epochs = min(self.n_steps // n_batches, 1)
if n_epochs <= 10:
logger.warning(
f"Setting low n_epochs: {n_epochs}. "
f"Please consider increasing `n_steps` hyper-parameter."
)
n_batches = math.ceil(n_obs // batch_size)
epochs = math.ceil(n_iters // n_batches)
if epochs <= 10:
logger.warning("User set `max_iters` in such a way that the trainer will only perform "
f" {epochs} epochs. Please consider increasing this value accordingly")
if epochs <= 1:
logger.warning("Epochs set to 1. Please review your `max_iters` value")
epochs = 1
return epochs
return n_epochs
def save(self, path: Path):
"""
- Saving any nn.Module state_dict
- Saving model_meta_data, this dict should contain any additional data that the
user needs to store. e.g class_names for classification models.
user needs to store. e.g. class_names for classification models.
"""
torch.save({

2
freqtrade/freqtradebot.py
View File

@@ -613,6 +613,8 @@ class FreqtradeBot(LoggingMixin):
for trade in Trade.get_open_trades():
# If there is any open orders, wait for them to finish.
if trade.open_order_id is None:
# Do a wallets update (will be ratelimited to once per hour)
self.wallets.update(False)
try:
self.check_and_call_adjust_trade_position(trade)
except DependencyException as exception:

24
freqtrade/misc.py
View File

@@ -192,30 +192,6 @@ def plural(num: float, singular: str, plural: Optional[str] = None) -> str:
return singular if (num == 1 or num == -1) else plural or singular + 's'
def render_template(templatefile: str, arguments: dict = {}) -> str:
from jinja2 import Environment, PackageLoader, select_autoescape
env = Environment(
loader=PackageLoader('freqtrade', 'templates'),
autoescape=select_autoescape(['html', 'xml'])
)
template = env.get_template(templatefile)
return template.render(**arguments)
def render_template_with_fallback(templatefile: str, templatefallbackfile: str,
arguments: dict = {}) -> str:
"""
Use templatefile if possible, otherwise fall back to templatefallbackfile
"""
from jinja2.exceptions import TemplateNotFound
try:
return render_template(templatefile, arguments)
except TemplateNotFound:
return render_template(templatefallbackfile, arguments)
def chunks(lst: List[Any], n: int) -> Iterator[List[Any]]:
"""
Split lst into chunks of the size n.

45
freqtrade/optimize/backtesting.py
View File

@@ -369,13 +369,14 @@ class Backtesting:
# Cleanup from prior runs
pair_data.drop(HEADERS[5:] + ['buy', 'sell'], axis=1, errors='ignore')
df_analyzed = self.strategy.ft_advise_signals(pair_data, {'pair': pair})
# Trim startup period from analyzed dataframe
df_analyzed = processed[pair] = pair_data = trim_dataframe(
df_analyzed, self.timerange, startup_candles=self.required_startup)
# Update dataprovider cache
self.dataprovider._set_cached_df(
pair, self.timeframe, df_analyzed, self.config['candle_type_def'])
# Trim startup period from analyzed dataframe
df_analyzed = processed[pair] = pair_data = trim_dataframe(
df_analyzed, self.timerange, startup_candles=self.required_startup)
# Create a copy of the dataframe before shifting, that way the entry signal/tag
# remains on the correct candle for callbacks.
df_analyzed = df_analyzed.copy()
@@ -567,8 +568,7 @@ class Backtesting:
pos_trade = self._get_exit_for_signal(trade, row, exit_, amount)
if pos_trade is not None:
order = pos_trade.orders[-1]
if self._get_order_filled(order.ft_price, row):
order.close_bt_order(current_date, trade)
if self._try_close_open_order(order, trade, current_date, row):
trade.recalc_trade_from_orders()
self.wallets.update()
return pos_trade
@@ -579,6 +579,19 @@ class Backtesting:
""" Rate is within candle, therefore filled"""
return row[LOW_IDX] <= rate <= row[HIGH_IDX]
def _try_close_open_order(
self, order: Optional[Order], trade: LocalTrade, current_date: datetime,
row: Tuple) -> bool:
"""
Check if an order is open and if it should've filled.
:return: True if the order filled.
"""
if order and self._get_order_filled(order.ft_price, row):
order.close_bt_order(current_date, trade)
trade.open_order_id = None
return True
return False
def _get_exit_for_signal(
self, trade: LocalTrade, row: Tuple, exit_: ExitCheckTuple,
amount: Optional[float] = None) -> Optional[LocalTrade]:
@@ -903,9 +916,7 @@ class Backtesting:
)
order._trade_bt = trade
trade.orders.append(order)
if pos_adjust and self._get_order_filled(order.ft_price, row):
order.close_bt_order(current_time, trade)
else:
if not self._try_close_open_order(order, trade, current_time, row):
trade.open_order_id = str(self.order_id_counter)
trade.recalc_trade_from_orders()
@@ -1121,23 +1132,18 @@ class Backtesting:
for trade in list(LocalTrade.bt_trades_open_pp[pair]):
# 3. Process entry orders.
order = trade.select_order(trade.entry_side, is_open=True)
if order and self._get_order_filled(order.ft_price, row):
order.close_bt_order(current_time, trade)
trade.open_order_id = None
if self._try_close_open_order(order, trade, current_time, row):
self.wallets.update()
# 4. Create exit orders (if any)
# 4. Create exit orders (if any)
if not trade.open_order_id:
self._check_trade_exit(trade, row) # Place exit order if necessary
# 5. Process exit orders.
# 5. Process exit orders.
order = trade.select_order(trade.exit_side, is_open=True)
if order and self._get_order_filled(order.ft_price, row):
order.close_bt_order(current_time, trade)
trade.open_order_id = None
if order and self._try_close_open_order(order, trade, current_time, row):
sub_trade = order.safe_amount_after_fee != trade.amount
if sub_trade:
order.close_bt_order(current_time, trade)
trade.recalc_trade_from_orders()
else:
trade.close_date = current_time
@@ -1191,7 +1197,8 @@ class Backtesting:
row_index += 1
indexes[pair] = row_index
self.dataprovider._set_dataframe_max_index(row_index)
self.dataprovider._set_dataframe_max_index(self.required_startup + row_index)
self.dataprovider._set_dataframe_max_date(current_time)
current_detail_time: datetime = row[DATE_IDX].to_pydatetime()
trade_dir: Optional[LongShort] = self.check_for_trade_entry(row)
@@ -1224,12 +1231,14 @@ class Backtesting:
is_first = True
current_time_det = current_time
for det_row in detail_data[HEADERS].values.tolist():
self.dataprovider._set_dataframe_max_date(current_time_det)
open_trade_count_start = self.backtest_loop(
det_row, pair, current_time_det, end_date,
open_trade_count_start, trade_dir, is_first)
current_time_det += timedelta(minutes=self.timeframe_detail_min)
is_first = False
else:
self.dataprovider._set_dataframe_max_date(current_time)
open_trade_count_start = self.backtest_loop(
row, pair, current_time, end_date,
open_trade_count_start, trade_dir)

43
freqtrade/optimize/lookahead_analysis.py
View File

@@ -48,6 +48,7 @@ class LookaheadAnalysis:
self.entry_varHolders: List[VarHolder] = []
self.exit_varHolders: List[VarHolder] = []
self.exchange: Optional[Any] = None
self._fee = None
# pull variables the scope of the lookahead_analysis-instance
self.local_config = deepcopy(config)
@@ -145,8 +146,13 @@ class LookaheadAnalysis:
str(self.dt_to_timestamp(varholder.to_dt)))
prepare_data_config['exchange']['pair_whitelist'] = pairs_to_load
if self._fee is not None:
# Don't re-calculate fee per pair, as fee might differ per pair.
prepare_data_config['fee'] = self._fee
backtesting = Backtesting(prepare_data_config, self.exchange)
self.exchange = backtesting.exchange
self._fee = backtesting.fee
backtesting._set_strategy(backtesting.strategylist[0])
varholder.data, varholder.timerange = backtesting.load_bt_data()
@@ -198,7 +204,7 @@ class LookaheadAnalysis:
self.prepare_data(exit_varHolder, [result_row['pair']])
# now we analyze a full trade of full_varholder and look for analyze its bias
def analyze_row(self, idx, result_row):
def analyze_row(self, idx: int, result_row):
# if force-sold, ignore this signal since here it will unconditionally exit.
if result_row.close_date == self.dt_to_timestamp(self.full_varHolder.to_dt):
return
@@ -209,12 +215,16 @@ class LookaheadAnalysis:
# fill entry_varHolder and exit_varHolder
self.fill_entry_and_exit_varHolders(result_row)
# this will trigger a logger-message
buy_or_sell_biased: bool = False
# register if buy signal is broken
if not self.report_signal(
self.entry_varHolders[idx].result,
"open_date",
self.entry_varHolders[idx].compared_dt):
self.current_analysis.false_entry_signals += 1
buy_or_sell_biased = True
# register if buy or sell signal is broken
if not self.report_signal(
@@ -222,6 +232,13 @@ class LookaheadAnalysis:
"close_date",
self.exit_varHolders[idx].compared_dt):
self.current_analysis.false_exit_signals += 1
buy_or_sell_biased = True
if buy_or_sell_biased:
logger.info(f"found lookahead-bias in trade "
f"pair: {result_row['pair']}, "
f"timerange:{result_row['open_date']} - {result_row['close_date']}, "
f"idx: {idx}")
# check if the indicators themselves contain biased data
self.analyze_indicators(self.full_varHolder, self.entry_varHolders[idx], result_row['pair'])
@@ -251,9 +268,33 @@ class LookaheadAnalysis:
# starting from the same datetime to avoid miss-reports of bias
for idx, result_row in self.full_varHolder.result['results'].iterrows():
if self.current_analysis.total_signals == self.targeted_trade_amount:
logger.info(f"Found targeted trade amount = {self.targeted_trade_amount} signals.")
break
if found_signals < self.minimum_trade_amount:
logger.info(f"only found {found_signals} "
f"which is smaller than "
f"minimum trade amount = {self.minimum_trade_amount}. "
f"Exiting this lookahead-analysis")
return None
if "force_exit" in result_row['exit_reason']:
logger.info("found force-exit in pair: {result_row['pair']}, "
f"timerange:{result_row['open_date']}-{result_row['close_date']}, "
f"idx: {idx}, skipping this one to avoid a false-positive.")
# just to keep the IDs of both full, entry and exit varholders the same
# to achieve a better debugging experience
self.entry_varHolders.append(VarHolder())
self.exit_varHolders.append(VarHolder())
continue
self.analyze_row(idx, result_row)
if len(self.entry_varHolders) < self.minimum_trade_amount:
logger.info(f"only found {found_signals} after skipping forced exits "
f"which is smaller than "
f"minimum trade amount = {self.minimum_trade_amount}. "
f"Exiting this lookahead-analysis")
# Restore verbosity, so it's not too quiet for the next strategy
restore_verbosity_for_bias_tester()
# check and report signals

13
freqtrade/optimize/lookahead_analysis_helpers.py
View File

@@ -137,6 +137,19 @@ class LookaheadAnalysisSubFunctions:
'just to avoid false positives')
config['dry_run_wallet'] = min_dry_run_wallet
if 'timerange' not in config:
# setting a timerange is enforced here
raise OperationalException(
"Please set a timerange. "
"Usually a few months are enough depending on your needs and strategy."
)
# fix stake_amount to 10k.
# in a combination with a wallet size of 1 billion it should always be able to trade
# no matter if they use custom_stake_amount as a small percentage of wallet size
# or fixate custom_stake_amount to a certain value.
logger.info('fixing stake_amount to 10k')
config['stake_amount'] = 10000
# enforce cache to be 'none', shift it to 'none' if not already
# (since the default value is 'day')
if config.get('backtest_cache') is None:

4
freqtrade/optimize/optimize_reports/bt_storage.py
View File

@@ -12,7 +12,7 @@ logger = logging.getLogger(__name__)
def store_backtest_stats(
recordfilename: Path, stats: BacktestResultType, dtappendix: str) -> None:
recordfilename: Path, stats: BacktestResultType, dtappendix: str) -> Path:
"""
Stores backtest results
:param recordfilename: Path object, which can either be a filename or a directory.
@@ -41,6 +41,8 @@ def store_backtest_stats(
latest_filename = Path.joinpath(filename.parent, LAST_BT_RESULT_FN)
file_dump_json(latest_filename, {'latest_backtest': str(filename.name)})
return filename
def _store_backtest_analysis_data(
recordfilename: Path, data: Dict[str, Dict],

34
freqtrade/persistence/trade_model.py
View File

@@ -48,7 +48,7 @@ class Order(ModelBase):
id: Mapped[int] = mapped_column(Integer, primary_key=True)
ft_trade_id: Mapped[int] = mapped_column(Integer, ForeignKey('trades.id'), index=True)
_trade_live: Mapped["Trade"] = relationship("Trade", back_populates="orders")
_trade_live: Mapped["Trade"] = relationship("Trade", back_populates="orders", lazy="immediate")
_trade_bt: "LocalTrade" = None # type: ignore
# order_side can only be 'buy', 'sell' or 'stoploss'
@@ -614,11 +614,9 @@ class LocalTrade:
"""
Method used internally to set self.stop_loss.
"""
stop_loss_norm = price_to_precision(stop_loss, self.price_precision, self.precision_mode,
rounding_mode=ROUND_DOWN if self.is_short else ROUND_UP)
if not self.stop_loss:
self.initial_stop_loss = stop_loss_norm
self.stop_loss = stop_loss_norm
self.initial_stop_loss = stop_loss
self.stop_loss = stop_loss
self.stop_loss_pct = -1 * abs(percent)
@@ -642,26 +640,27 @@ class LocalTrade:
else:
new_loss = float(current_price * (1 - abs(stoploss / leverage)))
stop_loss_norm = price_to_precision(new_loss, self.price_precision, self.precision_mode,
rounding_mode=ROUND_DOWN if self.is_short else ROUND_UP)
# no stop loss assigned yet
if self.initial_stop_loss_pct is None or refresh:
self.__set_stop_loss(new_loss, stoploss)
self.__set_stop_loss(stop_loss_norm, stoploss)
self.initial_stop_loss = price_to_precision(
new_loss, self.price_precision, self.precision_mode,
stop_loss_norm, self.price_precision, self.precision_mode,
rounding_mode=ROUND_DOWN if self.is_short else ROUND_UP)
self.initial_stop_loss_pct = -1 * abs(stoploss)
# evaluate if the stop loss needs to be updated
else:
higher_stop = new_loss > self.stop_loss
lower_stop = new_loss < self.stop_loss
higher_stop = stop_loss_norm > self.stop_loss
lower_stop = stop_loss_norm < self.stop_loss
# stop losses only walk up, never down!,
# ? But adding more to a leveraged trade would create a lower liquidation price,
# ? decreasing the minimum stoploss
if (higher_stop and not self.is_short) or (lower_stop and self.is_short):
logger.debug(f"{self.pair} - Adjusting stoploss...")
self.__set_stop_loss(new_loss, stoploss)
self.__set_stop_loss(stop_loss_norm, stoploss)
else:
logger.debug(f"{self.pair} - Keeping current stoploss...")
@@ -746,10 +745,8 @@ class LocalTrade:
self.open_order_id = None
self.recalc_trade_from_orders(is_closing=True)
if show_msg:
logger.info(
'Marking %s as closed as the trade is fulfilled and found no open orders for it.',
self
)
logger.info(f"Marking {self} as closed as the trade is fulfilled "
"and found no open orders for it.")
def update_fee(self, fee_cost: float, fee_currency: Optional[str], fee_rate: Optional[float],
side: str) -> None:
@@ -1187,12 +1184,13 @@ class LocalTrade:
return LocalTrade.bt_open_open_trade_count
@staticmethod
def stoploss_reinitialization(desired_stoploss):
def stoploss_reinitialization(desired_stoploss: float):
"""
Adjust initial Stoploss to desired stoploss for all open trades.
"""
trade: Trade
for trade in Trade.get_open_trades():
logger.info("Found open trade: %s", trade)
logger.info(f"Found open trade: {trade}")
# skip case if trailing-stop changed the stoploss already.
if (trade.stop_loss == trade.initial_stop_loss
@@ -1201,7 +1199,7 @@ class LocalTrade:
logger.info(f"Stoploss for {trade} needs adjustment...")
# Force reset of stoploss
trade.stop_loss = None
trade.stop_loss = 0.0
trade.initial_stop_loss_pct = None
trade.adjust_stop_loss(trade.open_rate, desired_stoploss)
logger.info(f"New stoploss: {trade.stop_loss}.")

39
freqtrade/rpc/api_server/api_backtest.py
View File

@@ -10,14 +10,15 @@ from fastapi.exceptions import HTTPException
from freqtrade.configuration.config_validation import validate_config_consistency
from freqtrade.constants import Config
from freqtrade.data.btanalysis import (delete_backtest_result, get_backtest_resultlist,
load_and_merge_backtest_result)
from freqtrade.data.btanalysis import (delete_backtest_result, get_backtest_result,
get_backtest_resultlist, load_and_merge_backtest_result,
update_backtest_metadata)
from freqtrade.enums import BacktestState
from freqtrade.exceptions import DependencyException, OperationalException
from freqtrade.exchange.common import remove_exchange_credentials
from freqtrade.misc import deep_merge_dicts, is_file_in_dir
from freqtrade.rpc.api_server.api_schemas import (BacktestHistoryEntry, BacktestRequest,
BacktestResponse)
from freqtrade.rpc.api_server.api_schemas import (BacktestHistoryEntry, BacktestMetadataUpdate,
BacktestRequest, BacktestResponse)
from freqtrade.rpc.api_server.deps import get_config
from freqtrade.rpc.api_server.webserver_bgwork import ApiBG
from freqtrade.rpc.rpc import RPCException
@@ -74,10 +75,11 @@ def __run_backtest_bg(btconfig: Config):
ApiBG.bt['bt'].load_prior_backtest()
ApiBG.bt['bt'].abort = False
strategy_name = strat.get_strategy_name()
if (ApiBG.bt['bt'].results and
strat.get_strategy_name() in ApiBG.bt['bt'].results['strategy']):
strategy_name in ApiBG.bt['bt'].results['strategy']):
# When previous result hash matches - reuse that result and skip backtesting.
logger.info(f'Reusing result of previous backtest for {strat.get_strategy_name()}')
logger.info(f'Reusing result of previous backtest for {strategy_name}')
else:
min_date, max_date = ApiBG.bt['bt'].backtest_one_strategy(
strat, ApiBG.bt['data'], ApiBG.bt['timerange'])
@@ -87,10 +89,12 @@ def __run_backtest_bg(btconfig: Config):
min_date=min_date, max_date=max_date)
if btconfig.get('export', 'none') == 'trades':
store_backtest_stats(
fn = store_backtest_stats(
btconfig['exportfilename'], ApiBG.bt['bt'].results,
datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
)
ApiBG.bt['bt'].results['metadata'][strategy_name]['filename'] = str(fn.name)
ApiBG.bt['bt'].results['metadata'][strategy_name]['strategy'] = strategy_name
logger.info("Backtest finished.")
@@ -281,3 +285,24 @@ def api_delete_backtest_history_entry(file: str, config=Depends(get_config)):
delete_backtest_result(file_abs)
return get_backtest_resultlist(config['user_data_dir'] / 'backtest_results')
@router.patch('/backtest/history/{file}', response_model=List[BacktestHistoryEntry],
tags=['webserver', 'backtest'])
def api_update_backtest_history_entry(file: str, body: BacktestMetadataUpdate,
config=Depends(get_config)):
# Get backtest result history, read from metadata files
bt_results_base: Path = config['user_data_dir'] / 'backtest_results'
file_abs = (bt_results_base / file).with_suffix('.json')
# Ensure file is in backtest_results directory
if not is_file_in_dir(file_abs, bt_results_base):
raise HTTPException(status_code=404, detail="File not found.")
content = {
'notes': body.notes
}
try:
update_backtest_metadata(file_abs, body.strategy, content)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
return get_backtest_result(file_abs)

6
freqtrade/rpc/api_server/api_schemas.py
View File

@@ -524,6 +524,12 @@ class BacktestHistoryEntry(BaseModel):
strategy: str
run_id: str
backtest_start_time: int
notes: Optional[str] = ''
class BacktestMetadataUpdate(BaseModel):
strategy: str
notes: str = ''
class SysInfo(BaseModel):

3
freqtrade/rpc/api_server/api_v1.py
View File

@@ -50,7 +50,8 @@ logger = logging.getLogger(__name__)
# 2.29: Add /exchanges endpoint
# 2.30: new /pairlists endpoint
# 2.31: new /backtest/history/ delete endpoint
API_VERSION = 2.31
# 2.32: new /backtest/history/ patch endpoint
API_VERSION = 2.32
# Public API, requires no auth.
router_public = APIRouter()

1
freqtrade/rpc/discord.py
View File

@@ -20,6 +20,7 @@ class Discord(Webhook):
self._format = 'json'
self._retries = 1
self._retry_delay = 0.1
self._timeout = self._config['discord'].get('timeout', 10)
def cleanup(self) -> None:
"""

1
freqtrade/rpc/fiat_convert.py
View File

@@ -26,6 +26,7 @@ coingecko_mapping = {
'sol': 'solana',
'usdt': 'tether',
'busd': 'binance-usd',
'tusd': 'true-usd',
}

18
freqtrade/rpc/rpc.py
View File

@@ -605,17 +605,13 @@ class RPC:
est_stake = balance.free
est_bot_stake = amount
else:
try:
pair = self._freqtrade.exchange.get_valid_pair_combination(coin, stake_currency)
rate: Optional[float] = tickers.get(pair, {}).get('last', None)
if rate:
if pair.startswith(stake_currency) and not pair.endswith(stake_currency):
rate = 1.0 / rate
est_stake = rate * balance.total
est_bot_stake = rate * amount
except (ExchangeError):
logger.warning(f"Could not get rate for pair {coin}.")
raise ValueError()
pair = self._freqtrade.exchange.get_valid_pair_combination(coin, stake_currency)
rate: Optional[float] = tickers.get(pair, {}).get('last', None)
if rate:
if pair.startswith(stake_currency) and not pair.endswith(stake_currency):
rate = 1.0 / rate
est_stake = rate * balance.total
est_bot_stake = rate * amount
return est_stake, est_bot_stake

5
freqtrade/strategy/interface.py
View File

@@ -381,7 +381,7 @@ class IStrategy(ABC, HyperStrategyMixin):
For full documentation please go to https://www.freqtrade.io/en/latest/strategy-advanced/
When not implemented by a strategy, returns the initial stoploss value
When not implemented by a strategy, returns the initial stoploss value.
Only called when use_custom_stoploss is set to True.
:param pair: Pair that's currently analyzed
@@ -1181,7 +1181,8 @@ class IStrategy(ABC, HyperStrategyMixin):
bound = (low if trade.is_short else high)
bound_profit = current_profit if not bound else trade.calc_profit_ratio(bound)
if self.use_custom_stoploss and dir_correct:
stop_loss_value = strategy_safe_wrapper(self.custom_stoploss, default_retval=None
stop_loss_value = strategy_safe_wrapper(self.custom_stoploss, default_retval=None,
supress_error=True
)(pair=trade.pair, trade=trade,
current_time=current_time,
current_rate=(bound or current_rate),

14
freqtrade/templates/base_strategy.py.j2
View File

@@ -78,19 +78,7 @@ class {{ strategy }}(IStrategy):
buy_rsi = IntParameter(10, 40, default=30, space="buy")
sell_rsi = IntParameter(60, 90, default=70, space="sell")
# Optional order type mapping.
order_types = {
'entry': 'limit',
'exit': 'limit',
'stoploss': 'market',
'stoploss_on_exchange': False
}
# Optional order time in force.
order_time_in_force = {
'entry': 'GTC',
'exit': 'GTC'
}
{{ attributes | indent(4) }}
{{ plot_config | indent(4) }}
def informative_pairs(self):

13
freqtrade/templates/strategy_subtemplates/strategy_attributes_full.j2 Normal file
View File

@@ -0,0 +1,13 @@
# Optional order type mapping.
order_types = {
'entry': 'limit',
'exit': 'limit',
'stoploss': 'market',
'stoploss_on_exchange': False
}
# Optional order time in force.
order_time_in_force = {
'entry': 'GTC',
'exit': 'GTC'
}

0
freqtrade/templates/strategy_subtemplates/strategy_attributes_minimal.j2 Normal file
View File

1
freqtrade/types/backtest_result_type.py
View File

@@ -25,3 +25,4 @@ def get_BacktestResultType_default() -> BacktestResultType:
class BacktestHistoryEntryType(BacktestMetadataType):
filename: str
strategy: str
notes: str

1
freqtrade/util/__init__.py
View File

@@ -2,6 +2,7 @@ from freqtrade.util.datetime_helpers import (dt_floor_day, dt_from_ts, dt_humani
dt_utc, format_ms_time, shorten_date)
from freqtrade.util.ft_precise import FtPrecise
from freqtrade.util.periodic_cache import PeriodicCache
from freqtrade.util.template_renderer import render_template, render_template_with_fallback # noqa
__all__ = [

27
freqtrade/util/template_renderer.py Normal file
View File

@@ -0,0 +1,27 @@
"""
Jinja2 rendering utils, used to generate new strategy and configurations.
"""
def render_template(templatefile: str, arguments: dict = {}) -> str:
from jinja2 import Environment, PackageLoader, select_autoescape
env = Environment(
loader=PackageLoader('freqtrade', 'templates'),
autoescape=select_autoescape(['html', 'xml'])
)
template = env.get_template(templatefile)
return template.render(**arguments)
def render_template_with_fallback(templatefile: str, templatefallbackfile: str,
arguments: dict = {}) -> str:
"""
Use templatefile if possible, otherwise fall back to templatefallbackfile
"""
from jinja2.exceptions import TemplateNotFound
try:
return render_template(templatefile, arguments)
except TemplateNotFound:
return render_template(templatefallbackfile, arguments)

1
mkdocs.yml
View File

@@ -47,7 +47,6 @@ nav:
- Advanced Hyperopt: advanced-hyperopt.md
- Producer/Consumer mode: producer-consumer.md
- Edge Positioning: edge.md
- Sandbox Testing: sandbox-testing.md
- FAQ: faq.md
- SQL Cheat-sheet: sql_cheatsheet.md
- Strategy migration: strategy_migration.md

2
pyproject.toml
View File

@@ -63,7 +63,7 @@ ignore = ["freqtrade/vendor/**"]
[tool.ruff]
line-length = 100
extend-exclude = [".env"]
extend-exclude = [".env", ".venv"]
target-version = "py38"
extend-select = [
"C90", # mccabe

4
requirements-dev.txt
View File

@@ -7,8 +7,8 @@
-r docs/requirements-docs.txt
coveralls==3.3.1
ruff==0.0.282
mypy==1.4.1
ruff==0.0.284
mypy==1.5.0
pre-commit==3.3.3
pytest==7.4.0
pytest-asyncio==0.21.1

2
requirements-freqai-rl.txt
View File

@@ -8,4 +8,4 @@ gymnasium==0.28.1
stable_baselines3==2.0.0
sb3_contrib>=2.0.0a9
# Progress bar for stable-baselines3 and sb3-contrib
tqdm==4.65.0
tqdm==4.66.1

4
requirements-freqai.txt
View File

@@ -4,9 +4,9 @@
# Required for freqai
scikit-learn==1.1.3
joblib==1.3.1
joblib==1.3.2
catboost==1.2; 'arm' not in platform_machine
lightgbm==4.0.0
xgboost==1.7.6
tensorboard==2.13.0
tensorboard==2.14.0
datasieve==0.1.7

2
requirements-plot.txt
View File

@@ -1,4 +1,4 @@
# Include all requirements to run the bot.
-r requirements.txt
plotly==5.15.0
plotly==5.16.0

16
requirements.txt
View File

@@ -3,7 +3,7 @@ numpy==1.24.3; python_version <= '3.8'
pandas==2.0.3
pandas-ta==0.3.14b
ccxt==4.0.47
ccxt==4.0.59
cryptography==41.0.3; platform_machine != 'armv7l'
cryptography==40.0.1; platform_machine == 'armv7l'
aiohttp==3.8.5
@@ -15,7 +15,7 @@ arrow==1.2.3
cachetools==5.3.1
requests==2.31.0
urllib3==2.0.4
jsonschema==4.18.4
jsonschema==4.19.0
TA-Lib==0.4.27
technical==1.4.0
tabulate==0.9.0
@@ -23,7 +23,7 @@ pycoingecko==3.1.0
jinja2==3.1.2
tables==3.8.0
blosc==1.11.1
joblib==1.3.1
joblib==1.3.2
rich==13.5.2
pyarrow==12.0.1; platform_machine != 'armv7l'
@@ -33,24 +33,24 @@ py_find_1st==1.1.5
# Load ticker files 30% faster
python-rapidjson==1.10
# Properly format api responses
orjson==3.9.2
orjson==3.9.4
# Notify systemd
sdnotify==0.3.2
# API Server
fastapi==0.100.1
fastapi==0.101.0
pydantic==2.1.1
uvicorn==0.23.2
pyjwt==2.8.0
aiofiles==23.1.0
aiofiles==23.2.1
psutil==5.9.5
# Support for colorized terminal output
colorama==0.4.6
# Building config files interactively
questionary==1.10.0
prompt-toolkit==3.0.39
questionary==2.0.0
prompt-toolkit==3.0.36
# Extensions to datetime library
python-dateutil==2.8.2

53
setup.sh
View File

@@ -11,7 +11,7 @@ function check_installed_pip() {
${PYTHON} -m pip > /dev/null
if [ $? -ne 0 ]; then
echo_block "Installing Pip for ${PYTHON}"
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
curl https://bootstrap.pypa.io/get-pip.py -s -o get-pip.py
${PYTHON} get-pip.py
rm get-pip.py
fi
@@ -41,12 +41,12 @@ function check_installed_python() {
}
function updateenv() {
echo_block "Updating your virtual env"
if [ ! -f .env/bin/activate ]; then
echo_block "Updating your virtual environment"
if [ ! -f .venv/bin/activate ]; then
echo "Something went wrong, no virtual environment found."
exit 1
fi
source .env/bin/activate
source .venv/bin/activate
SYS_ARCH=$(uname -m)
echo "pip install in-progress. Please wait..."
${PYTHON} -m pip install --upgrade pip wheel setuptools
@@ -186,7 +186,14 @@ function install_redhat() {
# Upgrade the bot
function update() {
git pull
if [ -f .env/bin/activate ]; then
# Old environment found - updating to new environment.
recreate_environments
fi
updateenv
echo "Update completed."
echo_block "Don't forget to activate your virtual enviorment with 'source .venv/bin/activate'!"
}
function check_git_changes() {
@@ -199,6 +206,27 @@ function check_git_changes() {
fi
}
function recreate_environments() {
if [ -d ".env" ]; then
# Remove old virtual env
echo "- Deleting your previous virtual env"
echo "Warning: Your new environment will be at .venv!"
rm -rf .env
fi
if [ -d ".venv" ]; then
echo "- Deleting your previous virtual env"
rm -rf .venv
fi
echo
${PYTHON} -m venv .venv
if [ $? -ne 0 ]; then
echo "Could not create virtual environment. Leaving now"
exit 1
fi
}
# Reset Develop or Stable branch
function reset() {
echo_block "Resetting branch and virtual env"
@@ -225,17 +253,8 @@ function reset() {
else
echo "Reset ignored because you are not on 'stable' or 'develop'."
fi
recreate_environments
if [ -d ".env" ]; then
echo "- Deleting your previous virtual env"
rm -rf .env
fi
echo
${PYTHON} -m venv .env
if [ $? -ne 0 ]; then
echo "Could not create virtual environment. Leaving now"
exit 1
fi
updateenv
}
@@ -266,9 +285,9 @@ function install() {
reset
config
echo_block "Run the bot !"
echo "You can now use the bot by executing 'source .env/bin/activate; freqtrade <subcommand>'."
echo "You can see the list of available bot sub-commands by executing 'source .env/bin/activate; freqtrade --help'."
echo "You verify that freqtrade is installed successfully by running 'source .env/bin/activate; freqtrade --version'."
echo "You can now use the bot by executing 'source .venv/bin/activate; freqtrade <subcommand>'."
echo "You can see the list of available bot sub-commands by executing 'source .venv/bin/activate; freqtrade --help'."
echo "You verify that freqtrade is installed successfully by running 'source .venv/bin/activate; freqtrade --version'."
}
function plot() {

13
tests/data/test_converter.py
View File

@@ -34,26 +34,21 @@ def test_ohlcv_to_dataframe(ohlcv_history_list, caplog):
assert log_has('Converting candle (OHLCV) data to dataframe for pair UNITTEST/BTC.', caplog)
def test_trades_to_ohlcv(ohlcv_history_list, caplog):
def test_trades_to_ohlcv(trades_history, caplog):
caplog.set_level(logging.DEBUG)
with pytest.raises(ValueError, match="Trade-list empty."):
trades_to_ohlcv([], '1m')
trades = [
[1570752011620, "13519807", None, "sell", 0.00141342, 23.0, 0.03250866],
[1570752011620, "13519808", None, "sell", 0.00141266, 54.0, 0.07628364],
[1570752017964, "13519809", None, "sell", 0.00141266, 8.0, 0.01130128]]
df = trades_to_ohlcv(trades, '1m')
df = trades_to_ohlcv(trades_history, '1m')
assert not df.empty
assert len(df) == 1
assert 'open' in df.columns
assert 'high' in df.columns
assert 'low' in df.columns
assert 'close' in df.columns
assert df.loc[:, 'high'][0] == 0.00141342
assert df.loc[:, 'low'][0] == 0.00141266
assert df.loc[:, 'high'][0] == 0.019627
assert df.loc[:, 'low'][0] == 0.019626
def test_ohlcv_fill_up_missing_data(testdatadir, caplog):

11
tests/data/test_dataprovider.py
View File

@@ -129,9 +129,14 @@ def test_get_pair_dataframe(mocker, default_conf, ohlcv_history, candle_type):
default_conf["runmode"] = RunMode.BACKTEST
dp = DataProvider(default_conf, exchange)
assert dp.runmode == RunMode.BACKTEST
assert isinstance(dp.get_pair_dataframe(
"UNITTEST/BTC", timeframe, candle_type=candle_type), DataFrame)
# assert dp.get_pair_dataframe("NONESENSE/AAA", timeframe).empty
df = dp.get_pair_dataframe("UNITTEST/BTC", timeframe, candle_type=candle_type)
assert isinstance(df, DataFrame)
assert len(df) == 3 # ohlcv_history mock has just 3 rows
dp._set_dataframe_max_date(ohlcv_history.iloc[-1]['date'])
df = dp.get_pair_dataframe("UNITTEST/BTC", timeframe, candle_type=candle_type)
assert isinstance(df, DataFrame)
assert len(df) == 2 # ohlcv_history is limited to 2 rows now
def test_available_pairs(mocker, default_conf, ohlcv_history):

2
tests/exchange/test_binance.py
View File

@@ -35,7 +35,7 @@ def test__get_params_binance(default_conf, mocker, side, type, time_in_force, ex
])
def test_create_stoploss_order_binance(default_conf, mocker, limitratio, expected, side, trademode):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_buy_{randint(0, 10 ** 6)}'
order_type = 'stop_loss_limit' if trademode == TradingMode.SPOT else 'stop'
api_mock.create_order = MagicMock(return_value={

4
tests/exchange/test_bybit.py
View File

@@ -3,7 +3,6 @@ from unittest.mock import MagicMock
from freqtrade.enums.marginmode import MarginMode
from freqtrade.enums.tradingmode import TradingMode
from freqtrade.exchange.exchange_utils import timeframe_to_msecs
from tests.conftest import get_mock_coro, get_patched_exchange
from tests.exchange.test_exchange import ccxt_exceptionhandlers
@@ -37,12 +36,10 @@ async def test_bybit_fetch_funding_rate(default_conf, mocker):
assert api_mock.fetch_funding_rate_history.call_count == 1
assert api_mock.fetch_funding_rate_history.call_args_list[0][0][0] == 'BTC/USDT:USDT'
kwargs = api_mock.fetch_funding_rate_history.call_args_list[0][1]
assert kwargs['params'] == {}
assert kwargs['since'] is None
api_mock.fetch_funding_rate_history.reset_mock()
since_ms = 1610000000000
since_ms_end = since_ms + (timeframe_to_msecs('4h') * limit)
# Test fetch_funding_rate_history (current data)
await exchange._fetch_funding_rate_history(
pair='BTC/USDT:USDT',
@@ -54,7 +51,6 @@ async def test_bybit_fetch_funding_rate(default_conf, mocker):
assert api_mock.fetch_funding_rate_history.call_count == 1
assert api_mock.fetch_funding_rate_history.call_args_list[0][0][0] == 'BTC/USDT:USDT'
kwargs = api_mock.fetch_funding_rate_history.call_args_list[0][1]
assert kwargs['params'] == {'until': since_ms_end}
assert kwargs['since'] == since_ms

51
tests/exchange/test_exchange.py
View File

@@ -556,41 +556,6 @@ def test_get_min_pair_stake_amount_real_data(mocker, default_conf) -> None:
assert result == 4000
def test_set_sandbox(default_conf, mocker):
"""
Test working scenario
"""
api_mock = MagicMock()
api_mock.load_markets = MagicMock(return_value={
'ETH/BTC': '', 'LTC/BTC': '', 'XRP/BTC': '', 'NEO/BTC': ''
})
url_mock = PropertyMock(return_value={'test': "api-public.sandbox.gdax.com",
'api': 'https://api.gdax.com'})
type(api_mock).urls = url_mock
exchange = get_patched_exchange(mocker, default_conf, api_mock)
liveurl = exchange._api.urls['api']
default_conf['exchange']['sandbox'] = True
exchange.set_sandbox(exchange._api, default_conf['exchange'], 'Logname')
assert exchange._api.urls['api'] != liveurl
def test_set_sandbox_exception(default_conf, mocker):
"""
Test Fail scenario
"""
api_mock = MagicMock()
api_mock.load_markets = MagicMock(return_value={
'ETH/BTC': '', 'LTC/BTC': '', 'XRP/BTC': '', 'NEO/BTC': ''
})
url_mock = PropertyMock(return_value={'api': 'https://api.gdax.com'})
type(api_mock).urls = url_mock
with pytest.raises(OperationalException, match=r'does not provide a sandbox api'):
exchange = get_patched_exchange(mocker, default_conf, api_mock)
default_conf['exchange']['sandbox'] = True
exchange.set_sandbox(exchange._api, default_conf['exchange'], 'Logname')
def test__load_async_markets(default_conf, mocker, caplog):
mocker.patch(f'{EXMS}._init_ccxt')
mocker.patch(f'{EXMS}.validate_pairs')
@@ -1372,7 +1337,7 @@ def test_create_dry_run_order_market_fill(default_conf, mocker, side, rate, amou
@pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_create_order(default_conf, mocker, side, ordertype, rate, marketprice, exchange_name):
api_mock = MagicMock()
order_id = 'test_prod_{}_{}'.format(side, randint(0, 10 ** 6))
order_id = f'test_prod_{side}_{randint(0, 10 ** 6)}'
api_mock.options = {} if not marketprice else {"createMarketBuyOrderRequiresPrice": True}
api_mock.create_order = MagicMock(return_value={
'id': order_id,
@@ -1452,7 +1417,7 @@ def test_buy_dry_run(default_conf, mocker, exchange_name):
@pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_buy_prod(default_conf, mocker, exchange_name):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_buy_{randint(0, 10 ** 6)}'
order_type = 'market'
time_in_force = 'gtc'
api_mock.options = {}
@@ -1541,7 +1506,7 @@ def test_buy_prod(default_conf, mocker, exchange_name):
@pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_buy_considers_time_in_force(default_conf, mocker, exchange_name):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_buy_{randint(0, 10 ** 6)}'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
'id': order_id,
@@ -1608,7 +1573,7 @@ def test_sell_dry_run(default_conf, mocker):
@pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_sell_prod(default_conf, mocker, exchange_name):
api_mock = MagicMock()
order_id = 'test_prod_sell_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_sell_{randint(0, 10 ** 6)}'
order_type = 'market'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
@@ -1686,7 +1651,7 @@ def test_sell_prod(default_conf, mocker, exchange_name):
@pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_sell_considers_time_in_force(default_conf, mocker, exchange_name):
api_mock = MagicMock()
order_id = 'test_prod_sell_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_sell_{randint(0, 10 ** 6)}'
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'symbol': 'ETH/BTC',
@@ -2100,7 +2065,7 @@ def test_get_historic_ohlcv(default_conf, mocker, caplog, exchange_name, candle_
exchange._async_get_candle_history = Mock(wraps=mock_candle_hist)
# one_call calculation * 1.8 should do 2 calls
since = 5 * 60 * exchange.ohlcv_candle_limit('5m', CandleType.SPOT) * 1.8
since = 5 * 60 * exchange.ohlcv_candle_limit('5m', candle_type) * 1.8
ret = exchange.get_historic_ohlcv(
pair,
"5m",
@@ -3557,7 +3522,7 @@ def test_get_valid_pair_combination(default_conf, mocker, markets):
assert ex.get_valid_pair_combination("ETH", "BTC") == "ETH/BTC"
assert ex.get_valid_pair_combination("BTC", "ETH") == "ETH/BTC"
with pytest.raises(DependencyException, match=r"Could not combine.* to get a valid pair."):
with pytest.raises(ValueError, match=r"Could not combine.* to get a valid pair."):
ex.get_valid_pair_combination("NOPAIR", "ETH")
@@ -5392,7 +5357,7 @@ def test_get_liquidation_price(
])
def test_stoploss_contract_size(mocker, default_conf, contract_size, order_amount):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_buy_{randint(0, 10 ** 6)}'
api_mock.create_order = MagicMock(return_value={
'id': order_id,

2
tests/exchange/test_huobi.py
View File

@@ -16,7 +16,7 @@ from tests.exchange.test_exchange import ccxt_exceptionhandlers
])
def test_create_stoploss_order_huobi(default_conf, mocker, limitratio, expected, side):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_buy_{randint(0, 10 ** 6)}'
order_type = 'stop-limit'
api_mock.create_order = MagicMock(return_value={

6
tests/exchange/test_kraken.py
View File

@@ -15,7 +15,7 @@ STOPLOSS_LIMIT_ORDERTYPE = 'stop-loss-limit'
def test_buy_kraken_trading_agreement(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_buy_{randint(0, 10 ** 6)}'
order_type = 'limit'
time_in_force = 'ioc'
api_mock.options = {}
@@ -56,7 +56,7 @@ def test_buy_kraken_trading_agreement(default_conf, mocker):
def test_sell_kraken_trading_agreement(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_sell_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_sell_{randint(0, 10 ** 6)}'
order_type = 'market'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
@@ -181,7 +181,7 @@ def test_get_balances_prod(default_conf, mocker):
])
def test_create_stoploss_order_kraken(default_conf, mocker, ordertype, side, adjustedprice):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_buy_{randint(0, 10 ** 6)}'
api_mock.create_order = MagicMock(return_value={
'id': order_id,

4
tests/exchange/test_kucoin.py
View File

@@ -17,7 +17,7 @@ from tests.exchange.test_exchange import ccxt_exceptionhandlers
])
def test_create_stoploss_order_kucoin(default_conf, mocker, limitratio, expected, side, order_type):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_id = f'test_prod_buy_{randint(0, 10 ** 6)}'
api_mock.create_order = MagicMock(return_value={
'id': order_id,
@@ -136,7 +136,7 @@ def test_stoploss_adjust_kucoin(mocker, default_conf):
])
def test_kucoin_create_order(default_conf, mocker, side, ordertype, rate):
api_mock = MagicMock()
order_id = 'test_prod_{}_{}'.format(side, randint(0, 10 ** 6))
order_id = f'test_prod_{side}_{randint(0, 10 ** 6)}'
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {

0
tests/exchange_online/__init__.py Normal file
View File

334
tests/exchange_online/conftest.py Normal file
View File

@@ -0,0 +1,334 @@
from copy import deepcopy
from pathlib import Path
from typing import Tuple
import pytest
from freqtrade.constants import Config
from freqtrade.exchange.exchange import Exchange
from freqtrade.resolvers.exchange_resolver import ExchangeResolver
from tests.conftest import EXMS, get_default_conf_usdt
EXCHANGE_FIXTURE_TYPE = Tuple[Exchange, str]
# Exchanges that should be tested online
EXCHANGES = {
'bittrex': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': False,
'timeframe': '1h',
'leverage_tiers_public': False,
'leverage_in_spot_market': False,
},
'binance': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'use_ci_proxy': True,
'hasQuoteVolume': True,
'timeframe': '1h',
'futures': True,
'futures_pair': 'BTC/USDT:USDT',
'hasQuoteVolumeFutures': True,
'leverage_tiers_public': False,
'leverage_in_spot_market': False,
'trades_lookback_hours': 4,
'private_methods': [
'fapiPrivateGetPositionSideDual',
'fapiPrivateGetMultiAssetsMargin'
],
'sample_order': [{
"symbol": "SOLUSDT",
"orderId": 3551312894,
"orderListId": -1,
"clientOrderId": "x-R4DD3S8297c73a11ccb9dc8f2811ba",
"transactTime": 1674493798550,
"price": "15.50000000",
"origQty": "1.10000000",
"executedQty": "0.00000000",
"cummulativeQuoteQty": "0.00000000",
"status": "NEW",
"timeInForce": "GTC",
"type": "LIMIT",
"side": "BUY",
"workingTime": 1674493798550,
"fills": [],
"selfTradePreventionMode": "NONE",
}]
},
'binanceus': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': True,
'timeframe': '1h',
'futures': False,
'sample_order': [{
"symbol": "SOLUSDT",
"orderId": 3551312894,
"orderListId": -1,
"clientOrderId": "x-R4DD3S8297c73a11ccb9dc8f2811ba",
"transactTime": 1674493798550,
"price": "15.50000000",
"origQty": "1.10000000",
"executedQty": "0.00000000",
"cummulativeQuoteQty": "0.00000000",
"status": "NEW",
"timeInForce": "GTC",
"type": "LIMIT",
"side": "BUY",
"workingTime": 1674493798550,
"fills": [],
"selfTradePreventionMode": "NONE",
}]
},
'kraken': {
'pair': 'BTC/USD',
'stake_currency': 'USD',
'hasQuoteVolume': True,
'timeframe': '1h',
'leverage_tiers_public': False,
'leverage_in_spot_market': True,
'trades_lookback_hours': 12,
},
'kucoin': {
'pair': 'XRP/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': True,
'timeframe': '1h',
'leverage_tiers_public': False,
'leverage_in_spot_market': True,
'sample_order': [
{'id': '63d6742d0adc5570001d2bbf7'}, # create order
{
'id': '63d6742d0adc5570001d2bbf7',
'symbol': 'SOL-USDT',
'opType': 'DEAL',
'type': 'limit',
'side': 'buy',
'price': '15.5',
'size': '1.1',
'funds': '0',
'dealFunds': '17.05',
'dealSize': '1.1',
'fee': '0.000065252',
'feeCurrency': 'USDT',
'stp': '',
'stop': '',
'stopTriggered': False,
'stopPrice': '0',
'timeInForce': 'GTC',
'postOnly': False,
'hidden': False,
'iceberg': False,
'visibleSize': '0',
'cancelAfter': 0,
'channel': 'API',
'clientOid': '0a053870-11bf-41e5-be61-b272a4cb62e1',
'remark': None,
'tags': 'partner:ccxt',
'isActive': False,
'cancelExist': False,
'createdAt': 1674493798550,
'tradeType': 'TRADE'
}],
},
'gate': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': True,
'timeframe': '1h',
'futures': True,
'futures_pair': 'BTC/USDT:USDT',
'hasQuoteVolumeFutures': True,
'leverage_tiers_public': True,
'leverage_in_spot_market': True,
'sample_order': [
{
"id": "276266139423",
"text": "apiv4",
"create_time": "1674493798",
"update_time": "1674493798",
"create_time_ms": "1674493798550",
"update_time_ms": "1674493798550",
"status": "closed",
"currency_pair": "SOL_USDT",
"type": "limit",
"account": "spot",
"side": "buy",
"amount": "1.1",
"price": "15.5",
"time_in_force": "gtc",
"iceberg": "0",
"left": "0",
"fill_price": "17.05",
"filled_total": "17.05",
"avg_deal_price": "15.5",
"fee": "0.0000018",
"fee_currency": "SOL",
"point_fee": "0",
"gt_fee": "0",
"gt_maker_fee": "0",
"gt_taker_fee": "0.0015",
"gt_discount": True,
"rebated_fee": "0",
"rebated_fee_currency": "USDT"
},
{
# market order
'id': '276401180529',
'text': 'apiv4',
'create_time': '1674493798',
'update_time': '1674493798',
'create_time_ms': '1674493798550',
'update_time_ms': '1674493798550',
'status': 'cancelled',
'currency_pair': 'SOL_USDT',
'type': 'market',
'account': 'spot',
'side': 'buy',
'amount': '17.05',
'price': '0',
'time_in_force': 'ioc',
'iceberg': '0',
'left': '0.0000000016228',
'fill_price': '17.05',
'filled_total': '17.05',
'avg_deal_price': '15.5',
'fee': '0',
'fee_currency': 'SOL',
'point_fee': '0.0199999999967544',
'gt_fee': '0',
'gt_maker_fee': '0',
'gt_taker_fee': '0',
'gt_discount': False,
'rebated_fee': '0',
'rebated_fee_currency': 'USDT'
}
],
},
'okx': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': True,
'timeframe': '1h',
'futures': True,
'futures_pair': 'BTC/USDT:USDT',
'hasQuoteVolumeFutures': False,
'leverage_tiers_public': True,
'leverage_in_spot_market': True,
'private_methods': ['fetch_accounts'],
},
'bybit': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': True,
'use_ci_proxy': True,
'timeframe': '1h',
'futures_pair': 'BTC/USDT:USDT',
'futures': True,
'leverage_tiers_public': True,
'leverage_in_spot_market': True,
'sample_order': [
{
"orderId": "1274754916287346280",
"orderLinkId": "1666798627015730",
"symbol": "SOLUSDT",
"createTime": "1674493798550",
"orderPrice": "15.5",
"orderQty": "1.1",
"orderType": "LIMIT",
"side": "BUY",
"status": "NEW",
"timeInForce": "GTC",
"accountId": "5555555",
"execQty": "0",
"orderCategory": "0"
}
]
},
'huobi': {
'pair': 'ETH/BTC',
'stake_currency': 'BTC',
'hasQuoteVolume': True,
'timeframe': '1h',
'futures': False,
},
'bitvavo': {
'pair': 'BTC/EUR',
'stake_currency': 'EUR',
'hasQuoteVolume': True,
'timeframe': '1h',
'leverage_tiers_public': False,
'leverage_in_spot_market': False,
},
}
@pytest.fixture(scope="class")
def exchange_conf():
config = get_default_conf_usdt((Path(__file__).parent / "testdata").resolve())
config['exchange']['pair_whitelist'] = []
config['exchange']['key'] = ''
config['exchange']['secret'] = ''
config['dry_run'] = False
config['entry_pricing']['use_order_book'] = True
config['exit_pricing']['use_order_book'] = True
return config
def set_test_proxy(config: Config, use_proxy: bool) -> Config:
# Set proxy to test in CI.
import os
if use_proxy and (proxy := os.environ.get('CI_WEB_PROXY')):
config1 = deepcopy(config)
config1['exchange']['ccxt_config'] = {
"httpsProxy": proxy,
}
return config1
return config
def get_exchange(exchange_name, exchange_conf):
exchange_conf = set_test_proxy(
exchange_conf, EXCHANGES[exchange_name].get('use_ci_proxy', False))
exchange_conf['exchange']['name'] = exchange_name
exchange_conf['stake_currency'] = EXCHANGES[exchange_name]['stake_currency']
exchange = ExchangeResolver.load_exchange(exchange_conf, validate=True,
load_leverage_tiers=True)
yield exchange, exchange_name
def get_futures_exchange(exchange_name, exchange_conf, class_mocker):
if EXCHANGES[exchange_name].get('futures') is not True:
pytest.skip(f"Exchange {exchange_name} does not support futures.")
else:
exchange_conf = deepcopy(exchange_conf)
exchange_conf = set_test_proxy(
exchange_conf, EXCHANGES[exchange_name].get('use_ci_proxy', False))
exchange_conf['trading_mode'] = 'futures'
exchange_conf['margin_mode'] = 'isolated'
class_mocker.patch(
'freqtrade.exchange.binance.Binance.fill_leverage_tiers')
class_mocker.patch(f'{EXMS}.fetch_trading_fees')
class_mocker.patch('freqtrade.exchange.okx.Okx.additional_exchange_init')
class_mocker.patch('freqtrade.exchange.binance.Binance.additional_exchange_init')
class_mocker.patch('freqtrade.exchange.bybit.Bybit.additional_exchange_init')
class_mocker.patch(f'{EXMS}.load_cached_leverage_tiers', return_value=None)
class_mocker.patch(f'{EXMS}.cache_leverage_tiers')
yield from get_exchange(exchange_name, exchange_conf)
@pytest.fixture(params=EXCHANGES, scope="class")
def exchange(request, exchange_conf):
yield from get_exchange(request.param, exchange_conf)
@pytest.fixture(params=EXCHANGES, scope="class")
def exchange_futures(request, exchange_conf, class_mocker):
yield from get_futures_exchange(request.param, exchange_conf, class_mocker)

421
tests/exchange/test_ccxt_compat.py → tests/exchange_online/test_ccxt_compat.py
View File

@@ -5,338 +5,14 @@ However, these tests should give a good idea to determine if a new exchange is
suitable to run with freqtrade.
"""
from copy import deepcopy
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Tuple
import pytest
from freqtrade.constants import Config
from freqtrade.enums import CandleType
from freqtrade.exchange import timeframe_to_minutes, timeframe_to_prev_date
from freqtrade.exchange.exchange import Exchange, timeframe_to_msecs
from freqtrade.resolvers.exchange_resolver import ExchangeResolver
from tests.conftest import EXMS, get_default_conf_usdt
EXCHANGE_FIXTURE_TYPE = Tuple[Exchange, str]
# Exchanges that should be tested
EXCHANGES = {
'bittrex': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': False,
'timeframe': '1h',
'leverage_tiers_public': False,
'leverage_in_spot_market': False,
},
'binance': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'use_ci_proxy': True,
'hasQuoteVolume': True,
'timeframe': '1h',
'futures': True,
'futures_pair': 'BTC/USDT:USDT',
'hasQuoteVolumeFutures': True,
'leverage_tiers_public': False,
'leverage_in_spot_market': False,
'trades_lookback_hours': 4,
'private_methods': [
'fapiPrivateGetPositionSideDual',
'fapiPrivateGetMultiAssetsMargin'
],
'sample_order': [{
"symbol": "SOLUSDT",
"orderId": 3551312894,
"orderListId": -1,
"clientOrderId": "x-R4DD3S8297c73a11ccb9dc8f2811ba",
"transactTime": 1674493798550,
"price": "15.50000000",
"origQty": "1.10000000",
"executedQty": "0.00000000",
"cummulativeQuoteQty": "0.00000000",
"status": "NEW",
"timeInForce": "GTC",
"type": "LIMIT",
"side": "BUY",
"workingTime": 1674493798550,
"fills": [],
"selfTradePreventionMode": "NONE",
}]
},
'binanceus': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': True,
'timeframe': '1h',
'futures': False,
'sample_order': [{
"symbol": "SOLUSDT",
"orderId": 3551312894,
"orderListId": -1,
"clientOrderId": "x-R4DD3S8297c73a11ccb9dc8f2811ba",
"transactTime": 1674493798550,
"price": "15.50000000",
"origQty": "1.10000000",
"executedQty": "0.00000000",
"cummulativeQuoteQty": "0.00000000",
"status": "NEW",
"timeInForce": "GTC",
"type": "LIMIT",
"side": "BUY",
"workingTime": 1674493798550,
"fills": [],
"selfTradePreventionMode": "NONE",
}]
},
'kraken': {
'pair': 'BTC/USD',
'stake_currency': 'USD',
'hasQuoteVolume': True,
'timeframe': '1h',
'leverage_tiers_public': False,
'leverage_in_spot_market': True,
'trades_lookback_hours': 12,
},
'kucoin': {
'pair': 'XRP/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': True,
'timeframe': '1h',
'leverage_tiers_public': False,
'leverage_in_spot_market': True,
'sample_order': [
{'id': '63d6742d0adc5570001d2bbf7'}, # create order
{
'id': '63d6742d0adc5570001d2bbf7',
'symbol': 'SOL-USDT',
'opType': 'DEAL',
'type': 'limit',
'side': 'buy',
'price': '15.5',
'size': '1.1',
'funds': '0',
'dealFunds': '17.05',
'dealSize': '1.1',
'fee': '0.000065252',
'feeCurrency': 'USDT',
'stp': '',
'stop': '',
'stopTriggered': False,
'stopPrice': '0',
'timeInForce': 'GTC',
'postOnly': False,
'hidden': False,
'iceberg': False,
'visibleSize': '0',
'cancelAfter': 0,
'channel': 'API',
'clientOid': '0a053870-11bf-41e5-be61-b272a4cb62e1',
'remark': None,
'tags': 'partner:ccxt',
'isActive': False,
'cancelExist': False,
'createdAt': 1674493798550,
'tradeType': 'TRADE'
}],
},
'gate': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': True,
'timeframe': '1h',
'futures': True,
'futures_pair': 'BTC/USDT:USDT',
'hasQuoteVolumeFutures': True,
'leverage_tiers_public': True,
'leverage_in_spot_market': True,
'sample_order': [
{
"id": "276266139423",
"text": "apiv4",
"create_time": "1674493798",
"update_time": "1674493798",
"create_time_ms": "1674493798550",
"update_time_ms": "1674493798550",
"status": "closed",
"currency_pair": "SOL_USDT",
"type": "limit",
"account": "spot",
"side": "buy",
"amount": "1.1",
"price": "15.5",
"time_in_force": "gtc",
"iceberg": "0",
"left": "0",
"fill_price": "17.05",
"filled_total": "17.05",
"avg_deal_price": "15.5",
"fee": "0.0000018",
"fee_currency": "SOL",
"point_fee": "0",
"gt_fee": "0",
"gt_maker_fee": "0",
"gt_taker_fee": "0.0015",
"gt_discount": True,
"rebated_fee": "0",
"rebated_fee_currency": "USDT"
},
{
# market order
'id': '276401180529',
'text': 'apiv4',
'create_time': '1674493798',
'update_time': '1674493798',
'create_time_ms': '1674493798550',
'update_time_ms': '1674493798550',
'status': 'cancelled',
'currency_pair': 'SOL_USDT',
'type': 'market',
'account': 'spot',
'side': 'buy',
'amount': '17.05',
'price': '0',
'time_in_force': 'ioc',
'iceberg': '0',
'left': '0.0000000016228',
'fill_price': '17.05',
'filled_total': '17.05',
'avg_deal_price': '15.5',
'fee': '0',
'fee_currency': 'SOL',
'point_fee': '0.0199999999967544',
'gt_fee': '0',
'gt_maker_fee': '0',
'gt_taker_fee': '0',
'gt_discount': False,
'rebated_fee': '0',
'rebated_fee_currency': 'USDT'
}
],
},
'okx': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': True,
'timeframe': '1h',
'futures': True,
'futures_pair': 'BTC/USDT:USDT',
'hasQuoteVolumeFutures': False,
'leverage_tiers_public': True,
'leverage_in_spot_market': True,
'private_methods': ['fetch_accounts'],
},
'bybit': {
'pair': 'BTC/USDT',
'stake_currency': 'USDT',
'hasQuoteVolume': True,
'use_ci_proxy': True,
'timeframe': '1h',
'futures_pair': 'BTC/USDT:USDT',
'futures': True,
'leverage_tiers_public': True,
'leverage_in_spot_market': True,
'sample_order': [
{
"orderId": "1274754916287346280",
"orderLinkId": "1666798627015730",
"symbol": "SOLUSDT",
"createTime": "1674493798550",
"orderPrice": "15.5",
"orderQty": "1.1",
"orderType": "LIMIT",
"side": "BUY",
"status": "NEW",
"timeInForce": "GTC",
"accountId": "5555555",
"execQty": "0",
"orderCategory": "0"
}
]
},
'huobi': {
'pair': 'ETH/BTC',
'stake_currency': 'BTC',
'hasQuoteVolume': True,
'timeframe': '1h',
'futures': False,
},
'bitvavo': {
'pair': 'BTC/EUR',
'stake_currency': 'EUR',
'hasQuoteVolume': True,
'timeframe': '1h',
'leverage_tiers_public': False,
'leverage_in_spot_market': False,
},
}
@pytest.fixture(scope="class")
def exchange_conf():
config = get_default_conf_usdt((Path(__file__).parent / "testdata").resolve())
config['exchange']['pair_whitelist'] = []
config['exchange']['key'] = ''
config['exchange']['secret'] = ''
config['dry_run'] = False
config['entry_pricing']['use_order_book'] = True
config['exit_pricing']['use_order_book'] = True
return config
def set_test_proxy(config: Config, use_proxy: bool) -> Config:
# Set proxy to test in CI.
import os
if use_proxy and (proxy := os.environ.get('CI_WEB_PROXY')):
config1 = deepcopy(config)
config1['exchange']['ccxt_config'] = {
"httpsProxy": proxy,
}
return config1
return config
@pytest.fixture(params=EXCHANGES, scope="class")
def exchange(request, exchange_conf):
exchange_conf = set_test_proxy(
exchange_conf, EXCHANGES[request.param].get('use_ci_proxy', False))
exchange_conf['exchange']['name'] = request.param
exchange_conf['stake_currency'] = EXCHANGES[request.param]['stake_currency']
exchange = ExchangeResolver.load_exchange(exchange_conf, validate=True)
yield exchange, request.param
@pytest.fixture(params=EXCHANGES, scope="class")
def exchange_futures(request, exchange_conf, class_mocker):
if EXCHANGES[request.param].get('futures') is not True:
yield None, request.param
else:
exchange_conf = set_test_proxy(
exchange_conf, EXCHANGES[request.param].get('use_ci_proxy', False))
exchange_conf = deepcopy(exchange_conf)
exchange_conf['exchange']['name'] = request.param
exchange_conf['trading_mode'] = 'futures'
exchange_conf['margin_mode'] = 'isolated'
exchange_conf['stake_currency'] = EXCHANGES[request.param]['stake_currency']
class_mocker.patch(
'freqtrade.exchange.binance.Binance.fill_leverage_tiers')
class_mocker.patch(f'{EXMS}.fetch_trading_fees')
class_mocker.patch('freqtrade.exchange.okx.Okx.additional_exchange_init')
class_mocker.patch('freqtrade.exchange.binance.Binance.additional_exchange_init')
class_mocker.patch('freqtrade.exchange.bybit.Bybit.additional_exchange_init')
class_mocker.patch(f'{EXMS}.load_cached_leverage_tiers', return_value=None)
class_mocker.patch(f'{EXMS}.cache_leverage_tiers')
exchange = ExchangeResolver.load_exchange(
exchange_conf, validate=True, load_leverage_tiers=True)
yield exchange, request.param
from freqtrade.exchange.exchange import timeframe_to_msecs
from tests.exchange_online.conftest import EXCHANGE_FIXTURE_TYPE, EXCHANGES
@pytest.mark.longrun
@@ -371,9 +47,6 @@ class TestCCXTExchange:
def test_load_markets_futures(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange_futures
if not exchange:
# exchange_futures only returns values for supported exchanges
return
pair = EXCHANGES[exchangename]['pair']
pair = EXCHANGES[exchangename].get('futures_pair', pair)
markets = exchange.markets
@@ -391,7 +64,7 @@ class TestCCXTExchange:
assert po['id'] is not None
if len(order.keys()) < 5:
# Kucoin case
assert po['status'] == 'closed'
assert po['status'] is None
continue
assert po['timestamp'] == 1674493798550
assert isinstance(po['datetime'], str)
@@ -511,7 +184,8 @@ class TestCCXTExchange:
now = datetime.now(timezone.utc) - timedelta(minutes=(timeframe_to_minutes(timeframe) * 2))
assert exch.klines(pair_tf).iloc[-1]['date'] >= timeframe_to_prev_date(timeframe, now)
def ccxt__async_get_candle_history(self, exchange, exchangename, pair, timeframe, candle_type):
def ccxt__async_get_candle_history(
self, exchange, exchangename, pair, timeframe, candle_type, factor=0.9):
timeframe_ms = timeframe_to_msecs(timeframe)
now = timeframe_to_prev_date(
@@ -532,11 +206,11 @@ class TestCCXTExchange:
assert res[1] == timeframe
assert res[2] == candle_type
candles = res[3]
factor = 0.9
candle_count = exchange.ohlcv_candle_limit(timeframe, candle_type, since_ms) * factor
candle_count1 = (now.timestamp() * 1000 - since_ms) // timeframe_ms * factor
assert len(candles) >= min(candle_count, candle_count1), \
f"{len(candles)} < {candle_count} in {timeframe}, Offset: {offset} {factor}"
# Check if first-timeframe is either the start, or start + 1
assert candles[0][0] == since_ms or (since_ms + timeframe_ms)
def test_ccxt__async_get_candle_history(self, exchange: EXCHANGE_FIXTURE_TYPE):
@@ -544,8 +218,6 @@ class TestCCXTExchange:
if exchangename in ('bittrex'):
# For some weired reason, this test returns random lengths for bittrex.
pytest.skip("Exchange doesn't provide stable ohlcv history")
if exchangename in ('bitvavo'):
pytest.skip("Exchange Downtime ")
if not exc._ft_has['ohlcv_has_history']:
pytest.skip("Exchange does not support candle history")
@@ -554,21 +226,29 @@ class TestCCXTExchange:
self.ccxt__async_get_candle_history(
exc, exchangename, pair, timeframe, CandleType.SPOT)
def test_ccxt__async_get_candle_history_futures(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
@pytest.mark.parametrize('candle_type', [
CandleType.FUTURES,
CandleType.FUNDING_RATE,
CandleType.MARK,
])
def test_ccxt__async_get_candle_history_futures(
self, exchange_futures: EXCHANGE_FIXTURE_TYPE, candle_type):
exchange, exchangename = exchange_futures
if not exchange:
# exchange_futures only returns values for supported exchanges
return
pair = EXCHANGES[exchangename].get('futures_pair', EXCHANGES[exchangename]['pair'])
timeframe = EXCHANGES[exchangename]['timeframe']
if candle_type == CandleType.FUNDING_RATE:
timeframe = exchange._ft_has.get('funding_fee_timeframe',
exchange._ft_has['mark_ohlcv_timeframe'])
self.ccxt__async_get_candle_history(
exchange, exchangename, pair, timeframe, CandleType.FUTURES)
exchange,
exchangename,
pair=pair,
timeframe=timeframe,
candle_type=candle_type,
)
def test_ccxt_fetch_funding_rate_history(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange_futures
if not exchange:
# exchange_futures only returns values for supported exchanges
return
pair = EXCHANGES[exchangename].get('futures_pair', EXCHANGES[exchangename]['pair'])
since = int((datetime.now(timezone.utc) - timedelta(days=5)).timestamp() * 1000)
@@ -604,9 +284,6 @@ class TestCCXTExchange:
def test_ccxt_fetch_mark_price_history(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange_futures
if not exchange:
# exchange_futures only returns values for supported exchanges
return
pair = EXCHANGES[exchangename].get('futures_pair', EXCHANGES[exchangename]['pair'])
since = int((datetime.now(timezone.utc) - timedelta(days=5)).timestamp() * 1000)
pair_tf = (pair, '1h', CandleType.MARK)
@@ -628,9 +305,6 @@ class TestCCXTExchange:
def test_ccxt__calculate_funding_fees(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange_futures
if not exchange:
# exchange_futures only returns values for supported exchanges
return
pair = EXCHANGES[exchangename].get('futures_pair', EXCHANGES[exchangename]['pair'])
since = datetime.now(timezone.utc) - timedelta(days=5)
@@ -677,31 +351,29 @@ class TestCCXTExchange:
def test_ccxt_get_max_leverage_futures(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures
if futures:
leverage_tiers_public = EXCHANGES[futures_name].get('leverage_tiers_public')
if leverage_tiers_public:
futures_pair = EXCHANGES[futures_name].get(
'futures_pair',
EXCHANGES[futures_name]['pair']
)
futures_leverage = futures.get_max_leverage(futures_pair, 20)
assert (isinstance(futures_leverage, float) or isinstance(futures_leverage, int))
assert futures_leverage >= 1.0
def test_ccxt_get_contract_size(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures
if futures:
leverage_tiers_public = EXCHANGES[futures_name].get('leverage_tiers_public')
if leverage_tiers_public:
futures_pair = EXCHANGES[futures_name].get(
'futures_pair',
EXCHANGES[futures_name]['pair']
)
contract_size = futures.get_contract_size(futures_pair)
assert (isinstance(contract_size, float) or isinstance(contract_size, int))
assert contract_size >= 0.0
futures_leverage = futures.get_max_leverage(futures_pair, 20)
assert (isinstance(futures_leverage, float) or isinstance(futures_leverage, int))
assert futures_leverage >= 1.0
def test_ccxt_get_contract_size(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures
futures_pair = EXCHANGES[futures_name].get(
'futures_pair',
EXCHANGES[futures_name]['pair']
)
contract_size = futures.get_contract_size(futures_pair)
assert (isinstance(contract_size, float) or isinstance(contract_size, int))
assert contract_size >= 0.0
def test_ccxt_load_leverage_tiers(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures
if futures and EXCHANGES[futures_name].get('leverage_tiers_public'):
if EXCHANGES[futures_name].get('leverage_tiers_public'):
leverage_tiers = futures.load_leverage_tiers()
futures_pair = EXCHANGES[futures_name].get(
'futures_pair',
@@ -734,7 +406,7 @@ class TestCCXTExchange:
def test_ccxt_dry_run_liquidation_price(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures
if futures and EXCHANGES[futures_name].get('leverage_tiers_public'):
if EXCHANGES[futures_name].get('leverage_tiers_public'):
futures_pair = EXCHANGES[futures_name].get(
'futures_pair',
@@ -767,14 +439,13 @@ class TestCCXTExchange:
def test_ccxt_get_max_pair_stake_amount(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures
if futures:
futures_pair = EXCHANGES[futures_name].get(
'futures_pair',
EXCHANGES[futures_name]['pair']
)
max_stake_amount = futures.get_max_pair_stake_amount(futures_pair, 40000)
assert (isinstance(max_stake_amount, float))
assert max_stake_amount >= 0.0
futures_pair = EXCHANGES[futures_name].get(
'futures_pair',
EXCHANGES[futures_name]['pair']
)
max_stake_amount = futures.get_max_pair_stake_amount(futures_pair, 40000)
assert (isinstance(max_stake_amount, float))
assert max_stake_amount >= 0.0
def test_private_method_presence(self, exchange: EXCHANGE_FIXTURE_TYPE):
exch, exchangename = exchange

4
tests/freqai/conftest.py
View File

@@ -97,9 +97,9 @@ def mock_pytorch_mlp_model_training_parameters() -> Dict[str, Any]:
return {
"learning_rate": 3e-4,
"trainer_kwargs": {
"max_iters": 1,
"n_steps": None,
"batch_size": 64,
"max_n_eval_batches": 1,
"n_epochs": 1,
},
"model_kwargs": {
"hidden_dim": 32,

17
tests/optimize/test_backtesting.py
View File

@@ -20,7 +20,7 @@ from freqtrade.data.dataprovider import DataProvider
from freqtrade.data.history import get_timerange
from freqtrade.enums import CandleType, ExitType, RunMode
from freqtrade.exceptions import DependencyException, OperationalException
from freqtrade.exchange.exchange import timeframe_to_next_date
from freqtrade.exchange import timeframe_to_next_date, timeframe_to_prev_date
from freqtrade.optimize.backtest_caching import get_backtest_metadata_filename, get_strategy_run_id
from freqtrade.optimize.backtesting import Backtesting
from freqtrade.persistence import LocalTrade, Trade
@@ -1122,10 +1122,10 @@ def test_backtest_dataprovider_analyzed_df(default_conf, fee, mocker, testdatadi
processed = backtesting.strategy.advise_all_indicators(data)
min_date, max_date = get_timerange(processed)
global count
count = 0
def tmp_confirm_entry(pair, current_time, **kwargs):
nonlocal count
dp = backtesting.strategy.dp
df, _ = dp.get_analyzed_dataframe(pair, backtesting.strategy.timeframe)
current_candle = df.iloc[-1].squeeze()
@@ -1135,8 +1135,13 @@ def test_backtest_dataprovider_analyzed_df(default_conf, fee, mocker, testdatadi
assert candle_date == current_time
# These asserts don't properly raise as they are nested,
# therefore we increment count and assert for that.
global count
count = count + 1
df = dp.get_pair_dataframe(pair, backtesting.strategy.timeframe)
prior_time = timeframe_to_prev_date(backtesting.strategy.timeframe,
candle_date - timedelta(seconds=1))
assert prior_time == df.iloc[-1].squeeze()['date']
assert df.iloc[-1].squeeze()['date'] < current_time
count += 1
backtesting.strategy.confirm_trade_entry = tmp_confirm_entry
backtesting.backtest(
@@ -1354,11 +1359,11 @@ def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair, testdatadir)
# Cached data correctly removed amounts
offset = 1 if tres == 0 else 0
removed_candles = len(data[pair]) - offset - backtesting.strategy.startup_candle_count
removed_candles = len(data[pair]) - offset
assert len(backtesting.dataprovider.get_analyzed_dataframe(pair, '5m')[0]) == removed_candles
assert len(
backtesting.dataprovider.get_analyzed_dataframe('NXT/BTC', '5m')[0]
) == len(data['NXT/BTC']) - 1 - backtesting.strategy.startup_candle_count
) == len(data['NXT/BTC']) - 1
backtesting.strategy.max_open_trades = 1
backtesting.config.update({'max_open_trades': 1})

24
tests/optimize/test_lookahead_analysis.py
View File

@@ -17,6 +17,8 @@ from tests.conftest import EXMS, get_args, log_has_re, patch_exchange
def lookahead_conf(default_conf_usdt):
default_conf_usdt['minimum_trade_amount'] = 10
default_conf_usdt['targeted_trade_amount'] = 20
default_conf_usdt['timerange'] = '20220101-20220501'
default_conf_usdt['strategy_path'] = str(
Path(__file__).parent.parent / "strategy/strats/lookahead_bias")
default_conf_usdt['strategy'] = 'strategy_test_v3_with_lookahead_bias'
@@ -43,7 +45,9 @@ def test_start_lookahead_analysis(mocker):
"--pairs",
"UNITTEST/BTC",
"--max-open-trades",
"1"
"1",
"--timerange",
"20220101-20220201"
]
pargs = get_args(args)
pargs['config'] = None
@@ -72,6 +76,24 @@ def test_start_lookahead_analysis(mocker):
match=r"Targeted trade amount can't be smaller than minimum trade amount.*"):
start_lookahead_analysis(pargs)
# Missing timerange
args = [
"lookahead-analysis",
"--strategy",
"strategy_test_v3_with_lookahead_bias",
"--strategy-path",
str(Path(__file__).parent.parent / "strategy/strats/lookahead_bias"),
"--pairs",
"UNITTEST/BTC",
"--max-open-trades",
"1",
]
pargs = get_args(args)
pargs['config'] = None
with pytest.raises(OperationalException,
match=r"Please set a timerange\..*"):
start_lookahead_analysis(pargs)
def test_lookahead_helper_invalid_config(lookahead_conf) -> None:
conf = deepcopy(lookahead_conf)

85
tests/rpc/test_rpc_apiserver.py
View File

@@ -10,6 +10,7 @@ from unittest.mock import ANY, MagicMock, PropertyMock
import pandas as pd
import pytest
import rapidjson
import uvicorn
from fastapi import FastAPI, WebSocketDisconnect
from fastapi.exceptions import HTTPException
@@ -80,6 +81,16 @@ def client_post(client: TestClient, url, data={}):
})
def client_patch(client: TestClient, url, data={}):
return client.patch(url,
json=data,
headers={'Authorization': _basic_auth_str(_TEST_USER, _TEST_PASS),
'Origin': 'http://example.com',
'content-type': 'application/json'
})
def client_get(client: TestClient, url):
# Add fake Origin to ensure CORS kicks in
return client.get(url, headers={'Authorization': _basic_auth_str(_TEST_USER, _TEST_PASS),
@@ -1763,7 +1774,7 @@ def test_api_pairlists_evaluate(botclient, tmpdir, mocker):
rc = client_get(client, f"{BASE_URI}/pairlists/evaluate/{job_id}")
assert_response(rc)
response = rc.json()
assert response['result']['whitelist'] == ['ETH/BTC', 'LTC/BTC', 'XRP/BTC', 'NEO/BTC',]
assert response['result']['whitelist'] == ['ETH/BTC', 'LTC/BTC', 'XRP/BTC', 'NEO/BTC']
assert response['result']['length'] == 4
# Restart with additional filter, reducing the list to 2
@@ -2010,6 +2021,7 @@ def test_api_backtest_history(botclient, mocker, testdatadir):
assert len(result) == 3
fn = result[0]['filename']
assert fn == "backtest-result_multistrat"
assert result[0]['notes'] == ''
strategy = result[0]['strategy']
rc = client_get(client, f"{BASE_URI}/backtest/history/result?filename={fn}&strategy={strategy}")
assert_response(rc)
@@ -2023,7 +2035,7 @@ def test_api_backtest_history(botclient, mocker, testdatadir):
assert result2['backtest_result']['strategy'][strategy]
def test_api_delete_backtest_history_entry(botclient, mocker, tmp_path: Path):
def test_api_delete_backtest_history_entry(botclient, tmp_path: Path):
ftbot, client = botclient
# Create a temporary directory and file
@@ -2051,6 +2063,75 @@ def test_api_delete_backtest_history_entry(botclient, mocker, tmp_path: Path):
assert not meta_path.exists()
def test_api_patch_backtest_history_entry(botclient, tmp_path: Path):
ftbot, client = botclient
# Create a temporary directory and file
bt_results_base = tmp_path / "backtest_results"
bt_results_base.mkdir()
file_path = bt_results_base / "test.json"
file_path.touch()
meta_path = file_path.with_suffix('.meta.json')
with meta_path.open('w') as metafile:
rapidjson.dump({
CURRENT_TEST_STRATEGY: {
"run_id": "6e542efc8d5e62cef6e5be0ffbc29be81a6e751d",
"backtest_start_time": 1690176003}
}, metafile)
def read_metadata():
with meta_path.open('r') as metafile:
return rapidjson.load(metafile)
rc = client_patch(client, f"{BASE_URI}/backtest/history/randomFile.json")
assert_response(rc, 503)
ftbot.config['user_data_dir'] = tmp_path
ftbot.config['runmode'] = RunMode.WEBSERVER
rc = client_patch(client, f"{BASE_URI}/backtest/history/randomFile.json", {
"strategy": CURRENT_TEST_STRATEGY,
})
assert rc.status_code == 404
# Nonexisting strategy
rc = client_patch(client, f"{BASE_URI}/backtest/history/{file_path.name}", {
"strategy": f"{CURRENT_TEST_STRATEGY}xxx",
})
assert rc.status_code == 400
assert rc.json()['detail'] == 'Strategy not in metadata.'
# no Notes
rc = client_patch(client, f"{BASE_URI}/backtest/history/{file_path.name}", {
"strategy": CURRENT_TEST_STRATEGY,
})
assert rc.status_code == 200
res = rc.json()
assert isinstance(res, list)
assert len(res) == 1
assert res[0]['strategy'] == CURRENT_TEST_STRATEGY
assert res[0]['notes'] == ''
fileres = read_metadata()
assert fileres[CURRENT_TEST_STRATEGY]['run_id'] == res[0]['run_id']
assert fileres[CURRENT_TEST_STRATEGY]['notes'] == ''
rc = client_patch(client, f"{BASE_URI}/backtest/history/{file_path.name}", {
"strategy": CURRENT_TEST_STRATEGY,
"notes": "FooBar",
})
assert rc.status_code == 200
res = rc.json()
assert isinstance(res, list)
assert len(res) == 1
assert res[0]['strategy'] == CURRENT_TEST_STRATEGY
assert res[0]['notes'] == 'FooBar'
fileres = read_metadata()
assert fileres[CURRENT_TEST_STRATEGY]['run_id'] == res[0]['run_id']
assert fileres[CURRENT_TEST_STRATEGY]['notes'] == 'FooBar'
def test_health(botclient):
ftbot, client = botclient

2
tests/strategy/strats/strategy_test_v3.py
View File

@@ -197,7 +197,7 @@ class StrategyTestV3(IStrategy):
if current_profit < -0.0075:
orders = trade.select_filled_orders(trade.entry_side)
return round(orders[0].safe_cost, 0)
return round(orders[0].stake_amount, 0)
return None

12
tests/strategy/test_default_strategy.py
View File

@@ -25,13 +25,13 @@ def test_strategy_test_v3(dataframe_1m, fee, is_short, side):
strategy = StrategyTestV3({})
metadata = {'pair': 'ETH/BTC'}
assert type(strategy.minimal_roi) is dict
assert type(strategy.stoploss) is float
assert type(strategy.timeframe) is str
assert isinstance(strategy.minimal_roi, dict)
assert isinstance(strategy.stoploss, float)
assert isinstance(strategy.timeframe, str)
indicators = strategy.populate_indicators(dataframe_1m, metadata)
assert type(indicators) is DataFrame
assert type(strategy.populate_buy_trend(indicators, metadata)) is DataFrame
assert type(strategy.populate_sell_trend(indicators, metadata)) is DataFrame
assert isinstance(indicators, DataFrame)
assert isinstance(strategy.populate_buy_trend(indicators, metadata), DataFrame)
assert isinstance(strategy.populate_sell_trend(indicators, metadata), DataFrame)
trade = Trade(
open_rate=19_000,

3
tests/test_configuration.py
View File

@@ -1038,8 +1038,7 @@ def test_load_config_stoploss_exchange_limit_ratio(all_conf) -> None:
validate_config_schema(all_conf)
@pytest.mark.parametrize("keys", [("exchange", "sandbox", False),
("exchange", "key", ""),
@pytest.mark.parametrize("keys", [("exchange", "key", ""),
("exchange", "secret", ""),
("exchange", "password", ""),
])

15
tests/test_freqtradebot.py
View File

@@ -1508,15 +1508,15 @@ def test_handle_sle_cancel_cant_recreate(mocker, default_conf_usdt, fee, caplog,
@pytest.mark.parametrize("is_short", [False, True])
def test_create_stoploss_order_invalid_order(
mocker, default_conf_usdt, caplog, fee, is_short, limit_order, limit_order_open
mocker, default_conf_usdt, caplog, fee, is_short, limit_order
):
open_order = limit_order_open[entry_side(is_short)]
open_order = limit_order[entry_side(is_short)]
order = limit_order[exit_side(is_short)]
rpc_mock = patch_RPCManager(mocker)
patch_exchange(mocker)
create_order_mock = MagicMock(side_effect=[
open_order,
{'id': order['id']}
order,
])
mocker.patch.multiple(
EXMS,
@@ -1541,6 +1541,7 @@ def test_create_stoploss_order_invalid_order(
trade = Trade.session.scalars(select(Trade)).first()
trade.is_short = is_short
caplog.clear()
rpc_mock.reset_mock()
freqtrade.create_stoploss_order(trade, 200)
assert trade.stoploss_order_id is None
assert trade.exit_reason == ExitType.EMERGENCY_EXIT.value
@@ -1554,9 +1555,11 @@ def test_create_stoploss_order_invalid_order(
assert create_order_mock.call_args[1]['amount'] == trade.amount
# Rpc is sending first buy, then sell
assert rpc_mock.call_count == 3
assert rpc_mock.call_args_list[2][0][0]['sell_reason'] == ExitType.EMERGENCY_EXIT.value
assert rpc_mock.call_args_list[2][0][0]['order_type'] == 'market'
assert rpc_mock.call_count == 2
assert rpc_mock.call_args_list[0][0][0]['sell_reason'] == ExitType.EMERGENCY_EXIT.value
assert rpc_mock.call_args_list[0][0][0]['order_type'] == 'market'
assert rpc_mock.call_args_list[0][0][0]['type'] == 'exit'
assert rpc_mock.call_args_list[1][0][0]['type'] == 'exit_fill'
@pytest.mark.parametrize("is_short", [False, True])

17
tests/test_misc.py
View File

@@ -9,8 +9,7 @@ import pytest
from freqtrade.misc import (dataframe_to_json, decimals_per_coin, deep_merge_dicts, file_dump_json,
file_load_json, is_file_in_dir, json_to_dataframe, pair_to_filename,
parse_db_uri_for_logging, plural, render_template,
render_template_with_fallback, round_coin_value, safe_value_fallback,
parse_db_uri_for_logging, plural, round_coin_value, safe_value_fallback,
safe_value_fallback2)
@@ -177,20 +176,6 @@ def test_plural() -> None:
assert plural(-1.5, "ox", "oxen") == "oxen"
def test_render_template_fallback(mocker):
from jinja2.exceptions import TemplateNotFound
with pytest.raises(TemplateNotFound):
val = render_template(
templatefile='subtemplates/indicators_does-not-exist.j2',)
val = render_template_with_fallback(
templatefile='strategy_subtemplates/indicators_does-not-exist.j2',
templatefallbackfile='strategy_subtemplates/indicators_minimal.j2',
)
assert isinstance(val, str)
assert 'if self.dp' in val
@pytest.mark.parametrize('conn_url,expected', [
("postgresql+psycopg2://scott123:scott123@host:1245/dbname",
"postgresql+psycopg2://scott123:*****@host:1245/dbname"),

2
tests/utils/test_datetime_helpers.py
View File

@@ -63,7 +63,7 @@ def test_format_ms_time() -> None:
# Date 2018-04-10 18:02:01
date_in_epoch_ms = 1523383321000
date = format_ms_time(date_in_epoch_ms)
assert type(date) is str
assert isinstance(date, str)
res = datetime(2018, 4, 10, 18, 2, 1, tzinfo=timezone.utc)
assert date == res.astimezone(None).strftime('%Y-%m-%dT%H:%M:%S')
res = datetime(2017, 12, 13, 8, 2, 1, tzinfo=timezone.utc)

17
tests/utils/test_rendering_utils.py Normal file
View File

@@ -0,0 +1,17 @@
import pytest
from freqtrade.util import render_template, render_template_with_fallback
def test_render_template_fallback():
from jinja2.exceptions import TemplateNotFound
with pytest.raises(TemplateNotFound):
val = render_template(
templatefile='subtemplates/indicators_does-not-exist.j2',)
val = render_template_with_fallback(
templatefile='strategy_subtemplates/indicators_does-not-exist.j2',
templatefallbackfile='strategy_subtemplates/indicators_minimal.j2',
)
assert isinstance(val, str)
assert 'if self.dp' in val