CTYPTO/freqtrade
1
0
Fork 0
mirror of https://github.com/freqtrade/freqtrade.git synced 2026-01-31 11:20:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'freqtrade:develop' into develop

Browse Source
This commit is contained in:
Richard Jozsa
2023-05-22 08:36:25 +00:00
committed by GitHub
parent d26aa231fc 6d9b8a4a99
commit 39f4fb8797
24 changed files with 460 additions and 362 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
docker-compose.yml
View File

@@ -6,6 +6,15 @@ services:
# image: freqtradeorg/freqtrade:develop
# Use plotting image
# image: freqtradeorg/freqtrade:develop_plot
# # Enable GPU Image and GPU Resources (only relevant for freqAI)
# # Make sure to uncomment the whole deploy section
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: 1
# capabilities: [gpu]
# Build step - only needed when additional dependencies are needed
# build:
# context: .
@@ -16,7 +25,7 @@ services:
- "./user_data:/freqtrade/user_data"
# Expose api on port 8080 (localhost only)
# Please read the https://www.freqtrade.io/en/stable/rest-api/ documentation
# before enabling this.
# for more information.
ports:
- "127.0.0.1:8080:8080"
# Default command used when running `docker compose up`

36
docker/docker-compose-freqai.yml Normal file
View File

@@ -0,0 +1,36 @@
---
version: '3'
services:
freqtrade:
image: freqtradeorg/freqtrade:stable_freqaitorch
# # Enable GPU Image and GPU Resources
# # Make sure to uncomment the whole deploy section
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: 1
# capabilities: [gpu]
# Build step - only needed when additional dependencies are needed
# build:
# context: .
# dockerfile: "./docker/Dockerfile.custom"
restart: unless-stopped
container_name: freqtrade
volumes:
- "./user_data:/freqtrade/user_data"
# Expose api on port 8080 (localhost only)
# Please read the https://www.freqtrade.io/en/stable/rest-api/ documentation
# for more information.
ports:
- "127.0.0.1:8080:8080"
# Default command used when running `docker compose up`
command: >
trade
--logfile /freqtrade/user_data/logs/freqtrade.log
--db-url sqlite:////freqtrade/user_data/tradesv3.sqlite
--config /freqtrade/user_data/config.json
--freqai-model XGBoostClassifier
--strategy SampleStrategy

4
docs/freqai-configuration.md
View File

@@ -248,9 +248,11 @@ The easiest way to quickly run a pytorch model is with the following command (fo
freqtrade trade --config config_examples/config_freqai.example.json --strategy FreqaiExampleStrategy --freqaimodel PyTorchMLPRegressor --strategy-path freqtrade/templates
```
!!! note "Installation/docker"
!!! Note "Installation/docker"
The PyTorch module requires large packages such as `torch`, which should be explicitly requested during `./setup.sh -i` by answering "y" to the question "Do you also want dependencies for freqai-rl or PyTorch (~700mb additional space required) [y/N]?".
Users who prefer docker should ensure they use the docker image appended with `_freqaitorch`.
We do provide an explicit docker-compose file for this in `docker/docker-compose-freqai.yml` - which can be used via `docker compose -f docker/docker-compose-freqai.yml run ...` - or can be copied to replace the original docker file.
This docker-compose file also contains a (disabled) section to enable GPU resources within docker containers. This obviously assumes the system has GPU resources available.
### Structure

180
docs/freqai-reinforcement-learning.md
View File

@@ -145,94 +145,94 @@ As you begin to modify the strategy and the prediction model, you will quickly r
The best reward functions are ones that are continuously differentiable, and well scaled. In other words, adding a single large negative penalty to a rare event is not a good idea, and the neural net will not be able to learn that function. Instead, it is better to add a small negative penalty to a common event. This will help the agent learn faster. Not only this, but you can help improve the continuity of your rewards/penalties by having them scale with severity according to some linear/exponential functions. In other words, you'd slowly scale the penalty as the duration of the trade increases. This is better than a single large penalty occuring at a single point in time.
```python
from freqtrade.freqai.prediction_models.ReinforcementLearner import ReinforcementLearner
from freqtrade.freqai.RL.Base5ActionRLEnv import Actions, Base5ActionRLEnv, Positions
from freqtrade.freqai.prediction_models.ReinforcementLearner import ReinforcementLearner
from freqtrade.freqai.RL.Base5ActionRLEnv import Actions, Base5ActionRLEnv, Positions
class MyCoolRLModel(ReinforcementLearner):
class MyCoolRLModel(ReinforcementLearner):
"""
User created RL prediction model.
Save this file to `freqtrade/user_data/freqaimodels`
then use it with:
freqtrade trade --freqaimodel MyCoolRLModel --config config.json --strategy SomeCoolStrat
Here the users can override any of the functions
available in the `IFreqaiModel` inheritance tree. Most importantly for RL, this
is where the user overrides `MyRLEnv` (see below), to define custom
`calculate_reward()` function, or to override any other parts of the environment.
This class also allows users to override any other part of the IFreqaiModel tree.
For example, the user can override `def fit()` or `def train()` or `def predict()`
to take fine-tuned control over these processes.
Another common override may be `def data_cleaning_predict()` where the user can
take fine-tuned control over the data handling pipeline.
"""
class MyRLEnv(Base5ActionRLEnv):
"""
User created RL prediction model.
User made custom environment. This class inherits from BaseEnvironment and gym.env.
Users can override any functions from those parent classes. Here is an example
of a user customized `calculate_reward()` function.
Save this file to `freqtrade/user_data/freqaimodels`
then use it with:
freqtrade trade --freqaimodel MyCoolRLModel --config config.json --strategy SomeCoolStrat
Here the users can override any of the functions
available in the `IFreqaiModel` inheritance tree. Most importantly for RL, this
is where the user overrides `MyRLEnv` (see below), to define custom
`calculate_reward()` function, or to override any other parts of the environment.
This class also allows users to override any other part of the IFreqaiModel tree.
For example, the user can override `def fit()` or `def train()` or `def predict()`
to take fine-tuned control over these processes.
Another common override may be `def data_cleaning_predict()` where the user can
take fine-tuned control over the data handling pipeline.
Warning!
This is function is a showcase of functionality designed to show as many possible
environment control features as possible. It is also designed to run quickly
on small computers. This is a benchmark, it is *not* for live production.
"""
class MyRLEnv(Base5ActionRLEnv):
"""
User made custom environment. This class inherits from BaseEnvironment and gym.env.
Users can override any functions from those parent classes. Here is an example
of a user customized `calculate_reward()` function.
def calculate_reward(self, action: int) -> float:
# first, penalize if the action is not valid
if not self._is_valid(action):
return -2
pnl = self.get_unrealized_profit()
Warning!
This is function is a showcase of functionality designed to show as many possible
environment control features as possible. It is also designed to run quickly
on small computers. This is a benchmark, it is *not* for live production.
"""
def calculate_reward(self, action: int) -> float:
# first, penalize if the action is not valid
if not self._is_valid(action):
return -2
pnl = self.get_unrealized_profit()
factor = 100
factor = 100
pair = self.pair.replace(':', '')
pair = self.pair.replace(':', '')
# you can use feature values from dataframe
# Assumes the shifted RSI indicator has been generated in the strategy.
rsi_now = self.raw_features[f"%-rsi-period_10_shift-1_{pair}_"
f"{self.config['timeframe']}"].iloc[self._current_tick]
# you can use feature values from dataframe
# Assumes the shifted RSI indicator has been generated in the strategy.
rsi_now = self.raw_features[f"%-rsi-period_10_shift-1_{pair}_"
f"{self.config['timeframe']}"].iloc[self._current_tick]
# reward agent for entering trades
if (action in (Actions.Long_enter.value, Actions.Short_enter.value)
and self._position == Positions.Neutral):
if rsi_now < 40:
factor = 40 / rsi_now
else:
factor = 1
return 25 * factor
# reward agent for entering trades
if (action in (Actions.Long_enter.value, Actions.Short_enter.value)
and self._position == Positions.Neutral):
if rsi_now < 40:
factor = 40 / rsi_now
else:
factor = 1
return 25 * factor
# discourage agent from not entering trades
if action == Actions.Neutral.value and self._position == Positions.Neutral:
return -1
max_trade_duration = self.rl_config.get('max_trade_duration_candles', 300)
trade_duration = self._current_tick - self._last_trade_tick
if trade_duration <= max_trade_duration:
factor *= 1.5
elif trade_duration > max_trade_duration:
factor *= 0.5
# discourage sitting in position
if self._position in (Positions.Short, Positions.Long) and \
action == Actions.Neutral.value:
return -1 * trade_duration / max_trade_duration
# close long
if action == Actions.Long_exit.value and self._position == Positions.Long:
if pnl > self.profit_aim * self.rr:
factor *= self.rl_config['model_reward_parameters'].get('win_reward_factor', 2)
return float(pnl * factor)
# close short
if action == Actions.Short_exit.value and self._position == Positions.Short:
if pnl > self.profit_aim * self.rr:
factor *= self.rl_config['model_reward_parameters'].get('win_reward_factor', 2)
return float(pnl * factor)
return 0.
# discourage agent from not entering trades
if action == Actions.Neutral.value and self._position == Positions.Neutral:
return -1
max_trade_duration = self.rl_config.get('max_trade_duration_candles', 300)
trade_duration = self._current_tick - self._last_trade_tick
if trade_duration <= max_trade_duration:
factor *= 1.5
elif trade_duration > max_trade_duration:
factor *= 0.5
# discourage sitting in position
if self._position in (Positions.Short, Positions.Long) and \
action == Actions.Neutral.value:
return -1 * trade_duration / max_trade_duration
# close long
if action == Actions.Long_exit.value and self._position == Positions.Long:
if pnl > self.profit_aim * self.rr:
factor *= self.rl_config['model_reward_parameters'].get('win_reward_factor', 2)
return float(pnl * factor)
# close short
if action == Actions.Short_exit.value and self._position == Positions.Short:
if pnl > self.profit_aim * self.rr:
factor *= self.rl_config['model_reward_parameters'].get('win_reward_factor', 2)
return float(pnl * factor)
return 0.
```
### Using Tensorboard
## Using Tensorboard
Reinforcement Learning models benefit from tracking training metrics. FreqAI has integrated Tensorboard to allow users to track training and evaluation performance across all coins and across all retrainings. Tensorboard is activated via the following command:
@@ -245,32 +245,30 @@ where `unique-id` is the `identifier` set in the `freqai` configuration file. Th
![tensorboard](assets/tensorboard.jpg)
### Custom logging
## Custom logging
FreqAI also provides a built in episodic summary logger called `self.tensorboard_log` for adding custom information to the Tensorboard log. By default, this function is already called once per step inside the environment to record the agent actions. All values accumulated for all steps in a single episode are reported at the conclusion of each episode, followed by a full reset of all metrics to 0 in preparation for the subsequent episode.
`self.tensorboard_log` can also be used anywhere inside the environment, for example, it can be added to the `calculate_reward` function to collect more detailed information about how often various parts of the reward were called:
```py
class MyRLEnv(Base5ActionRLEnv):
"""
User made custom environment. This class inherits from BaseEnvironment and gym.env.
Users can override any functions from those parent classes. Here is an example
of a user customized `calculate_reward()` function.
"""
def calculate_reward(self, action: int) -> float:
if not self._is_valid(action):
self.tensorboard_log("invalid")
return -2
```python
class MyRLEnv(Base5ActionRLEnv):
"""
User made custom environment. This class inherits from BaseEnvironment and gym.env.
Users can override any functions from those parent classes. Here is an example
of a user customized `calculate_reward()` function.
"""
def calculate_reward(self, action: int) -> float:
if not self._is_valid(action):
self.tensorboard_log("invalid")
return -2
```
!!! Note
The `self.tensorboard_log()` function is designed for tracking incremented objects only i.e. events, actions inside the training environment. If the event of interest is a float, the float can be passed as the second argument e.g. `self.tensorboard_log("float_metric1", 0.23)`. In this case the metric values are not incremented.
### Choosing a base environment
## Choosing a base environment
FreqAI provides three base environments, `Base3ActionRLEnvironment`, `Base4ActionEnvironment` and `Base5ActionEnvironment`. As the names imply, the environments are customized for agents that can select from 3, 4 or 5 actions. The `Base3ActionEnvironment` is the simplest, the agent can select from hold, long, or short. This environment can also be used for long-only bots (it automatically follows the `can_short` flag from the strategy), where long is the enter condition and short is the exit condition. Meanwhile, in the `Base4ActionEnvironment`, the agent can enter long, enter short, hold neutral, or exit position. Finally, in the `Base5ActionEnvironment`, the agent has the same actions as Base4, but instead of a single exit action, it separates exit long and exit short. The main changes stemming from the environment selection include:

3
docs/freqai.md
View File

@@ -78,6 +78,9 @@ pip install -r requirements-freqai.txt
If you are using docker, a dedicated tag with FreqAI dependencies is available as `:freqai`. As such - you can replace the image line in your docker compose file with `image: freqtradeorg/freqtrade:develop_freqai`. This image contains the regular FreqAI dependencies. Similar to native installs, Catboost will not be available on ARM based devices.
!!! note "docker-compose-freqai.yml"
We do provide an explicit docker-compose file for this in `docker/docker-compose-freqai.yml` - which can be used via `docker compose -f docker/docker-compose-freqai.yml run ...` - or can be copied to replace the original docker file. This docker-compose file also contains a (disabled) section to enable GPU resources within docker containers. This obviously assumes the system has GPU resources available.
### FreqAI position in open-source machine learning landscape
Forecasting chaotic time-series based systems, such as equity/cryptocurrency markets, requires a broad set of tools geared toward testing a wide range of hypotheses. Fortunately, a recent maturation of robust machine learning libraries (e.g. `scikit-learn`) has opened up a wide range of research possibilities. Scientists from a diverse range of fields can now easily prototype their studies on an abundance of established machine learning algorithms. Similarly, these user-friendly libraries enable "citzen scientists" to use their basic Python skills for data exploration. However, leveraging these machine learning libraries on historical and live chaotic data sources can be logistically difficult and expensive. Additionally, robust data collection, storage, and handling presents a disparate challenge. [`FreqAI`](#freqai) aims to provide a generalized and extensible open-sourced framework geared toward live deployments of adaptive modeling for market forecasting. The `FreqAI` framework is effectively a sandbox for the rich world of open-source machine learning libraries. Inside the `FreqAI` sandbox, users find they can combine a wide variety of third-party libraries to test creative hypotheses on a free live 24/7 chaotic data source - cryptocurrency exchange data.

2
docs/requirements-docs.txt
View File

@@ -1,6 +1,6 @@
markdown==3.3.7
mkdocs==1.4.3
mkdocs-material==9.1.12
mkdocs-material==9.1.14
mdx_truly_sane_lists==1.3
pymdown-extensions==10.0.1
jinja2==3.1.2

2
freqtrade/constants.py
View File

@@ -690,4 +690,6 @@ BidAsk = Literal['bid', 'ask']
OBLiteral = Literal['asks', 'bids']
Config = Dict[str, Any]
# Exchange part of the configuration.
ExchangeConfig = Dict[str, Any]
IntOrInf = float

2
freqtrade/exchange/__init__.py
View File

@@ -1,6 +1,6 @@
# flake8: noqa: F401
# isort: off
from freqtrade.exchange.common import remove_credentials, MAP_EXCHANGE_CHILDCLASS
from freqtrade.exchange.common import remove_exchange_credentials, MAP_EXCHANGE_CHILDCLASS
from freqtrade.exchange.exchange import Exchange
# isort: on
from freqtrade.exchange.binance import Binance

16
freqtrade/exchange/common.py
View File

@@ -4,7 +4,7 @@ import time
from functools import wraps
from typing import Any, Callable, Optional, TypeVar, cast, overload
from freqtrade.constants import Config
from freqtrade.constants import ExchangeConfig
from freqtrade.exceptions import DDosProtection, RetryableOrderError, TemporaryError
from freqtrade.mixins import LoggingMixin
@@ -89,18 +89,18 @@ EXCHANGE_HAS_OPTIONAL = [
]
def remove_credentials(config: Config) -> None:
def remove_exchange_credentials(exchange_config: ExchangeConfig, dry_run: bool) -> None:
"""
Removes exchange keys from the configuration and specifies dry-run
Used for backtesting / hyperopt / edge and utils.
Modifies the input dict!
"""
if config.get('dry_run', False):
config['exchange']['key'] = ''
config['exchange']['apiKey'] = ''
config['exchange']['secret'] = ''
config['exchange']['password'] = ''
config['exchange']['uid'] = ''
if dry_run:
exchange_config['key'] = ''
exchange_config['apiKey'] = ''
exchange_config['secret'] = ''
exchange_config['password'] = ''
exchange_config['uid'] = ''
def calculate_backoff(retrycount, max_retries):

36
freqtrade/exchange/exchange.py
View File

@@ -20,16 +20,16 @@ from dateutil import parser
from pandas import DataFrame, concat
from freqtrade.constants import (DEFAULT_AMOUNT_RESERVE_PERCENT, NON_OPEN_EXCHANGE_STATES, BidAsk,
BuySell, Config, EntryExit, ListPairsWithTimeframes, MakerTaker,
OBLiteral, PairWithTimeframe)
BuySell, Config, EntryExit, ExchangeConfig,
ListPairsWithTimeframes, MakerTaker, OBLiteral, PairWithTimeframe)
from freqtrade.data.converter import clean_ohlcv_dataframe, ohlcv_to_dataframe, trades_dict_to_list
from freqtrade.enums import OPTIMIZE_MODES, CandleType, MarginMode, TradingMode
from freqtrade.enums.pricetype import PriceType
from freqtrade.exceptions import (DDosProtection, ExchangeError, InsufficientFundsError,
InvalidOrderException, OperationalException, PricingError,
RetryableOrderError, TemporaryError)
from freqtrade.exchange.common import (API_FETCH_ORDER_RETRY_COUNT, remove_credentials, retrier,
retrier_async)
from freqtrade.exchange.common import (API_FETCH_ORDER_RETRY_COUNT, remove_exchange_credentials,
retrier, retrier_async)
from freqtrade.exchange.exchange_utils import (ROUND, ROUND_DOWN, ROUND_UP, CcxtModuleType,
amount_to_contract_precision, amount_to_contracts,
amount_to_precision, contracts_to_amount,
@@ -92,8 +92,8 @@ class Exchange:
# TradingMode.SPOT always supported and not required in this list
]
def __init__(self, config: Config, *, validate: bool = True,
load_leverage_tiers: bool = False) -> None:
def __init__(self, config: Config, *, exchange_config: Optional[ExchangeConfig] = None,
validate: bool = True, load_leverage_tiers: bool = False) -> None:
"""
Initializes this module with the given config,
it does basic validation whether the specified exchange and pairs are valid.
@@ -131,13 +131,13 @@ class Exchange:
# Holds all open sell orders for dry_run
self._dry_run_open_orders: Dict[str, Any] = {}
remove_credentials(config)
if config['dry_run']:
logger.info('Instance is running with dry_run enabled')
logger.info(f"Using CCXT {ccxt.__version__}")
exchange_config = config['exchange']
self.log_responses = exchange_config.get('log_responses', False)
exchange_conf: Dict[str, Any] = exchange_config if exchange_config else config['exchange']
remove_exchange_credentials(exchange_conf, config.get('dry_run', False))
self.log_responses = exchange_conf.get('log_responses', False)
# Leverage properties
self.trading_mode: TradingMode = config.get('trading_mode', TradingMode.SPOT)
@@ -152,8 +152,8 @@ class Exchange:
self._ft_has = deep_merge_dicts(self._ft_has, deepcopy(self._ft_has_default))
if self.trading_mode == TradingMode.FUTURES:
self._ft_has = deep_merge_dicts(self._ft_has_futures, self._ft_has)
if exchange_config.get('_ft_has_params'):
self._ft_has = deep_merge_dicts(exchange_config.get('_ft_has_params'),
if exchange_conf.get('_ft_has_params'):
self._ft_has = deep_merge_dicts(exchange_conf.get('_ft_has_params'),
self._ft_has)
logger.info("Overriding exchange._ft_has with config params, result: %s", self._ft_has)
@@ -165,18 +165,18 @@ class Exchange:
# Initialize ccxt objects
ccxt_config = self._ccxt_config
ccxt_config = deep_merge_dicts(exchange_config.get('ccxt_config', {}), ccxt_config)
ccxt_config = deep_merge_dicts(exchange_config.get('ccxt_sync_config', {}), ccxt_config)
ccxt_config = deep_merge_dicts(exchange_conf.get('ccxt_config', {}), ccxt_config)
ccxt_config = deep_merge_dicts(exchange_conf.get('ccxt_sync_config', {}), ccxt_config)
self._api = self._init_ccxt(exchange_config, ccxt_kwargs=ccxt_config)
self._api = self._init_ccxt(exchange_conf, ccxt_kwargs=ccxt_config)
ccxt_async_config = self._ccxt_config
ccxt_async_config = deep_merge_dicts(exchange_config.get('ccxt_config', {}),
ccxt_async_config = deep_merge_dicts(exchange_conf.get('ccxt_config', {}),
ccxt_async_config)
ccxt_async_config = deep_merge_dicts(exchange_config.get('ccxt_async_config', {}),
ccxt_async_config = deep_merge_dicts(exchange_conf.get('ccxt_async_config', {}),
ccxt_async_config)
self._api_async = self._init_ccxt(
exchange_config, ccxt_async, ccxt_kwargs=ccxt_async_config)
exchange_conf, ccxt_async, ccxt_kwargs=ccxt_async_config)
logger.info(f'Using Exchange "{self.name}"')
self.required_candle_call_count = 1
@@ -189,7 +189,7 @@ class Exchange:
self._startup_candle_count, config.get('timeframe', ''))
# Converts the interval provided in minutes in config to seconds
self.markets_refresh_interval: int = exchange_config.get(
self.markets_refresh_interval: int = exchange_conf.get(
"markets_refresh_interval", 60) * 60
if self.trading_mode != TradingMode.SPOT and load_leverage_tiers:

1
freqtrade/freqai/base_models/BasePyTorchRegressor.py
View File

@@ -47,4 +47,5 @@ class BasePyTorchRegressor(BasePyTorchModel):
self.model.model.eval()
y = self.model.model(x)
pred_df = DataFrame(y.detach().tolist(), columns=[dk.label_list[0]])
pred_df = dk.denormalize_labels_from_metadata(pred_df)
return (pred_df, dk.do_predict)

4
freqtrade/freqai/prediction_models/PyTorchTransformerRegressor.py
View File

@@ -119,11 +119,11 @@ class PyTorchTransformerRegressor(BasePyTorchRegressor):
x = x.unsqueeze(0)
# create empty torch tensor
self.model.model.eval()
yb = torch.empty(0)
yb = torch.empty(0).to(self.device)
if x.shape[1] > 1:
ws = self.window_size
for i in range(0, x.shape[1] - ws):
xb = x[:, i:i + ws, :]
xb = x[:, i:i + ws, :].to(self.device)
y = self.model.model(xb)
yb = torch.cat((yb, y), dim=0)
else:

4
freqtrade/freqai/torch/PyTorchTransformerModel.py
View File

@@ -1,7 +1,7 @@
import math
import torch
import torch.nn as nn
from torch import nn
"""
@@ -68,7 +68,7 @@ class PyTorchTransformerModel(nn.Module):
return x
class PositionalEncoding(torch.nn.Module):
class PositionalEncoding(nn.Module):
def __init__(self, d_model, max_len=5000):
"""
Args

8
freqtrade/freqtradebot.py
View File

@@ -13,7 +13,7 @@ from schedule import Scheduler
from freqtrade import constants
from freqtrade.configuration import validate_config_consistency
from freqtrade.constants import BuySell, Config, LongShort
from freqtrade.constants import BuySell, Config, ExchangeConfig, LongShort
from freqtrade.data.converter import order_book_to_dataframe
from freqtrade.data.dataprovider import DataProvider
from freqtrade.edge import Edge
@@ -23,6 +23,7 @@ from freqtrade.exceptions import (DependencyException, ExchangeError, Insufficie
InvalidOrderException, PricingError)
from freqtrade.exchange import (ROUND_DOWN, ROUND_UP, timeframe_to_minutes, timeframe_to_next_date,
timeframe_to_seconds)
from freqtrade.exchange.common import remove_exchange_credentials
from freqtrade.misc import safe_value_fallback, safe_value_fallback2
from freqtrade.mixins import LoggingMixin
from freqtrade.persistence import Order, PairLocks, Trade, init_db
@@ -63,6 +64,9 @@ class FreqtradeBot(LoggingMixin):
# Init objects
self.config = config
exchange_config: ExchangeConfig = deepcopy(config['exchange'])
# Remove credentials from original exchange config to avoid accidental credentail exposure
remove_exchange_credentials(config['exchange'], True)
self.strategy: IStrategy = StrategyResolver.load_strategy(self.config)
@@ -70,7 +74,7 @@ class FreqtradeBot(LoggingMixin):
validate_config_consistency(config)
self.exchange = ExchangeResolver.load_exchange(
self.config, load_leverage_tiers=True)
self.config, exchange_config=exchange_config, load_leverage_tiers=True)
init_db(self.config['db_url'])

10
freqtrade/resolvers/exchange_resolver.py
View File

@@ -2,9 +2,10 @@
This module loads custom exchanges
"""
import logging
from typing import Optional
import freqtrade.exchange as exchanges
from freqtrade.constants import Config
from freqtrade.constants import Config, ExchangeConfig
from freqtrade.exchange import MAP_EXCHANGE_CHILDCLASS, Exchange
from freqtrade.resolvers import IResolver
@@ -19,8 +20,8 @@ class ExchangeResolver(IResolver):
object_type = Exchange
@staticmethod
def load_exchange(config: Config, validate: bool = True,
load_leverage_tiers: bool = False) -> Exchange:
def load_exchange(config: Config, *, exchange_config: Optional[ExchangeConfig] = None,
validate: bool = True, load_leverage_tiers: bool = False) -> Exchange:
"""
Load the custom class from config parameter
:param exchange_name: name of the Exchange to load
@@ -37,13 +38,14 @@ class ExchangeResolver(IResolver):
kwargs={
'config': config,
'validate': validate,
'exchange_config': exchange_config,
'load_leverage_tiers': load_leverage_tiers}
)
except ImportError:
logger.info(
f"No {exchange_name} specific subclass found. Using the generic class instead.")
if not exchange:
exchange = Exchange(config, validate=validate)
exchange = Exchange(config, validate=validate, exchange_config=exchange_config,)
return exchange
@staticmethod

190
freqtrade/rpc/api_server/api_backtest.py
View File

@@ -8,14 +8,16 @@ from fastapi import APIRouter, BackgroundTasks, Depends
from fastapi.exceptions import HTTPException
from freqtrade.configuration.config_validation import validate_config_consistency
from freqtrade.constants import Config
from freqtrade.data.btanalysis import get_backtest_resultlist, load_and_merge_backtest_result
from freqtrade.enums import BacktestState
from freqtrade.exceptions import DependencyException, OperationalException
from freqtrade.exchange.common import remove_exchange_credentials
from freqtrade.misc import deep_merge_dicts
from freqtrade.rpc.api_server.api_schemas import (BacktestHistoryEntry, BacktestRequest,
BacktestResponse)
from freqtrade.rpc.api_server.deps import get_config, is_webserver_mode
from freqtrade.rpc.api_server.webserver import ApiServer
from freqtrade.rpc.api_server.webserver_bgwork import ApiBG
from freqtrade.rpc.rpc import RPCException
@@ -25,19 +27,92 @@ logger = logging.getLogger(__name__)
router = APIRouter()
def __run_backtest_bg(btconfig: Config):
from freqtrade.optimize.optimize_reports import generate_backtest_stats, store_backtest_stats
from freqtrade.resolvers import StrategyResolver
asyncio.set_event_loop(asyncio.new_event_loop())
try:
# Reload strategy
lastconfig = ApiBG.bt['last_config']
strat = StrategyResolver.load_strategy(btconfig)
validate_config_consistency(btconfig)
if (
not ApiBG.bt['bt']
or lastconfig.get('timeframe') != strat.timeframe
or lastconfig.get('timeframe_detail') != btconfig.get('timeframe_detail')
or lastconfig.get('timerange') != btconfig['timerange']
):
from freqtrade.optimize.backtesting import Backtesting
ApiBG.bt['bt'] = Backtesting(btconfig)
ApiBG.bt['bt'].load_bt_data_detail()
else:
ApiBG.bt['bt'].config = btconfig
ApiBG.bt['bt'].init_backtest()
# Only reload data if timeframe changed.
if (
not ApiBG.bt['data']
or not ApiBG.bt['timerange']
or lastconfig.get('timeframe') != strat.timeframe
or lastconfig.get('timerange') != btconfig['timerange']
):
ApiBG.bt['data'], ApiBG.bt['timerange'] = ApiBG.bt[
'bt'].load_bt_data()
lastconfig['timerange'] = btconfig['timerange']
lastconfig['timeframe'] = strat.timeframe
lastconfig['protections'] = btconfig.get('protections', [])
lastconfig['enable_protections'] = btconfig.get('enable_protections')
lastconfig['dry_run_wallet'] = btconfig.get('dry_run_wallet')
ApiBG.bt['bt'].enable_protections = btconfig.get('enable_protections', False)
ApiBG.bt['bt'].strategylist = [strat]
ApiBG.bt['bt'].results = {}
ApiBG.bt['bt'].load_prior_backtest()
ApiBG.bt['bt'].abort = False
if (ApiBG.bt['bt'].results and
strat.get_strategy_name() in ApiBG.bt['bt'].results['strategy']):
# When previous result hash matches - reuse that result and skip backtesting.
logger.info(f'Reusing result of previous backtest for {strat.get_strategy_name()}')
else:
min_date, max_date = ApiBG.bt['bt'].backtest_one_strategy(
strat, ApiBG.bt['data'], ApiBG.bt['timerange'])
ApiBG.bt['bt'].results = generate_backtest_stats(
ApiBG.bt['data'], ApiBG.bt['bt'].all_results,
min_date=min_date, max_date=max_date)
if btconfig.get('export', 'none') == 'trades':
store_backtest_stats(
btconfig['exportfilename'], ApiBG.bt['bt'].results,
datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
)
logger.info("Backtest finished.")
except (Exception, OperationalException, DependencyException) as e:
logger.exception(f"Backtesting caused an error: {e}")
ApiBG.bt['bt_error'] = str(e)
pass
finally:
ApiBG.bgtask_running = False
@router.post('/backtest', response_model=BacktestResponse, tags=['webserver', 'backtest'])
async def api_start_backtest( # noqa: C901
async def api_start_backtest(
bt_settings: BacktestRequest, background_tasks: BackgroundTasks,
config=Depends(get_config), ws_mode=Depends(is_webserver_mode)):
ApiServer._bt['bt_error'] = None
ApiBG.bt['bt_error'] = None
"""Start backtesting if not done so already"""
if ApiServer._bgtask_running:
if ApiBG.bgtask_running:
raise RPCException('Bot Background task already running')
if ':' in bt_settings.strategy:
raise HTTPException(status_code=500, detail="base64 encoded strategies are not allowed.")
btconfig = deepcopy(config)
remove_exchange_credentials(btconfig['exchange'], True)
settings = dict(bt_settings)
if settings.get('freqai', None) is not None:
settings['freqai'] = dict(settings['freqai'])
@@ -54,80 +129,9 @@ async def api_start_backtest( # noqa: C901
# Start backtesting
# Initialize backtesting object
def run_backtest():
from freqtrade.optimize.optimize_reports import (generate_backtest_stats,
store_backtest_stats)
from freqtrade.resolvers import StrategyResolver
asyncio.set_event_loop(asyncio.new_event_loop())
try:
# Reload strategy
lastconfig = ApiServer._bt['last_config']
strat = StrategyResolver.load_strategy(btconfig)
validate_config_consistency(btconfig)
if (
not ApiServer._bt['bt']
or lastconfig.get('timeframe') != strat.timeframe
or lastconfig.get('timeframe_detail') != btconfig.get('timeframe_detail')
or lastconfig.get('timerange') != btconfig['timerange']
):
from freqtrade.optimize.backtesting import Backtesting
ApiServer._bt['bt'] = Backtesting(btconfig)
ApiServer._bt['bt'].load_bt_data_detail()
else:
ApiServer._bt['bt'].config = btconfig
ApiServer._bt['bt'].init_backtest()
# Only reload data if timeframe changed.
if (
not ApiServer._bt['data']
or not ApiServer._bt['timerange']
or lastconfig.get('timeframe') != strat.timeframe
or lastconfig.get('timerange') != btconfig['timerange']
):
ApiServer._bt['data'], ApiServer._bt['timerange'] = ApiServer._bt[
'bt'].load_bt_data()
lastconfig['timerange'] = btconfig['timerange']
lastconfig['timeframe'] = strat.timeframe
lastconfig['protections'] = btconfig.get('protections', [])
lastconfig['enable_protections'] = btconfig.get('enable_protections')
lastconfig['dry_run_wallet'] = btconfig.get('dry_run_wallet')
ApiServer._bt['bt'].enable_protections = btconfig.get('enable_protections', False)
ApiServer._bt['bt'].strategylist = [strat]
ApiServer._bt['bt'].results = {}
ApiServer._bt['bt'].load_prior_backtest()
ApiServer._bt['bt'].abort = False
if (ApiServer._bt['bt'].results and
strat.get_strategy_name() in ApiServer._bt['bt'].results['strategy']):
# When previous result hash matches - reuse that result and skip backtesting.
logger.info(f'Reusing result of previous backtest for {strat.get_strategy_name()}')
else:
min_date, max_date = ApiServer._bt['bt'].backtest_one_strategy(
strat, ApiServer._bt['data'], ApiServer._bt['timerange'])
ApiServer._bt['bt'].results = generate_backtest_stats(
ApiServer._bt['data'], ApiServer._bt['bt'].all_results,
min_date=min_date, max_date=max_date)
if btconfig.get('export', 'none') == 'trades':
store_backtest_stats(
btconfig['exportfilename'], ApiServer._bt['bt'].results,
datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
)
logger.info("Backtest finished.")
except (Exception, OperationalException, DependencyException) as e:
logger.exception(f"Backtesting caused an error: {e}")
ApiServer._bt['bt_error'] = str(e)
pass
finally:
ApiServer._bgtask_running = False
background_tasks.add_task(run_backtest)
ApiServer._bgtask_running = True
background_tasks.add_task(__run_backtest_bg, btconfig=btconfig)
ApiBG.bgtask_running = True
return {
"status": "running",
@@ -145,18 +149,18 @@ def api_get_backtest(ws_mode=Depends(is_webserver_mode)):
Returns Result after backtesting has been ran.
"""
from freqtrade.persistence import LocalTrade
if ApiServer._bgtask_running:
if ApiBG.bgtask_running:
return {
"status": "running",
"running": True,
"step": (ApiServer._bt['bt'].progress.action if ApiServer._bt['bt']
"step": (ApiBG.bt['bt'].progress.action if ApiBG.bt['bt']
else str(BacktestState.STARTUP)),
"progress": ApiServer._bt['bt'].progress.progress if ApiServer._bt['bt'] else 0,
"progress": ApiBG.bt['bt'].progress.progress if ApiBG.bt['bt'] else 0,
"trade_count": len(LocalTrade.trades),
"status_msg": "Backtest running",
}
if not ApiServer._bt['bt']:
if not ApiBG.bt['bt']:
return {
"status": "not_started",
"running": False,
@@ -164,13 +168,13 @@ def api_get_backtest(ws_mode=Depends(is_webserver_mode)):
"progress": 0,
"status_msg": "Backtest not yet executed"
}
if ApiServer._bt['bt_error']:
if ApiBG.bt['bt_error']:
return {
"status": "error",
"running": False,
"step": "",
"progress": 0,
"status_msg": f"Backtest failed with {ApiServer._bt['bt_error']}"
"status_msg": f"Backtest failed with {ApiBG.bt['bt_error']}"
}
return {
@@ -179,14 +183,14 @@ def api_get_backtest(ws_mode=Depends(is_webserver_mode)):
"status_msg": "Backtest ended",
"step": "finished",
"progress": 1,
"backtest_result": ApiServer._bt['bt'].results,
"backtest_result": ApiBG.bt['bt'].results,
}
@router.delete('/backtest', response_model=BacktestResponse, tags=['webserver', 'backtest'])
def api_delete_backtest(ws_mode=Depends(is_webserver_mode)):
"""Reset backtesting"""
if ApiServer._bgtask_running:
if ApiBG.bgtask_running:
return {
"status": "running",
"running": True,
@@ -194,12 +198,12 @@ def api_delete_backtest(ws_mode=Depends(is_webserver_mode)):
"progress": 0,
"status_msg": "Backtest running",
}
if ApiServer._bt['bt']:
ApiServer._bt['bt'].cleanup()
del ApiServer._bt['bt']
ApiServer._bt['bt'] = None
del ApiServer._bt['data']
ApiServer._bt['data'] = None
if ApiBG.bt['bt']:
ApiBG.bt['bt'].cleanup()
del ApiBG.bt['bt']
ApiBG.bt['bt'] = None
del ApiBG.bt['data']
ApiBG.bt['data'] = None
logger.info("Backtesting reset")
return {
"status": "reset",
@@ -212,7 +216,7 @@ def api_delete_backtest(ws_mode=Depends(is_webserver_mode)):
@router.get('/backtest/abort', response_model=BacktestResponse, tags=['webserver', 'backtest'])
def api_backtest_abort(ws_mode=Depends(is_webserver_mode)):
if not ApiServer._bgtask_running:
if not ApiBG.bgtask_running:
return {
"status": "not_running",
"running": False,
@@ -220,7 +224,7 @@ def api_backtest_abort(ws_mode=Depends(is_webserver_mode)):
"progress": 0,
"status_msg": "Backtest ended",
}
ApiServer._bt['bt'].abort = True
ApiBG.bt['bt'].abort = True
return {
"status": "stopping",
"running": False,

7
freqtrade/rpc/api_server/deps.py
View File

@@ -6,6 +6,7 @@ from fastapi import Depends
from freqtrade.enums import RunMode
from freqtrade.persistence import Trade
from freqtrade.persistence.models import _request_id_ctx_var
from freqtrade.rpc.api_server.webserver_bgwork import ApiBG
from freqtrade.rpc.rpc import RPC, RPCException
from .webserver import ApiServer
@@ -43,11 +44,11 @@ def get_api_config() -> Dict[str, Any]:
def get_exchange(config=Depends(get_config)):
if not ApiServer._exchange:
if not ApiBG.exchange:
from freqtrade.resolvers import ExchangeResolver
ApiServer._exchange = ExchangeResolver.load_exchange(
ApiBG.exchange = ExchangeResolver.load_exchange(
config, load_leverage_tiers=False)
return ApiServer._exchange
return ApiBG.exchange
def get_message_stream():

15
freqtrade/rpc/api_server/webserver.py
View File

@@ -1,6 +1,6 @@
import logging
from ipaddress import IPv4Address
from typing import Any, Dict, Optional
from typing import Any, Optional
import orjson
import uvicorn
@@ -36,19 +36,8 @@ class ApiServer(RPCHandler):
__initialized = False
_rpc: RPC
# Backtesting type: Backtesting
_bt: Dict[str, Any] = {
'bt': None,
'data': None,
'timerange': None,
'last_config': {},
'bt_error': None,
}
_has_rpc: bool = False
_bgtask_running: bool = False
_config: Config = {}
# Exchange - only available in webserver mode.
_exchange = None
# websocket message stuff
_message_stream: Optional[MessageStream] = None
@@ -85,7 +74,7 @@ class ApiServer(RPCHandler):
"""
Attach rpc handler
"""
if not self._has_rpc:
if not ApiServer._has_rpc:
ApiServer._rpc = rpc
ApiServer._has_rpc = True
else:

16
freqtrade/rpc/api_server/webserver_bgwork.py Normal file
View File

@@ -0,0 +1,16 @@
from typing import Any, Dict
class ApiBG():
# Backtesting type: Backtesting
bt: Dict[str, Any] = {
'bt': None,
'data': None,
'timerange': None,
'last_config': {},
'bt_error': None,
}
bgtask_running: bool = False
# Exchange - only available in webserver mode.
exchange = None

2
requirements-dev.txt
View File

@@ -9,7 +9,7 @@
coveralls==3.3.1
ruff==0.0.267
mypy==1.3.0
pre-commit==3.3.1
pre-commit==3.3.2
pytest==7.3.1
pytest-asyncio==0.21.0
pytest-cov==4.0.0

6
requirements.txt
View File

@@ -2,7 +2,7 @@ numpy==1.24.3
pandas==2.0.1
pandas-ta==0.3.14b
ccxt==3.0.103
ccxt==3.1.5
cryptography==40.0.2; platform_machine != 'armv7l'
cryptography==40.0.1; platform_machine == 'armv7l'
aiohttp==3.8.4
@@ -11,7 +11,7 @@ python-telegram-bot==20.3
# can't be hard-pinned due to telegram-bot pinning httpx with ~
httpx>=0.23.3
arrow==1.2.3
cachetools==4.2.2
cachetools==5.3.0
requests==2.30.0
urllib3==2.0.2
jsonschema==4.17.3
@@ -38,7 +38,7 @@ orjson==3.8.12
sdnotify==0.3.2
# API Server
fastapi==0.95.1
fastapi==0.95.2
pydantic==1.10.7
uvicorn==0.22.0
pyjwt==2.7.0

10
tests/exchange/test_exchange.py
View File

@@ -20,7 +20,7 @@ from freqtrade.exchange import (Binance, Bittrex, Exchange, Kraken, amount_to_pr
timeframe_to_minutes, timeframe_to_msecs, timeframe_to_next_date,
timeframe_to_prev_date, timeframe_to_seconds)
from freqtrade.exchange.common import (API_FETCH_ORDER_RETRY_COUNT, API_RETRY_COUNT,
calculate_backoff, remove_credentials)
calculate_backoff, remove_exchange_credentials)
from freqtrade.exchange.exchange import amount_to_contract_precision
from freqtrade.resolvers.exchange_resolver import ExchangeResolver
from tests.conftest import (EXMS, generate_test_data_raw, get_mock_coro, get_patched_exchange,
@@ -137,16 +137,14 @@ def test_init(default_conf, mocker, caplog):
assert log_has('Instance is running with dry_run enabled', caplog)
def test_remove_credentials(default_conf, caplog) -> None:
def test_remove_exchange_credentials(default_conf) -> None:
conf = deepcopy(default_conf)
conf['dry_run'] = False
remove_credentials(conf)
remove_exchange_credentials(conf['exchange'], False)
assert conf['exchange']['key'] != ''
assert conf['exchange']['secret'] != ''
conf['dry_run'] = True
remove_credentials(conf)
remove_exchange_credentials(conf['exchange'], True)
assert conf['exchange']['key'] == ''
assert conf['exchange']['secret'] == ''
assert conf['exchange']['password'] == ''

225
tests/rpc/test_rpc_apiserver.py
View File

@@ -21,11 +21,13 @@ from freqtrade.__init__ import __version__
from freqtrade.enums import CandleType, RunMode, State, TradingMode
from freqtrade.exceptions import DependencyException, ExchangeError, OperationalException
from freqtrade.loggers import setup_logging, setup_logging_pre
from freqtrade.optimize.backtesting import Backtesting
from freqtrade.persistence import PairLocks, Trade
from freqtrade.rpc import RPC
from freqtrade.rpc.api_server import ApiServer
from freqtrade.rpc.api_server.api_auth import create_token, get_user_from_token
from freqtrade.rpc.api_server.uvicorn_threaded import UvicornServer
from freqtrade.rpc.api_server.webserver_bgwork import ApiBG
from tests.conftest import (CURRENT_TEST_STRATEGY, EXMS, create_mock_trades, get_mock_coro,
get_patched_freqtradebot, log_has, log_has_re, patch_get_signal)
@@ -1665,137 +1667,140 @@ def test_sysinfo(botclient):
def test_api_backtesting(botclient, mocker, fee, caplog, tmpdir):
ftbot, client = botclient
mocker.patch(f'{EXMS}.get_fee', fee)
try:
ftbot, client = botclient
mocker.patch(f'{EXMS}.get_fee', fee)
rc = client_get(client, f"{BASE_URI}/backtest")
# Backtest prevented in default mode
assert_response(rc, 502)
rc = client_get(client, f"{BASE_URI}/backtest")
# Backtest prevented in default mode
assert_response(rc, 502)
ftbot.config['runmode'] = RunMode.WEBSERVER
# Backtesting not started yet
rc = client_get(client, f"{BASE_URI}/backtest")
assert_response(rc)
ftbot.config['runmode'] = RunMode.WEBSERVER
# Backtesting not started yet
rc = client_get(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'not_started'
assert not result['running']
assert result['status_msg'] == 'Backtest not yet executed'
assert result['progress'] == 0
result = rc.json()
assert result['status'] == 'not_started'
assert not result['running']
assert result['status_msg'] == 'Backtest not yet executed'
assert result['progress'] == 0
# Reset backtesting
rc = client_delete(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'reset'
assert not result['running']
assert result['status_msg'] == 'Backtest reset'
ftbot.config['export'] = 'trades'
ftbot.config['backtest_cache'] = 'day'
ftbot.config['user_data_dir'] = Path(tmpdir)
ftbot.config['exportfilename'] = Path(tmpdir) / "backtest_results"
ftbot.config['exportfilename'].mkdir()
# Reset backtesting
rc = client_delete(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'reset'
assert not result['running']
assert result['status_msg'] == 'Backtest reset'
ftbot.config['export'] = 'trades'
ftbot.config['backtest_cache'] = 'day'
ftbot.config['user_data_dir'] = Path(tmpdir)
ftbot.config['exportfilename'] = Path(tmpdir) / "backtest_results"
ftbot.config['exportfilename'].mkdir()
# start backtesting
data = {
"strategy": CURRENT_TEST_STRATEGY,
"timeframe": "5m",
"timerange": "20180110-20180111",
"max_open_trades": 3,
"stake_amount": 100,
"dry_run_wallet": 1000,
"enable_protections": False
}
rc = client_post(client, f"{BASE_URI}/backtest", data=data)
assert_response(rc)
result = rc.json()
# start backtesting
data = {
"strategy": CURRENT_TEST_STRATEGY,
"timeframe": "5m",
"timerange": "20180110-20180111",
"max_open_trades": 3,
"stake_amount": 100,
"dry_run_wallet": 1000,
"enable_protections": False
}
rc = client_post(client, f"{BASE_URI}/backtest", data=data)
assert_response(rc)
result = rc.json()
assert result['status'] == 'running'
assert result['progress'] == 0
assert result['running']
assert result['status_msg'] == 'Backtest started'
assert result['status'] == 'running'
assert result['progress'] == 0
assert result['running']
assert result['status_msg'] == 'Backtest started'
rc = client_get(client, f"{BASE_URI}/backtest")
assert_response(rc)
rc = client_get(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'ended'
assert not result['running']
assert result['status_msg'] == 'Backtest ended'
assert result['progress'] == 1
assert result['backtest_result']
result = rc.json()
assert result['status'] == 'ended'
assert not result['running']
assert result['status_msg'] == 'Backtest ended'
assert result['progress'] == 1
assert result['backtest_result']
rc = client_get(client, f"{BASE_URI}/backtest/abort")
assert_response(rc)
result = rc.json()
assert result['status'] == 'not_running'
assert not result['running']
assert result['status_msg'] == 'Backtest ended'
rc = client_get(client, f"{BASE_URI}/backtest/abort")
assert_response(rc)
result = rc.json()
assert result['status'] == 'not_running'
assert not result['running']
assert result['status_msg'] == 'Backtest ended'
# Simulate running backtest
ApiServer._bgtask_running = True
rc = client_get(client, f"{BASE_URI}/backtest/abort")
assert_response(rc)
result = rc.json()
assert result['status'] == 'stopping'
assert not result['running']
assert result['status_msg'] == 'Backtest ended'
# Simulate running backtest
ApiBG.bgtask_running = True
rc = client_get(client, f"{BASE_URI}/backtest/abort")
assert_response(rc)
result = rc.json()
assert result['status'] == 'stopping'
assert not result['running']
assert result['status_msg'] == 'Backtest ended'
# Get running backtest...
rc = client_get(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'running'
assert result['running']
assert result['step'] == "backtest"
assert result['status_msg'] == "Backtest running"
# Get running backtest...
rc = client_get(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'running'
assert result['running']
assert result['step'] == "backtest"
assert result['status_msg'] == "Backtest running"
# Try delete with task still running
rc = client_delete(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'running'
# Try delete with task still running
rc = client_delete(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'running'
# Post to backtest that's still running
rc = client_post(client, f"{BASE_URI}/backtest", data=data)
assert_response(rc, 502)
result = rc.json()
assert 'Bot Background task already running' in result['error']
# Post to backtest that's still running
rc = client_post(client, f"{BASE_URI}/backtest", data=data)
assert_response(rc, 502)
result = rc.json()
assert 'Bot Background task already running' in result['error']
ApiServer._bgtask_running = False
ApiBG.bgtask_running = False
# Rerun backtest (should get previous result)
rc = client_post(client, f"{BASE_URI}/backtest", data=data)
assert_response(rc)
result = rc.json()
assert log_has_re('Reusing result of previous backtest.*', caplog)
# Rerun backtest (should get previous result)
rc = client_post(client, f"{BASE_URI}/backtest", data=data)
assert_response(rc)
result = rc.json()
assert log_has_re('Reusing result of previous backtest.*', caplog)
data['stake_amount'] = 101
data['stake_amount'] = 101
mocker.patch('freqtrade.optimize.backtesting.Backtesting.backtest_one_strategy',
side_effect=DependencyException('DeadBeef'))
rc = client_post(client, f"{BASE_URI}/backtest", data=data)
assert log_has("Backtesting caused an error: DeadBeef", caplog)
mocker.patch('freqtrade.optimize.backtesting.Backtesting.backtest_one_strategy',
side_effect=DependencyException('DeadBeef'))
rc = client_post(client, f"{BASE_URI}/backtest", data=data)
assert log_has("Backtesting caused an error: DeadBeef", caplog)
rc = client_get(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'error'
assert 'Backtest failed' in result['status_msg']
rc = client_get(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'error'
assert 'Backtest failed' in result['status_msg']
# Delete backtesting to avoid leakage since the backtest-object may stick around.
rc = client_delete(client, f"{BASE_URI}/backtest")
assert_response(rc)
# Delete backtesting to avoid leakage since the backtest-object may stick around.
rc = client_delete(client, f"{BASE_URI}/backtest")
assert_response(rc)
result = rc.json()
assert result['status'] == 'reset'
assert not result['running']
assert result['status_msg'] == 'Backtest reset'
result = rc.json()
assert result['status'] == 'reset'
assert not result['running']
assert result['status_msg'] == 'Backtest reset'
# Disallow base64 strategies
data['strategy'] = "xx:cHJpbnQoImhlbGxvIHdvcmxkIik="
rc = client_post(client, f"{BASE_URI}/backtest", data=data)
assert_response(rc, 500)
# Disallow base64 strategies
data['strategy'] = "xx:cHJpbnQoImhlbGxvIHdvcmxkIik="
rc = client_post(client, f"{BASE_URI}/backtest", data=data)
assert_response(rc, 500)
finally:
Backtesting.cleanup()
def test_api_backtest_history(botclient, mocker, testdatadir):

32
tests/test_freqtradebot.py
View File

@@ -121,7 +121,7 @@ def test_order_dict(default_conf_usdt, mocker, runmode, caplog) -> None:
freqtrade = FreqtradeBot(conf)
if runmode == RunMode.LIVE:
assert not log_has_re(".*stoploss_on_exchange .* dry-run", caplog)
assert not log_has_re(r".*stoploss_on_exchange .* dry-run", caplog)
assert freqtrade.strategy.order_types['stoploss_on_exchange']
caplog.clear()
@@ -136,7 +136,7 @@ def test_order_dict(default_conf_usdt, mocker, runmode, caplog) -> None:
}
freqtrade = FreqtradeBot(conf)
assert not freqtrade.strategy.order_types['stoploss_on_exchange']
assert not log_has_re(".*stoploss_on_exchange .* dry-run", caplog)
assert not log_has_re(r".*stoploss_on_exchange .* dry-run", caplog)
def test_get_trade_stake_amount(default_conf_usdt, mocker) -> None:
@@ -149,6 +149,34 @@ def test_get_trade_stake_amount(default_conf_usdt, mocker) -> None:
assert result == default_conf_usdt['stake_amount']
@pytest.mark.parametrize('runmode', [
RunMode.DRY_RUN,
RunMode.LIVE
])
def test_load_strategy_no_keys(default_conf_usdt, mocker, runmode, caplog) -> None:
patch_RPCManager(mocker)
patch_exchange(mocker)
conf = deepcopy(default_conf_usdt)
conf['runmode'] = runmode
erm = mocker.patch('freqtrade.freqtradebot.ExchangeResolver.load_exchange')
freqtrade = FreqtradeBot(conf)
strategy_config = freqtrade.strategy.config
assert id(strategy_config['exchange']) == id(conf['exchange'])
# Keys have been removed and are not passed to the exchange
assert strategy_config['exchange']['key'] == ''
assert strategy_config['exchange']['secret'] == ''
assert erm.call_count == 1
ex_conf = erm.call_args_list[0][1]['exchange_config']
assert id(ex_conf) != id(conf['exchange'])
# Keys are still present
assert ex_conf['key'] != ''
assert ex_conf['key'] == default_conf_usdt['exchange']['key']
assert ex_conf['secret'] != ''
assert ex_conf['secret'] == default_conf_usdt['exchange']['secret']
@pytest.mark.parametrize("amend_last,wallet,max_open,lsamr,expected", [
(False, 120, 2, 0.5, [60, None]),
(True, 120, 2, 0.5, [60, 58.8]),