CTYPTO/freqtrade
1
0
Fork 0
mirror of https://github.com/freqtrade/freqtrade.git synced 2025-12-18 22:01:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix merge conflicts with develop

Browse Source
This commit is contained in:
robcaulk
2023-05-14 12:00:03 +00:00
parent ab0f9d78ee af8fbad281
commit 8a9b2fc16f
55 changed files with 1882 additions and 311 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
.github/workflows/ci.yml vendored
View File

@@ -14,7 +14,7 @@ on:
- cron: '0 5 * * 4'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
group: "${{ github.workflow }}-${{ github.ref }}-${{ github.event_name }}"
cancel-in-progress: true
permissions:
repository-projects: read
@@ -77,6 +77,17 @@ jobs:
# Allow failure for coveralls
coveralls || true
- name: Check for repository changes
run: |
if [ -n "$(git status --porcelain)" ]; then
echo "Repository is dirty, changes detected:"
git status
git diff
exit 1
else
echo "Repository is clean, no changes detected."
fi
- name: Backtesting (multi)
run: |
cp config_examples/config_bittrex.example.json config.json
@@ -174,6 +185,17 @@ jobs:
run: |
pytest --random-order
- name: Check for repository changes
run: |
if [ -n "$(git status --porcelain)" ]; then
echo "Repository is dirty, changes detected:"
git status
git diff
exit 1
else
echo "Repository is clean, no changes detected."
fi
- name: Backtesting
run: |
cp config_examples/config_bittrex.example.json config.json
@@ -237,6 +259,18 @@ jobs:
run: |
pytest --random-order
- name: Check for repository changes
run: |
if (git status --porcelain) {
Write-Host "Repository is dirty, changes detected:"
git status
git diff
exit 1
}
else {
Write-Host "Repository is clean, no changes detected."
}
- name: Backtesting
run: |
cp config_examples/config_bittrex.example.json config.json
@@ -302,7 +336,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.10"
python-version: "3.11"
- name: Documentation build
run: |

2
docs/freqai-parameter-table.md
View File

@@ -18,7 +18,7 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| `purge_old_models` | Number of models to keep on disk (not relevant to backtesting). Default is 2, which means that dry/live runs will keep the latest 2 models on disk. Setting to 0 keeps all models. This parameter also accepts a boolean to maintain backwards compatibility. <br> **Datatype:** Integer. <br> Default: `2`.
| `save_backtest_models` | Save models to disk when running backtesting. Backtesting operates most efficiently by saving the prediction data and reusing them directly for subsequent runs (when you wish to tune entry/exit parameters). Saving backtesting models to disk also allows to use the same model files for starting a dry/live instance with the same model `identifier`. <br> **Datatype:** Boolean. <br> Default: `False` (no models are saved).
| `fit_live_predictions_candles` | Number of historical candles to use for computing target (label) statistics from prediction data, instead of from the training dataset (more information can be found [here](freqai-configuration.md#creating-a-dynamic-target-threshold)). <br> **Datatype:** Positive integer.
| `continual_learning` | Use the final state of the most recently trained model as starting point for the new model, allowing for incremental learning (more information can be found [here](freqai-running.md#continual-learning)). <br> **Datatype:** Boolean. <br> Default: `False`.
| `continual_learning` | Use the final state of the most recently trained model as starting point for the new model, allowing for incremental learning (more information can be found [here](freqai-running.md#continual-learning)). Beware that this is currently a naive approach to incremental learning, and it has a high probability of overfitting/getting stuck in local minima while the market moves away from your model. We have the connections here primarily for experimental purposes and so that it is ready for more mature approaches to continual learning in chaotic systems like the crypto market. <br> **Datatype:** Boolean. <br> Default: `False`.
| `write_metrics_to_disk` | Collect train timings, inference timings and cpu usage in json file. <br> **Datatype:** Boolean. <br> Default: `False`
| `data_kitchen_thread_count` | <br> Designate the number of threads you want to use for data processing (outlier methods, normalization, etc.). This has no impact on the number of threads used for training. If user does not set it (default), FreqAI will use max number of threads - 2 (leaving 1 physical core available for Freqtrade bot and FreqUI) <br> **Datatype:** Positive integer.

14
docs/freqai-reinforcement-learning.md
View File

@@ -135,7 +135,14 @@ Parameter details can be found [here](freqai-parameter-table.md), but in general
## Creating a custom reward function
As you begin to modify the strategy and the prediction model, you will quickly realize some important differences between the Reinforcement Learner and the Regressors/Classifiers. Firstly, the strategy does not set a target value (no labels!). Instead, you set the `calculate_reward()` function inside the `MyRLEnv` class (see below). A default `calculate_reward()` is provided inside `prediction_models/ReinforcementLearner.py` to demonstrate the necessary building blocks for creating rewards, but users are encouraged to create their own custom reinforcement learning model class (see below) and save it to `user_data/freqaimodels`. It is inside the `calculate_reward()` where creative theories about the market can be expressed. For example, you can reward your agent when it makes a winning trade, and penalize the agent when it makes a losing trade. Or perhaps, you wish to reward the agent for entering trades, and penalize the agent for sitting in trades too long. Below we show examples of how these rewards are all calculated:
!!! danger "Not for production"
Warning!
The reward function provided with the Freqtrade source code is a showcase of functionality designed to show/test as many possible environment control features as possible. It is also designed to run quickly on small computers. This is a benchmark, it is *not* for live production. Please beware that you will need to create your own custom_reward() function or use a template built by other users outside of the Freqtrade source code.
As you begin to modify the strategy and the prediction model, you will quickly realize some important differences between the Reinforcement Learner and the Regressors/Classifiers. Firstly, the strategy does not set a target value (no labels!). Instead, you set the `calculate_reward()` function inside the `MyRLEnv` class (see below). A default `calculate_reward()` is provided inside `prediction_models/ReinforcementLearner.py` to demonstrate the necessary building blocks for creating rewards, but this is *not* designed for production. Users *must* create their own custom reinforcement learning model class or use a pre-built one from outside the Freqtrade source code and save it to `user_data/freqaimodels`. It is inside the `calculate_reward()` where creative theories about the market can be expressed. For example, you can reward your agent when it makes a winning trade, and penalize the agent when it makes a losing trade. Or perhaps, you wish to reward the agent for entering trades, and penalize the agent for sitting in trades too long. Below we show examples of how these rewards are all calculated:
!!! note "Hint"
The best reward functions are ones that are continuously differentiable, and well scaled. In other words, adding a single large negative penalty to a rare event is not a good idea, and the neural net will not be able to learn that function. Instead, it is better to add a small negative penalty to a common event. This will help the agent learn faster. Not only this, but you can help improve the continuity of your rewards/penalties by having them scale with severity according to some linear/exponential functions. In other words, you'd slowly scale the penalty as the duration of the trade increases. This is better than a single large penalty occuring at a single point in time.
```python
from freqtrade.freqai.prediction_models.ReinforcementLearner import ReinforcementLearner
@@ -169,6 +176,11 @@ As you begin to modify the strategy and the prediction model, you will quickly r
User made custom environment. This class inherits from BaseEnvironment and gym.env.
Users can override any functions from those parent classes. Here is an example
of a user customized `calculate_reward()` function.
Warning!
This is function is a showcase of functionality designed to show as many possible
environment control features as possible. It is also designed to run quickly
on small computers. This is a benchmark, it is *not* for live production.
"""
def calculate_reward(self, action: int) -> float:
# first, penalize if the action is not valid

3
docs/freqai-running.md
View File

@@ -131,6 +131,9 @@ You can choose to adopt a continual learning scheme by setting `"continual_learn
???+ danger "Continual learning enforces a constant parameter space"
Since `continual_learning` means that the model parameter space *cannot* change between trainings, `principal_component_analysis` is automatically disabled when `continual_learning` is enabled. Hint: PCA changes the parameter space and the number of features, learn more about PCA [here](freqai-feature-engineering.md#data-dimensionality-reduction-with-principal-component-analysis).
???+ danger "Experimental functionality"
Beware that this is currently a naive approach to incremental learning, and it has a high probability of overfitting/getting stuck in local minima while the market moves away from your model. We have the mechanics available in FreqAI primarily for experimental purposes and so that it is ready for more mature approaches to continual learning in chaotic systems like the crypto market.
## Hyperopt
You can hyperopt using the same command as for [typical Freqtrade hyperopt](hyperopt.md):

9
docs/freqai.md
View File

@@ -34,6 +34,9 @@ freqtrade trade --config config_examples/config_freqai.example.json --strategy F
You will see the boot-up process of automatic data downloading, followed by simultaneous training and trading.
!!! danger "Not for production"
The example strategy provided with the Freqtrade source code is designed for showcasing/testing a wide variety of FreqAI features. It is also designed to run on small computers so that it can be used as a benchmark between developers and users. It is *not* designed to be run in production.
An example strategy, prediction model, and config to use as a starting points can be found in
`freqtrade/templates/FreqaiExampleStrategy.py`, `freqtrade/freqai/prediction_models/LightGBMRegressor.py`, and
`config_examples/config_freqai.example.json`, respectively.
@@ -69,11 +72,7 @@ pip install -r requirements-freqai.txt
```
!!! Note
Catboost will not be installed on arm devices (raspberry, Mac M1, ARM based VPS, ...), since it does not provide wheels for this platform.
!!! Note "python 3.11"
Some dependencies (Catboost, Torch) currently don't support python 3.11. Freqtrade therefore only supports python 3.10 for these models/dependencies.
Tests involving these dependencies are skipped on 3.11.
Catboost will not be installed on low-powered arm devices (raspberry), since it does not provide wheels for this platform.
### Usage with docker

4
docs/rest-api.md
View File

@@ -134,7 +134,9 @@ python3 scripts/rest_client.py --config rest_config.json <command> [optional par
| `reload_config` | Reloads the configuration file.
| `trades` | List last trades. Limited to 500 trades per call.
| `trade/<tradeid>` | Get specific trade.
| `delete_trade <trade_id>` | Remove trade from the database. Tries to close open orders. Requires manual handling of this trade on the exchange.
| `trade/<tradeid>` | DELETE - Remove trade from the database. Tries to close open orders. Requires manual handling of this trade on the exchange.
| `trade/<tradeid>/open-order` | DELETE - Cancel open order for this trade.
| `trade/<tradeid>/reload` | GET - Reload a trade from the Exchange. Only works in live, and can potentially help recover a trade that was manually sold on the exchange.
| `show_config` | Shows part of the current configuration with relevant settings to operation.
| `logs` | Shows last log messages.
| `status` | Lists all open trades.

1
docs/telegram-usage.md
View File

@@ -187,6 +187,7 @@ official commands. You can ask at any moment for help with `/help`.
| `/forcelong <pair> [rate]` | Instantly buys the given pair. Rate is optional and only applies to limit orders. (`force_entry_enable` must be set to True)
| `/forceshort <pair> [rate]` | Instantly shorts the given pair. Rate is optional and only applies to limit orders. This will only work on non-spot markets. (`force_entry_enable` must be set to True)
| `/delete <trade_id>` | Delete a specific trade from the Database. Tries to close open orders. Requires manual handling of this trade on the exchange.
| `/reload_trade <trade_id>` | Reload a trade from the Exchange. Only works in live, and can potentially help recover a trade that was manually sold on the exchange.
| `/cancel_open_order <trade_id> | /coo <trade_id>` | Cancel an open order for a trade.
| **Metrics** |
| `/profit [<n>]` | Display a summary of your profit/loss from close trades and some stats about your performance, over the last n days (all trades by default)

4
freqtrade/commands/data_commands.py
View File

@@ -52,7 +52,7 @@ def start_download_data(args: Dict[str, Any]) -> None:
pairs_not_available: List[str] = []
# Init exchange
exchange = ExchangeResolver.load_exchange(config['exchange']['name'], config, validate=False)
exchange = ExchangeResolver.load_exchange(config, validate=False)
markets = [p for p, m in exchange.markets.items() if market_is_active(m)
or config.get('include_inactive')]
@@ -125,7 +125,7 @@ def start_convert_trades(args: Dict[str, Any]) -> None:
"Please check the documentation on how to configure this.")
# Init exchange
exchange = ExchangeResolver.load_exchange(config['exchange']['name'], config, validate=False)
exchange = ExchangeResolver.load_exchange(config, validate=False)
# Manual validations of relevant settings
if not config['exchange'].get('skip_pair_validation', False):
exchange.validate_pairs(config['pairs'])

4
freqtrade/commands/list_commands.py
View File

@@ -114,7 +114,7 @@ def start_list_timeframes(args: Dict[str, Any]) -> None:
config['timeframe'] = None
# Init exchange
exchange = ExchangeResolver.load_exchange(config['exchange']['name'], config, validate=False)
exchange = ExchangeResolver.load_exchange(config, validate=False)
if args['print_one_column']:
print('\n'.join(exchange.timeframes))
@@ -133,7 +133,7 @@ def start_list_markets(args: Dict[str, Any], pairs_only: bool = False) -> None:
config = setup_utils_configuration(args, RunMode.UTIL_EXCHANGE)
# Init exchange
exchange = ExchangeResolver.load_exchange(config['exchange']['name'], config, validate=False)
exchange = ExchangeResolver.load_exchange(config, validate=False)
# By default only active pairs/markets are to be shown
active_only = not args.get('list_pairs_all', False)

2
freqtrade/commands/pairlist_commands.py
View File

@@ -18,7 +18,7 @@ def start_test_pairlist(args: Dict[str, Any]) -> None:
from freqtrade.plugins.pairlistmanager import PairListManager
config = setup_utils_configuration(args, RunMode.UTIL_EXCHANGE)
exchange = ExchangeResolver.load_exchange(config['exchange']['name'], config, validate=False)
exchange = ExchangeResolver.load_exchange(config, validate=False)
quote_currencies = args.get('quote_currencies')
if not quote_currencies:

1
freqtrade/enums/exittype.py
View File

@@ -15,6 +15,7 @@ class ExitType(Enum):
EMERGENCY_EXIT = "emergency_exit"
CUSTOM_EXIT = "custom_exit"
PARTIAL_EXIT = "partial_exit"
SOLD_ON_EXCHANGE = "sold_on_exchange"
NONE = ""
def __str__(self):

1460
freqtrade/exchange/binance_leverage_tiers.json
View File

File diff suppressed because it is too large Load Diff

5
freqtrade/exchange/common.py
View File

@@ -4,6 +4,7 @@ import time
from functools import wraps
from typing import Any, Callable, Optional, TypeVar, cast, overload
from freqtrade.constants import Config
from freqtrade.exceptions import DDosProtection, RetryableOrderError, TemporaryError
from freqtrade.mixins import LoggingMixin
@@ -84,10 +85,11 @@ EXCHANGE_HAS_OPTIONAL = [
# 'fetchPositions', # Futures trading
# 'fetchLeverageTiers', # Futures initialization
# 'fetchMarketLeverageTiers', # Futures initialization
# 'fetchOpenOrders', 'fetchClosedOrders', # 'fetchOrders', # Refinding balance...
]
def remove_credentials(config) -> None:
def remove_credentials(config: Config) -> None:
"""
Removes exchange keys from the configuration and specifies dry-run
Used for backtesting / hyperopt / edge and utils.
@@ -95,6 +97,7 @@ def remove_credentials(config) -> None:
"""
if config.get('dry_run', False):
config['exchange']['key'] = ''
config['exchange']['apiKey'] = ''
config['exchange']['secret'] = ''
config['exchange']['password'] = ''
config['exchange']['uid'] = ''

43
freqtrade/exchange/exchange.py
View File

@@ -92,7 +92,7 @@ class Exchange:
# TradingMode.SPOT always supported and not required in this list
]
def __init__(self, config: Config, validate: bool = True,
def __init__(self, config: Config, *, validate: bool = True,
load_leverage_tiers: bool = False) -> None:
"""
Initializes this module with the given config,
@@ -1432,6 +1432,47 @@ class Exchange:
except ccxt.BaseError as e:
raise OperationalException(e) from e
@retrier(retries=0)
def fetch_orders(self, pair: str, since: datetime) -> List[Dict]:
"""
Fetch all orders for a pair "since"
:param pair: Pair for the query
:param since: Starting time for the query
"""
if self._config['dry_run']:
return []
def fetch_orders_emulate() -> List[Dict]:
orders = []
if self.exchange_has('fetchClosedOrders'):
orders = self._api.fetch_closed_orders(pair, since=since_ms)
if self.exchange_has('fetchOpenOrders'):
orders_open = self._api.fetch_open_orders(pair, since=since_ms)
orders.extend(orders_open)
return orders
try:
since_ms = int((since.timestamp() - 10) * 1000)
if self.exchange_has('fetchOrders'):
try:
orders: List[Dict] = self._api.fetch_orders(pair, since=since_ms)
except ccxt.NotSupported:
# Some exchanges don't support fetchOrders
# attempt to fetch open and closed orders separately
orders = fetch_orders_emulate()
else:
orders = fetch_orders_emulate()
self._log_exchange_response('fetch_orders', orders)
orders = [self._order_contracts_to_amount(o) for o in orders]
return orders
except ccxt.DDoSProtection as e:
raise DDosProtection(e) from e
except (ccxt.NetworkError, ccxt.ExchangeError) as e:
raise TemporaryError(
f'Could not fetch positions due to {e.__class__.__name__}. Message: {e}') from e
except ccxt.BaseError as e:
raise OperationalException(e) from e
@retrier
def fetch_trading_fees(self) -> Dict[str, Any]:
"""

6
freqtrade/freqai/RL/BaseEnvironment.py
View File

@@ -306,6 +306,12 @@ class BaseEnvironment(gym.Env):
"""
An example reward function. This is the one function that users will likely
wish to inject their own creativity into.
Warning!
This is function is a showcase of functionality designed to show as many possible
environment control features as possible. It is also designed to run quickly
on small computers. This is a benchmark, it is *not* for live production.
:param action: int = The action made by the agent for the current candle.
:return:
float = the reward to give to the agent for current step (used for optimization

6
freqtrade/freqai/RL/BaseReinforcementLearningModel.py
View File

@@ -371,6 +371,12 @@ class BaseReinforcementLearningModel(IFreqaiModel):
"""
An example reward function. This is the one function that users will likely
wish to inject their own creativity into.
Warning!
This is function is a showcase of functionality designed to show as many possible
environment control features as possible. It is also designed to run quickly
on small computers. This is a benchmark, it is *not* for live production.
:param action: int = The action made by the agent for the current candle.
:return:
float = the reward to give to the agent for current step (used for optimization

25
freqtrade/freqai/prediction_models/PyTorchMLPClassifier.py
View File

@@ -74,17 +74,18 @@ class PyTorchMLPClassifier(BasePyTorchClassifier):
model.to(self.device)
optimizer = torch.optim.AdamW(model.parameters(), lr=self.learning_rate)
criterion = torch.nn.CrossEntropyLoss()
init_model = self.get_init_model(dk.pair)
trainer = PyTorchModelTrainer(
model=model,
optimizer=optimizer,
criterion=criterion,
model_meta_data={"class_names": class_names},
device=self.device,
init_model=init_model,
data_convertor=self.data_convertor,
tb_logger=self.tb_logger,
**self.trainer_kwargs,
)
# check if continual_learning is activated, and retreive the model to continue training
trainer = self.get_init_model(dk.pair)
if trainer is None:
trainer = PyTorchModelTrainer(
model=model,
optimizer=optimizer,
criterion=criterion,
model_meta_data={"class_names": class_names},
device=self.device,
data_convertor=self.data_convertor,
tb_logger=self.tb_logger,
**self.trainer_kwargs,
)
trainer.fit(data_dictionary, self.splits)
return trainer

23
freqtrade/freqai/prediction_models/PyTorchMLPRegressor.py
View File

@@ -69,16 +69,17 @@ class PyTorchMLPRegressor(BasePyTorchRegressor):
model.to(self.device)
optimizer = torch.optim.AdamW(model.parameters(), lr=self.learning_rate)
criterion = torch.nn.MSELoss()
init_model = self.get_init_model(dk.pair)
trainer = PyTorchModelTrainer(
model=model,
optimizer=optimizer,
criterion=criterion,
device=self.device,
init_model=init_model,
data_convertor=self.data_convertor,
tb_logger=self.tb_logger,
**self.trainer_kwargs,
)
# check if continual_learning is activated, and retreive the model to continue training
trainer = self.get_init_model(dk.pair)
if trainer is None:
trainer = PyTorchModelTrainer(
model=model,
optimizer=optimizer,
criterion=criterion,
device=self.device,
data_convertor=self.data_convertor,
tb_logger=self.tb_logger,
**self.trainer_kwargs,
)
trainer.fit(data_dictionary, self.splits)
return trainer

25
freqtrade/freqai/prediction_models/PyTorchTransformerRegressor.py
View File

@@ -74,18 +74,19 @@ class PyTorchTransformerRegressor(BasePyTorchRegressor):
model.to(self.device)
optimizer = torch.optim.AdamW(model.parameters(), lr=self.learning_rate)
criterion = torch.nn.MSELoss()
init_model = self.get_init_model(dk.pair)
trainer = PyTorchTransformerTrainer(
model=model,
optimizer=optimizer,
criterion=criterion,
device=self.device,
init_model=init_model,
data_convertor=self.data_convertor,
window_size=self.window_size,
tb_logger=self.tb_logger,
**self.trainer_kwargs,
)
# check if continual_learning is activated, and retreive the model to continue training
trainer = self.get_init_model(dk.pair)
if trainer is None:
trainer = PyTorchTransformerTrainer(
model=model,
optimizer=optimizer,
criterion=criterion,
device=self.device,
data_convertor=self.data_convertor,
window_size=self.window_size,
tb_logger=self.tb_logger,
**self.trainer_kwargs,
)
trainer.fit(data_dictionary, self.splits)
return trainer

6
freqtrade/freqai/prediction_models/ReinforcementLearner.py
View File

@@ -97,6 +97,12 @@ class ReinforcementLearner(BaseReinforcementLearningModel):
"""
An example reward function. This is the one function that users will likely
wish to inject their own creativity into.
Warning!
This is function is a showcase of functionality designed to show as many possible
environment control features as possible. It is also designed to run quickly
on small computers. This is a benchmark, it is *not* for live production.
:param action: int = The action made by the agent for the current candle.
:return:
float = the reward to give to the agent for current step (used for optimization

3
freqtrade/freqai/torch/PyTorchModelTrainer.py
View File

@@ -25,7 +25,6 @@ class PyTorchModelTrainer(PyTorchTrainerInterface):
optimizer: Optimizer,
criterion: nn.Module,
device: str,
init_model: Dict,
data_convertor: PyTorchDataConvertor,
model_meta_data: Dict[str, Any] = {},
window_size: int = 1,
@@ -58,8 +57,6 @@ class PyTorchModelTrainer(PyTorchTrainerInterface):
self.data_convertor = data_convertor
self.window_size: int = window_size
self.tb_logger = tb_logger
if init_model:
self.load_from_checkpoint(init_model)
def fit(self, data_dictionary: Dict[str, pd.DataFrame], splits: List[str]):
"""

57
freqtrade/freqtradebot.py
View File

@@ -1,9 +1,9 @@
"""
Freqtrade is the main module of this bot. It contains the class Freqtrade()
"""
import copy
import logging
import traceback
from copy import deepcopy
from datetime import datetime, time, timedelta, timezone
from math import isclose
from threading import Lock
@@ -70,7 +70,7 @@ class FreqtradeBot(LoggingMixin):
validate_config_consistency(config)
self.exchange = ExchangeResolver.load_exchange(
self.config['exchange']['name'], self.config, load_leverage_tiers=True)
self.config, load_leverage_tiers=True)
init_db(self.config['db_url'])
@@ -451,6 +451,42 @@ class FreqtradeBot(LoggingMixin):
except ExchangeError:
logger.warning(f"Error updating {order.order_id}.")
def handle_onexchange_order(self, trade: Trade):
"""
Try refinding a order that is not in the database.
Only used balance disappeared, which would make exiting impossible.
"""
try:
orders = self.exchange.fetch_orders(trade.pair, trade.open_date_utc)
for order in orders:
trade_order = [o for o in trade.orders if o.order_id == order['id']]
if trade_order:
continue
logger.info(f"Found previously unknown order {order['id']} for {trade.pair}.")
order_obj = Order.parse_from_ccxt_object(order, trade.pair, order['side'])
order_obj.order_filled_date = datetime.fromtimestamp(
safe_value_fallback(order, 'lastTradeTimestamp', 'timestamp') // 1000,
tz=timezone.utc)
trade.orders.append(order_obj)
# TODO: how do we handle open_order_id ...
Trade.commit()
prev_exit_reason = trade.exit_reason
trade.exit_reason = ExitType.SOLD_ON_EXCHANGE.value
self.update_trade_state(trade, order['id'], order)
logger.info(f"handled order {order['id']}")
if not trade.is_open:
# Trade was just closed
trade.close_date = order_obj.order_filled_date
Trade.commit()
break
else:
trade.exit_reason = prev_exit_reason
Trade.commit()
except ExchangeError:
logger.warning("Error finding onexchange order")
#
# BUY / enter positions / open trades logic and methods
#
@@ -461,7 +497,7 @@ class FreqtradeBot(LoggingMixin):
"""
trades_created = 0
whitelist = copy.deepcopy(self.active_pair_whitelist)
whitelist = deepcopy(self.active_pair_whitelist)
if not whitelist:
self.log_once("Active pair whitelist is empty.", logger.info)
return trades_created
@@ -982,7 +1018,7 @@ class FreqtradeBot(LoggingMixin):
'base_currency': self.exchange.get_pair_base_currency(trade.pair),
'fiat_currency': self.config.get('fiat_display_currency', None),
'amount': order.safe_amount_after_fee if fill else (order.amount or trade.amount),
'open_date': trade.open_date or datetime.utcnow(),
'open_date': trade.open_date_utc or datetime.now(timezone.utc),
'current_rate': current_rate,
'sub_trade': sub_trade,
}
@@ -1034,6 +1070,13 @@ class FreqtradeBot(LoggingMixin):
"""
trades_closed = 0
for trade in trades:
if not self.wallets.check_exit_amount(trade):
logger.warning(
f'Not enough {trade.safe_base_currency} in wallet to exit {trade}. '
'Trying to recover.')
self.handle_onexchange_order(trade)
try:
try:
if (self.strategy.order_types.get('stoploss_on_exchange') and
@@ -1536,13 +1579,13 @@ class FreqtradeBot(LoggingMixin):
# Update wallets to ensure amounts tied up in a stoploss is now free!
self.wallets.update()
if self.trading_mode == TradingMode.FUTURES:
# A safe exit amount isn't needed for futures, you can just exit/close the position
return amount
trade_base_currency = self.exchange.get_pair_base_currency(pair)
wallet_amount = self.wallets.get_free(trade_base_currency)
logger.debug(f"{pair} - Wallet: {wallet_amount} - Trade-amount: {amount}")
if wallet_amount >= amount:
# A safe exit amount isn't needed for futures, you can just exit/close the position
return amount
elif wallet_amount > amount * 0.98:
logger.info(f"{pair} - Falling back to wallet-amount {wallet_amount} -> {amount}.")
@@ -1698,8 +1741,8 @@ class FreqtradeBot(LoggingMixin):
'enter_tag': trade.enter_tag,
'sell_reason': trade.exit_reason, # Deprecated
'exit_reason': trade.exit_reason,
'open_date': trade.open_date,
'close_date': trade.close_date or datetime.utcnow(),
'open_date': trade.open_date_utc,
'close_date': trade.close_date_utc or datetime.now(timezone.utc),
'stake_amount': trade.stake_amount,
'stake_currency': self.config['stake_currency'],
'base_currency': self.exchange.get_pair_base_currency(trade.pair),

3
freqtrade/optimize/backtesting.py
View File

@@ -89,8 +89,7 @@ class Backtesting:
self.rejected_df: Dict[str, Dict] = {}
self._exchange_name = self.config['exchange']['name']
self.exchange = ExchangeResolver.load_exchange(
self._exchange_name, self.config, load_leverage_tiers=True)
self.exchange = ExchangeResolver.load_exchange(self.config, load_leverage_tiers=True)
self.dataprovider = DataProvider(self.config, self.exchange)
if self.config.get('strategy_list'):

2
freqtrade/optimize/edge_cli.py
View File

@@ -32,7 +32,7 @@ class EdgeCli:
# Ensure using dry-run
self.config['dry_run'] = True
self.config['stake_amount'] = constants.UNLIMITED_STAKE_AMOUNT
self.exchange = ExchangeResolver.load_exchange(self.config['exchange']['name'], self.config)
self.exchange = ExchangeResolver.load_exchange(self.config)
self.strategy = StrategyResolver.load_strategy(self.config)
self.strategy.dp = DataProvider(config, self.exchange)

2
freqtrade/persistence/trade_model.py
View File

@@ -425,7 +425,7 @@ class LocalTrade():
@property
def close_date_utc(self):
return self.close_date.replace(tzinfo=timezone.utc)
return self.close_date.replace(tzinfo=timezone.utc) if self.close_date else None
@property
def entry_side(self) -> str:

4
freqtrade/plot/plotting.py
View File

@@ -633,7 +633,7 @@ def load_and_plot_trades(config: Config):
"""
strategy = StrategyResolver.load_strategy(config)
exchange = ExchangeResolver.load_exchange(config['exchange']['name'], config)
exchange = ExchangeResolver.load_exchange(config)
IStrategy.dp = DataProvider(config, exchange)
strategy.ft_bot_start()
strategy.bot_loop_start(datetime.now(timezone.utc))
@@ -678,7 +678,7 @@ def plot_profit(config: Config) -> None:
if 'timeframe' not in config:
raise OperationalException('Timeframe must be set in either config or via --timeframe.')
exchange = ExchangeResolver.load_exchange(config['exchange']['name'], config)
exchange = ExchangeResolver.load_exchange(config)
plot_elements = init_plotscript(config, list(exchange.markets))
trades = plot_elements['trades']
# Filter trades to relevant pairs

3
freqtrade/resolvers/exchange_resolver.py
View File

@@ -19,13 +19,14 @@ class ExchangeResolver(IResolver):
object_type = Exchange
@staticmethod
def load_exchange(exchange_name: str, config: Config, validate: bool = True,
def load_exchange(config: Config, validate: bool = True,
load_leverage_tiers: bool = False) -> Exchange:
"""
Load the custom class from config parameter
:param exchange_name: name of the Exchange to load
:param config: configuration dictionary
"""
exchange_name: str = config['exchange']['name']
# Map exchange name to avoid duplicate classes for identical exchanges
exchange_name = MAP_EXCHANGE_CHILDCLASS.get(exchange_name, exchange_name)
exchange_name = exchange_name.title()

11
freqtrade/rpc/api_server/api_v1.py
View File

@@ -44,7 +44,8 @@ logger = logging.getLogger(__name__)
# 2.24: Add cancel_open_order endpoint
# 2.25: Add several profit values to /status endpoint
# 2.26: increase /balance output
API_VERSION = 2.26
# 2.27: Add /trades/<id>/reload endpoint
API_VERSION = 2.27
# Public API, requires no auth.
router_public = APIRouter()
@@ -127,11 +128,17 @@ def trades_delete(tradeid: int, rpc: RPC = Depends(get_rpc)):
@router.delete('/trades/{tradeid}/open-order', response_model=OpenTradeSchema, tags=['trading'])
def cancel_open_order(tradeid: int, rpc: RPC = Depends(get_rpc)):
def trade_cancel_open_order(tradeid: int, rpc: RPC = Depends(get_rpc)):
rpc._rpc_cancel_open_order(tradeid)
return rpc._rpc_trade_status([tradeid])[0]
@router.get('/trades/{tradeid}/reload', response_model=OpenTradeSchema, tags=['trading'])
def trade_reload(tradeid: int, rpc: RPC = Depends(get_rpc)):
rpc._rpc_reload_trade_from_exchange(tradeid)
return rpc._rpc_trade_status([tradeid])[0]
# TODO: Missing response model
@router.get('/edge', tags=['info'])
def edge(rpc: RPC = Depends(get_rpc)):

2
freqtrade/rpc/api_server/deps.py
View File

@@ -46,7 +46,7 @@ def get_exchange(config=Depends(get_config)):
if not ApiServer._exchange:
from freqtrade.resolvers import ExchangeResolver
ApiServer._exchange = ExchangeResolver.load_exchange(
config['exchange']['name'], config, load_leverage_tiers=False)
config, load_leverage_tiers=False)
return ApiServer._exchange

12
freqtrade/rpc/rpc.py
View File

@@ -740,6 +740,18 @@ class RPC:
return {'status': 'No more entries will occur from now. Run /reload_config to reset.'}
def _rpc_reload_trade_from_exchange(self, trade_id: int) -> Dict[str, str]:
"""
Handler for reload_trade_from_exchange.
Reloads a trade from it's orders, should manual interaction have happened.
"""
trade = Trade.get_trades(trade_filter=[Trade.id == trade_id]).first()
if not trade:
raise RPCException(f"Could not find trade with id {trade_id}.")
self._freqtrade.handle_onexchange_order(trade)
return {'status': 'Reloaded from orders from exchange'}
def __exec_force_exit(self, trade: Trade, ordertype: Optional[str],
amount: Optional[float] = None) -> None:
# Check if there is there is an open order

13
freqtrade/rpc/telegram.py
View File

@@ -196,6 +196,7 @@ class Telegram(RPCHandler):
self._force_enter, order_side=SignalDirection.LONG)),
CommandHandler('forceshort', partial(
self._force_enter, order_side=SignalDirection.SHORT)),
CommandHandler('reload_trade', self._reload_trade_from_exchange),
CommandHandler('trades', self._trades),
CommandHandler('delete', self._delete_trade),
CommandHandler(['coo', 'cancel_open_order'], self._cancel_open_order),
@@ -1074,6 +1075,17 @@ class Telegram(RPCHandler):
msg = self._rpc._rpc_stopentry()
await self._send_msg(f"Status: `{msg['status']}`")
@authorized_only
async def _reload_trade_from_exchange(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /reload_trade <tradeid>.
"""
if not context.args or len(context.args) == 0:
raise RPCException("Trade-id not set.")
trade_id = int(context.args[0])
msg = self._rpc._rpc_reload_trade_from_exchange(trade_id)
await self._send_msg(f"Status: `{msg['status']}`")
@authorized_only
async def _force_exit(self, update: Update, context: CallbackContext) -> None:
"""
@@ -1561,6 +1573,7 @@ class Telegram(RPCHandler):
"*/fx <trade_id>|all:* `Alias to /forceexit`\n"
f"{force_enter_text if self._config.get('force_entry_enable', False) else ''}"
"*/delete <trade_id>:* `Instantly delete the given trade in the database`\n"
"*/reload_trade <trade_id>:* `Relade trade from exchange Orders`\n"
"*/cancel_open_order <trade_id>:* `Cancels open orders for trade. "
"Only valid when the trade has open orders.`\n"
"*/coo <trade_id>|all:* `Alias to /cancel_open_order`\n"

13
freqtrade/templates/FreqaiExampleStrategy.py
View File

@@ -15,12 +15,15 @@ logger = logging.getLogger(__name__)
class FreqaiExampleStrategy(IStrategy):
"""
Example strategy showing how the user connects their own
IFreqaiModel to the strategy. Namely, the user uses:
self.freqai.start(dataframe, metadata)
IFreqaiModel to the strategy.
to make predictions on their data. feature_engineering_*() automatically
generate the variety of features indicated by the user in the
canonical freqtrade configuration file under config['freqai'].
Warning! This is a showcase of functionality,
which means that it is designed to show various functions of FreqAI
and it runs on all computers. We use this showcase to help users
understand how to build a strategy, and we use it as a benchmark
to help debug possible problems.
This means this is *not* meant to be run live in production.
"""
minimal_roi = {"0": 0.1, "240": -1}

29
freqtrade/wallets.py
View File

@@ -181,6 +181,35 @@ class Wallets:
def get_all_positions(self) -> Dict[str, PositionWallet]:
return self._positions
def _check_exit_amount(self, trade: Trade) -> bool:
if trade.trading_mode != TradingMode.FUTURES:
# Slightly higher offset than in safe_exit_amount.
wallet_amount: float = self.get_total(trade.safe_base_currency) * (2 - 0.981)
else:
# wallet_amount: float = self.wallets.get_free(trade.safe_base_currency)
position = self._positions.get(trade.pair)
if position is None:
# We don't own anything :O
return False
wallet_amount = position.position
if wallet_amount >= trade.amount:
return True
return False
def check_exit_amount(self, trade: Trade) -> bool:
"""
Checks if the exit amount is available in the wallet.
:param trade: Trade to check
:return: True if the exit amount is available, False otherwise
"""
if not self._check_exit_amount(trade):
# Update wallets just to make sure
self.update()
return self._check_exit_amount(trade)
return True
def get_starting_balance(self) -> float:
"""
Retrieves starting balance - based on either available capital,

3
requirements-freqai.txt
View File

@@ -5,7 +5,8 @@
# Required for freqai
scikit-learn==1.1.3
joblib==1.2.0
catboost==1.1.1; platform_machine != 'aarch64' and 'arm' not in platform_machine and python_version < '3.11'
catboost==1.1.1; sys_platform == 'darwin' and python_version < '3.9'
catboost==1.2; 'arm' not in platform_machine and (sys_platform != 'darwin' or python_version >= '3.9')
lightgbm==3.3.5
xgboost==1.7.5
tensorboard==2.13.0

4
setup.py
View File

@@ -69,9 +69,9 @@ setup(
],
install_requires=[
# from requirements.txt
'ccxt>=2.6.26',
'ccxt>=3.0.0',
'SQLAlchemy>=2.0.6',
'python-telegram-bot>=13.4',
'python-telegram-bot>=20.1',
'arrow>=0.17.0',
'cachetools',
'requests',

4
setup.sh
View File

@@ -25,7 +25,7 @@ function check_installed_python() {
exit 2
fi
for v in 10 9 8
for v in 11 10 9 8
do
PYTHON="python3.${v}"
which $PYTHON
@@ -258,7 +258,7 @@ function install() {
install_redhat
else
echo "This script does not support your OS."
echo "If you have Python version 3.8 - 3.10, pip, virtualenv, ta-lib you can continue."
echo "If you have Python version 3.8 - 3.11, pip, virtualenv, ta-lib you can continue."
echo "Wait 10 seconds to continue the next install steps or use ctrl+c to interrupt this shell."
sleep 10
fi

11
tests/conftest.py
View File

@@ -181,7 +181,7 @@ def get_patched_exchange(mocker, config, api_mock=None, id='binance',
patch_exchange(mocker, api_mock, id, mock_markets, mock_supported_modes)
config['exchange']['name'] = id
try:
exchange = ExchangeResolver.load_exchange(id, config, load_leverage_tiers=True)
exchange = ExchangeResolver.load_exchange(config, load_leverage_tiers=True)
except ImportError:
exchange = Exchange(config)
return exchange
@@ -411,6 +411,14 @@ def patch_gc(mocker) -> None:
mocker.patch("freqtrade.main.gc_set_threshold")
@pytest.fixture(autouse=True)
def user_dir(mocker, tmpdir) -> Path:
user_dir = Path(tmpdir) / "user_data"
mocker.patch('freqtrade.configuration.configuration.create_userdata_dir',
return_value=user_dir)
return user_dir
@pytest.fixture(autouse=True)
def patch_coingekko(mocker) -> None:
"""
@@ -485,7 +493,6 @@ def get_default_conf(testdatadir):
},
"exchange": {
"name": "binance",
"enabled": True,
"key": "key",
"secret": "secret",
"pair_whitelist": [

7
tests/data/test_entryexitanalysis.py
View File

@@ -18,8 +18,9 @@ def entryexitanalysis_cleanup() -> None:
Backtesting.cleanup()
def test_backtest_analysis_nomock(default_conf, mocker, caplog, testdatadir, tmpdir, capsys):
def test_backtest_analysis_nomock(default_conf, mocker, caplog, testdatadir, user_dir, capsys):
caplog.set_level(logging.INFO)
(user_dir / 'backtest_results').mkdir(parents=True, exist_ok=True)
default_conf.update({
"use_exit_signal": True,
@@ -80,7 +81,7 @@ def test_backtest_analysis_nomock(default_conf, mocker, caplog, testdatadir, tmp
'backtesting',
'--config', 'config.json',
'--datadir', str(testdatadir),
'--user-data-dir', str(tmpdir),
'--user-data-dir', str(user_dir),
'--timeframe', '5m',
'--timerange', '1515560100-1517287800',
'--export', 'signals',
@@ -98,7 +99,7 @@ def test_backtest_analysis_nomock(default_conf, mocker, caplog, testdatadir, tmp
'backtesting-analysis',
'--config', 'config.json',
'--datadir', str(testdatadir),
'--user-data-dir', str(tmpdir),
'--user-data-dir', str(user_dir),
]
# test group 0 and indicator list

4
tests/exchange/test_ccxt_compat.py
View File

@@ -302,7 +302,7 @@ def exchange(request, exchange_conf):
exchange_conf, EXCHANGES[request.param].get('use_ci_proxy', False))
exchange_conf['exchange']['name'] = request.param
exchange_conf['stake_currency'] = EXCHANGES[request.param]['stake_currency']
exchange = ExchangeResolver.load_exchange(request.param, exchange_conf, validate=True)
exchange = ExchangeResolver.load_exchange(exchange_conf, validate=True)
yield exchange, request.param
@@ -330,7 +330,7 @@ def exchange_futures(request, exchange_conf, class_mocker):
class_mocker.patch(f'{EXMS}.cache_leverage_tiers')
exchange = ExchangeResolver.load_exchange(
request.param, exchange_conf, validate=True, load_leverage_tiers=True)
exchange_conf, validate=True, load_leverage_tiers=True)
yield exchange, request.param

97
tests/exchange/test_exchange.py
View File

@@ -228,27 +228,30 @@ def test_exchange_resolver(default_conf, mocker, caplog):
mocker.patch(f'{EXMS}.validate_timeframes')
mocker.patch(f'{EXMS}.validate_stakecurrency')
mocker.patch(f'{EXMS}.validate_pricing')
exchange = ExchangeResolver.load_exchange('zaif', default_conf)
default_conf['exchange']['name'] = 'zaif'
exchange = ExchangeResolver.load_exchange(default_conf)
assert isinstance(exchange, Exchange)
assert log_has_re(r"No .* specific subclass found. Using the generic class instead.", caplog)
caplog.clear()
exchange = ExchangeResolver.load_exchange('Bittrex', default_conf)
default_conf['exchange']['name'] = 'Bittrex'
exchange = ExchangeResolver.load_exchange(default_conf)
assert isinstance(exchange, Exchange)
assert isinstance(exchange, Bittrex)
assert not log_has_re(r"No .* specific subclass found. Using the generic class instead.",
caplog)
caplog.clear()
exchange = ExchangeResolver.load_exchange('kraken', default_conf)
default_conf['exchange']['name'] = 'kraken'
exchange = ExchangeResolver.load_exchange(default_conf)
assert isinstance(exchange, Exchange)
assert isinstance(exchange, Kraken)
assert not isinstance(exchange, Binance)
assert not log_has_re(r"No .* specific subclass found. Using the generic class instead.",
caplog)
exchange = ExchangeResolver.load_exchange('binance', default_conf)
default_conf['exchange']['name'] = 'binance'
exchange = ExchangeResolver.load_exchange(default_conf)
assert isinstance(exchange, Exchange)
assert isinstance(exchange, Binance)
assert not isinstance(exchange, Kraken)
@@ -257,7 +260,8 @@ def test_exchange_resolver(default_conf, mocker, caplog):
caplog)
# Test mapping
exchange = ExchangeResolver.load_exchange('binanceus', default_conf)
default_conf['exchange']['name'] = 'binanceus'
exchange = ExchangeResolver.load_exchange(default_conf)
assert isinstance(exchange, Exchange)
assert isinstance(exchange, Binance)
assert not isinstance(exchange, Kraken)
@@ -990,19 +994,20 @@ def test_validate_pricing(default_conf, mocker):
mocker.patch(f'{EXMS}.validate_timeframes')
mocker.patch(f'{EXMS}.validate_stakecurrency')
mocker.patch(f'{EXMS}.name', 'Binance')
ExchangeResolver.load_exchange('binance', default_conf)
default_conf['exchange']['name'] = 'binance'
ExchangeResolver.load_exchange(default_conf)
has.update({'fetchTicker': False})
with pytest.raises(OperationalException, match="Ticker pricing not available for .*"):
ExchangeResolver.load_exchange('binance', default_conf)
ExchangeResolver.load_exchange(default_conf)
has.update({'fetchTicker': True})
default_conf['exit_pricing']['use_order_book'] = True
ExchangeResolver.load_exchange('binance', default_conf)
ExchangeResolver.load_exchange(default_conf)
has.update({'fetchL2OrderBook': False})
with pytest.raises(OperationalException, match="Orderbook not available for .*"):
ExchangeResolver.load_exchange('binance', default_conf)
ExchangeResolver.load_exchange(default_conf)
has.update({'fetchL2OrderBook': True})
@@ -1011,7 +1016,7 @@ def test_validate_pricing(default_conf, mocker):
default_conf['margin_mode'] = MarginMode.ISOLATED
with pytest.raises(OperationalException, match="Ticker pricing not available for .*"):
ExchangeResolver.load_exchange('binance', default_conf)
ExchangeResolver.load_exchange(default_conf)
def test_validate_ordertypes(default_conf, mocker):
@@ -1091,12 +1096,13 @@ def test_validate_ordertypes_stop_advanced(default_conf, mocker, exchange_name,
'stoploss_on_exchange': True,
'stoploss_price_type': stopadv,
}
default_conf['exchange']['name'] = exchange_name
if expected:
ExchangeResolver.load_exchange(exchange_name, default_conf)
ExchangeResolver.load_exchange(default_conf)
else:
with pytest.raises(OperationalException,
match=r'On exchange stoploss price type is not supported for .*'):
ExchangeResolver.load_exchange(exchange_name, default_conf)
ExchangeResolver.load_exchange(default_conf)
def test_validate_order_types_not_in_config(default_conf, mocker):
@@ -1773,6 +1779,71 @@ def test_fetch_positions(default_conf, mocker, exchange_name):
"fetch_positions", "fetch_positions")
@pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_fetch_orders(default_conf, mocker, exchange_name, limit_order):
api_mock = MagicMock()
api_mock.fetch_orders = MagicMock(return_value=[
limit_order['buy'],
limit_order['sell'],
])
api_mock.fetch_open_orders = MagicMock(return_value=[limit_order['buy']])
api_mock.fetch_closed_orders = MagicMock(return_value=[limit_order['buy']])
mocker.patch(f'{EXMS}.exchange_has', return_value=True)
start_time = datetime.now(timezone.utc) - timedelta(days=5)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name)
# Not available in dry-run
assert exchange.fetch_orders('mocked', start_time) == []
assert api_mock.fetch_orders.call_count == 0
default_conf['dry_run'] = False
exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name)
res = exchange.fetch_orders('mocked', start_time)
assert api_mock.fetch_orders.call_count == 1
assert api_mock.fetch_open_orders.call_count == 0
assert api_mock.fetch_closed_orders.call_count == 0
assert len(res) == 2
res = exchange.fetch_orders('mocked', start_time)
api_mock.fetch_orders.reset_mock()
def has_resp(_, endpoint):
if endpoint == 'fetchOrders':
return False
if endpoint == 'fetchClosedOrders':
return True
if endpoint == 'fetchOpenOrders':
return True
mocker.patch(f'{EXMS}.exchange_has', has_resp)
# happy path without fetchOrders
res = exchange.fetch_orders('mocked', start_time)
assert api_mock.fetch_orders.call_count == 0
assert api_mock.fetch_open_orders.call_count == 1
assert api_mock.fetch_closed_orders.call_count == 1
mocker.patch(f'{EXMS}.exchange_has', return_value=True)
ccxt_exceptionhandlers(mocker, default_conf, api_mock, exchange_name,
"fetch_orders", "fetch_orders", retries=1,
pair='mocked', since=start_time)
# Unhappy path - first fetch-orders call fails.
api_mock.fetch_orders = MagicMock(side_effect=ccxt.NotSupported())
api_mock.fetch_open_orders.reset_mock()
api_mock.fetch_closed_orders.reset_mock()
res = exchange.fetch_orders('mocked', start_time)
assert api_mock.fetch_orders.call_count == 1
assert api_mock.fetch_open_orders.call_count == 1
assert api_mock.fetch_closed_orders.call_count == 1
def test_fetch_trading_fees(default_conf, mocker):
api_mock = MagicMock()
tick = {

2
tests/freqai/test_freqai_interface.py
View File

@@ -29,7 +29,7 @@ def is_arm() -> bool:
def can_run_model(model: str) -> None:
if (is_arm() or is_py11()) and "Catboost" in model:
if is_arm() and "Catboost" in model:
pytest.skip("CatBoost is not supported on ARM.")
is_pytorch_model = 'Reinforcement' in model or 'PyTorch' in model

5
tests/freqai/test_models/ReinforcementLearner_test_3ac.py
View File

@@ -18,6 +18,11 @@ class ReinforcementLearner_test_3ac(ReinforcementLearner):
"""
User can override any function in BaseRLEnv and gym.Env. Here the user
sets a custom reward based on profit and trade duration.
Warning!
This is function is a showcase of functionality designed to show as many possible
environment control features as possible. It is also designed to run quickly
on small computers. This is a benchmark, it is *not* for live production.
"""
def calculate_reward(self, action: int) -> float:

5
tests/freqai/test_models/ReinforcementLearner_test_4ac.py
View File

@@ -18,6 +18,11 @@ class ReinforcementLearner_test_4ac(ReinforcementLearner):
"""
User can override any function in BaseRLEnv and gym.Env. Here the user
sets a custom reward based on profit and trade duration.
Warning!
This is function is a showcase of functionality designed to show as many possible
environment control features as possible. It is also designed to run quickly
on small computers. This is a benchmark, it is *not* for live production.
"""
def calculate_reward(self, action: int) -> float:

4
tests/persistence/test_persistence.py
View File

@@ -239,7 +239,7 @@ def test_interest(fee, exchange, is_short, lev, minutes, rate, interest,
stake_amount=20.0,
amount=30.0,
open_rate=2.0,
open_date=datetime.utcnow() - timedelta(minutes=minutes),
open_date=datetime.now(timezone.utc) - timedelta(minutes=minutes),
fee_open=fee.return_value,
fee_close=fee.return_value,
exchange=exchange,
@@ -2063,7 +2063,7 @@ def test_trade_truncates_string_fields():
stake_amount=20.0,
amount=30.0,
open_rate=2.0,
open_date=datetime.utcnow() - timedelta(minutes=20),
open_date=datetime.now(timezone.utc) - timedelta(minutes=20),
fee_open=0.001,
fee_close=0.001,
exchange='binance',

10
tests/plugins/test_protections.py
View File

@@ -1,5 +1,5 @@
import random
from datetime import datetime, timedelta
from datetime import datetime, timedelta, timezone
import pytest
@@ -24,8 +24,8 @@ def generate_mock_trade(pair: str, fee: float, is_open: bool,
stake_amount=0.01,
fee_open=fee,
fee_close=fee,
open_date=datetime.utcnow() - timedelta(minutes=min_ago_open or 200),
close_date=datetime.utcnow() - timedelta(minutes=min_ago_close or 30),
open_date=datetime.now(timezone.utc) - timedelta(minutes=min_ago_open or 200),
close_date=datetime.now(timezone.utc) - timedelta(minutes=min_ago_close or 30),
open_rate=open_rate,
is_open=is_open,
amount=0.01 / open_rate,
@@ -87,9 +87,9 @@ def test_protectionmanager(mocker, default_conf):
for handler in freqtrade.protections._protection_handlers:
assert handler.name in constants.AVAILABLE_PROTECTIONS
if not handler.has_global_stop:
assert handler.global_stop(datetime.utcnow(), '*') is None
assert handler.global_stop(datetime.now(timezone.utc), '*') is None
if not handler.has_local_stop:
assert handler.stop_per_pair('XRP/BTC', datetime.utcnow(), '*') is None
assert handler.stop_per_pair('XRP/BTC', datetime.now(timezone.utc), '*') is None
@pytest.mark.parametrize('timeframe,expected,protconf', [

5
tests/rpc/test_rpc.py
View File

@@ -261,8 +261,7 @@ def test_rpc_status_table(default_conf, ticker, fee, mocker) -> None:
assert isnan(fiat_profit_sum)
def test__rpc_timeunit_profit(default_conf_usdt, ticker, fee,
limit_buy_order, limit_sell_order, markets, mocker) -> None:
def test__rpc_timeunit_profit(default_conf_usdt, ticker, fee, markets, mocker) -> None:
mocker.patch('freqtrade.rpc.telegram.Telegram', MagicMock())
mocker.patch.multiple(
EXMS,
@@ -295,7 +294,7 @@ def test__rpc_timeunit_profit(default_conf_usdt, ticker, fee,
assert day['starting_balance'] in (pytest.approx(1062.37), pytest.approx(1066.46))
assert day['fiat_value'] in (0.0, )
# ensure first day is current date
assert str(days['data'][0]['date']) == str(datetime.utcnow().date())
assert str(days['data'][0]['date']) == str(datetime.now(timezone.utc).date())
# Try invalid data
with pytest.raises(RPCException, match=r'.*must be an integer greater than 0*'):

31
tests/rpc/test_rpc_apiserver.py
View File

@@ -601,7 +601,7 @@ def test_api_daily(botclient, mocker, ticker, fee, markets):
assert len(rc.json()['data']) == 7
assert rc.json()['stake_currency'] == 'BTC'
assert rc.json()['fiat_display_currency'] == 'USD'
assert rc.json()['data'][0]['date'] == str(datetime.utcnow().date())
assert rc.json()['data'][0]['date'] == str(datetime.now(timezone.utc).date())
@pytest.mark.parametrize('is_short', [True, False])
@@ -740,6 +740,33 @@ def test_api_delete_open_order(botclient, mocker, fee, markets, ticker, is_short
assert cancel_mock.call_count == 1
@pytest.mark.parametrize('is_short', [True, False])
def test_api_trade_reload_trade(botclient, mocker, fee, markets, ticker, is_short):
ftbot, client = botclient
patch_get_signal(ftbot, enter_long=not is_short, enter_short=is_short)
stoploss_mock = MagicMock()
cancel_mock = MagicMock()
ftbot.handle_onexchange_order = MagicMock()
mocker.patch.multiple(
EXMS,
markets=PropertyMock(return_value=markets),
fetch_ticker=ticker,
cancel_order=cancel_mock,
cancel_stoploss_order=stoploss_mock,
)
rc = client_get(client, f"{BASE_URI}/trades/10/reload")
assert_response(rc, 502)
assert 'Could not find trade with id 10.' in rc.json()['error']
assert ftbot.handle_onexchange_order.call_count == 0
create_mock_trades(fee, is_short=is_short)
Trade.commit()
rc = client_get(client, f"{BASE_URI}/trades/5/reload")
assert ftbot.handle_onexchange_order.call_count == 1
def test_api_logs(botclient):
ftbot, client = botclient
rc = client_get(client, f"{BASE_URI}/logs")
@@ -1197,7 +1224,7 @@ def test_api_force_entry(botclient, mocker, fee, endpoint):
stake_amount=1,
open_rate=0.245441,
open_order_id="123456",
open_date=datetime.utcnow(),
open_date=datetime.now(timezone.utc),
is_open=False,
is_short=False,
fee_close=fee.return_value,

38
tests/rpc/test_rpc_telegram.py
View File

@@ -52,7 +52,7 @@ def default_conf(default_conf) -> dict:
@pytest.fixture
def update():
message = Message(0, datetime.utcnow(), Chat(0, 0))
message = Message(0, datetime.now(timezone.utc), Chat(0, 0))
_update = Update(0, message=message)
return _update
@@ -143,8 +143,8 @@ def test_telegram_init(default_conf, mocker, caplog) -> None:
message_str = ("rpc.telegram is listening for following commands: [['status'], ['profit'], "
"['balance'], ['start'], ['stop'], "
"['forceexit', 'forcesell', 'fx'], ['forcebuy', 'forcelong'], ['forceshort'], "
"['trades'], ['delete'], ['cancel_open_order', 'coo'], ['performance'], "
"['buys', 'entries'], ['exits', 'sells'], ['mix_tags'], "
"['reload_trade'], ['trades'], ['delete'], ['cancel_open_order', 'coo'], "
"['performance'], ['buys', 'entries'], ['exits', 'sells'], ['mix_tags'], "
"['stats'], ['daily'], ['weekly'], ['monthly'], "
"['count'], ['locks'], ['delete_locks', 'unlock'], "
"['reload_conf', 'reload_config'], ['show_conf', 'show_config'], "
@@ -213,7 +213,7 @@ async def test_authorized_only_unauthorized(default_conf, mocker, caplog) -> Non
patch_exchange(mocker)
caplog.set_level(logging.DEBUG)
chat = Chat(0xdeadbeef, 0)
message = Message(randint(1, 100), datetime.utcnow(), chat)
message = Message(randint(1, 100), datetime.now(timezone.utc), chat)
update = Update(randint(1, 100), message=message)
default_conf['telegram']['enabled'] = False
@@ -520,7 +520,7 @@ async def test_daily_handle(default_conf_usdt, update, ticker, fee, mocker, time
assert msg_mock.call_count == 1
assert "Daily Profit over the last 2 days</b>:" in msg_mock.call_args_list[0][0][0]
assert 'Day ' in msg_mock.call_args_list[0][0][0]
assert str(datetime.utcnow().date()) in msg_mock.call_args_list[0][0][0]
assert str(datetime.now(timezone.utc).date()) in msg_mock.call_args_list[0][0][0]
assert ' 6.83 USDT' in msg_mock.call_args_list[0][0][0]
assert ' 7.51 USD' in msg_mock.call_args_list[0][0][0]
assert '(2)' in msg_mock.call_args_list[0][0][0]
@@ -533,8 +533,9 @@ async def test_daily_handle(default_conf_usdt, update, ticker, fee, mocker, time
await telegram._daily(update=update, context=context)
assert msg_mock.call_count == 1
assert "Daily Profit over the last 7 days</b>:" in msg_mock.call_args_list[0][0][0]
assert str(datetime.utcnow().date()) in msg_mock.call_args_list[0][0][0]
assert str((datetime.utcnow() - timedelta(days=5)).date()) in msg_mock.call_args_list[0][0][0]
assert str(datetime.now(timezone.utc).date()) in msg_mock.call_args_list[0][0][0]
assert str((datetime.now(timezone.utc) - timedelta(days=5)).date()
) in msg_mock.call_args_list[0][0][0]
assert ' 6.83 USDT' in msg_mock.call_args_list[0][0][0]
assert ' 7.51 USD' in msg_mock.call_args_list[0][0][0]
assert '(2)' in msg_mock.call_args_list[0][0][0]
@@ -608,7 +609,7 @@ async def test_weekly_handle(default_conf_usdt, update, ticker, fee, mocker, tim
assert "Weekly Profit over the last 2 weeks (starting from Monday)</b>:" \
in msg_mock.call_args_list[0][0][0]
assert 'Monday ' in msg_mock.call_args_list[0][0][0]
today = datetime.utcnow().date()
today = datetime.now(timezone.utc).date()
first_iso_day_of_current_week = today - timedelta(days=today.weekday())
assert str(first_iso_day_of_current_week) in msg_mock.call_args_list[0][0][0]
assert ' 2.74 USDT' in msg_mock.call_args_list[0][0][0]
@@ -677,7 +678,7 @@ async def test_monthly_handle(default_conf_usdt, update, ticker, fee, mocker, ti
assert msg_mock.call_count == 1
assert 'Monthly Profit over the last 2 months</b>:' in msg_mock.call_args_list[0][0][0]
assert 'Month ' in msg_mock.call_args_list[0][0][0]
today = datetime.utcnow().date()
today = datetime.now(timezone.utc).date()
current_month = f"{today.year}-{today.month:02} "
assert current_month in msg_mock.call_args_list[0][0][0]
assert ' 2.74 USDT' in msg_mock.call_args_list[0][0][0]
@@ -1763,6 +1764,25 @@ async def test_telegram_delete_trade(mocker, update, default_conf, fee, is_short
assert "Please make sure to take care of this asset" in msg_mock.call_args_list[0][0][0]
@pytest.mark.parametrize('is_short', [True, False])
async def test_telegram_reload_trade_from_exchange(mocker, update, default_conf, fee, is_short):
telegram, _, msg_mock = get_telegram_testobject(mocker, default_conf)
context = MagicMock()
context.args = []
await telegram._reload_trade_from_exchange(update=update, context=context)
assert "Trade-id not set." in msg_mock.call_args_list[0][0][0]
msg_mock.reset_mock()
create_mock_trades(fee, is_short=is_short)
context.args = [5]
await telegram._reload_trade_from_exchange(update=update, context=context)
assert "Status: `Reloaded from orders from exchange`" in msg_mock.call_args_list[0][0][0]
@pytest.mark.parametrize('is_short', [True, False])
async def test_telegram_delete_open_order(mocker, update, default_conf, fee, is_short, ticker):

6
tests/strategy/test_default_strategy.py
View File

@@ -1,4 +1,4 @@
from datetime import datetime
from datetime import datetime, timezone
import pytest
from pandas import DataFrame
@@ -43,12 +43,12 @@ def test_strategy_test_v3(dataframe_1m, fee, is_short, side):
assert strategy.confirm_trade_entry(pair='ETH/BTC', order_type='limit', amount=0.1,
rate=20000, time_in_force='gtc',
current_time=datetime.utcnow(),
current_time=datetime.now(timezone.utc),
side=side, entry_tag=None) is True
assert strategy.confirm_trade_exit(pair='ETH/BTC', trade=trade, order_type='limit', amount=0.1,
rate=20000, time_in_force='gtc', exit_reason='roi',
sell_reason='roi',
current_time=datetime.utcnow(),
current_time=datetime.now(timezone.utc),
side=side) is True
assert strategy.custom_stoploss(pair='ETH/BTC', trade=trade, current_time=datetime.now(),

4
tests/test_configuration.py
View File

@@ -1271,7 +1271,7 @@ def test_pairlist_resolving_with_config_pl_not_exists(mocker, default_conf):
configuration.get_config()
def test_pairlist_resolving_fallback(mocker):
def test_pairlist_resolving_fallback(mocker, tmpdir):
mocker.patch.object(Path, "exists", MagicMock(return_value=True))
mocker.patch.object(Path, "open", MagicMock(return_value=MagicMock()))
mocker.patch("freqtrade.configuration.configuration.load_file",
@@ -1290,7 +1290,7 @@ def test_pairlist_resolving_fallback(mocker):
assert config['pairs'] == ['ETH/BTC', 'XRP/BTC']
assert config['exchange']['name'] == 'binance'
assert config['datadir'] == Path.cwd() / "user_data/data/binance"
assert config['datadir'] == Path(tmpdir) / "user_data/data/binance"
@pytest.mark.parametrize("setting", [

45
tests/test_freqtradebot.py
View File

@@ -5552,6 +5552,51 @@ def test_handle_insufficient_funds(mocker, default_conf_usdt, fee, is_short, cap
assert log_has(f"Error updating {order['id']}.", caplog)
@pytest.mark.usefixtures("init_persistence")
@pytest.mark.parametrize("is_short", [False, True])
def test_handle_onexchange_order(mocker, default_conf_usdt, limit_order, is_short, caplog):
freqtrade = get_patched_freqtradebot(mocker, default_conf_usdt)
mock_uts = mocker.spy(freqtrade, 'update_trade_state')
entry_order = limit_order[entry_side(is_short)]
exit_order = limit_order[exit_side(is_short)]
mock_fo = mocker.patch(f'{EXMS}.fetch_orders', return_value=[
entry_order,
exit_order,
])
order_id = entry_order['id']
trade = Trade(
open_order_id=order_id,
pair='ETH/USDT',
fee_open=0.001,
fee_close=0.001,
open_rate=entry_order['price'],
open_date=arrow.utcnow().datetime,
stake_amount=entry_order['cost'],
amount=entry_order['amount'],
exchange="binance",
is_short=is_short,
leverage=1,
)
trade.orders.append(Order.parse_from_ccxt_object(
entry_order, 'ADA/USDT', entry_side(is_short))
)
Trade.session.add(trade)
freqtrade.handle_onexchange_order(trade)
assert log_has_re(r"Found previously unknown order .*", caplog)
assert mock_uts.call_count == 1
assert mock_fo.call_count == 1
trade = Trade.session.scalars(select(Trade)).first()
assert len(trade.orders) == 2
assert trade.is_open is False
assert trade.exit_reason == ExitType.SOLD_ON_EXCHANGE.value
def test_get_valid_price(mocker, default_conf_usdt) -> None:
patch_RPCManager(mocker)
patch_exchange(mocker)

5
tests/test_integration.py
View File

@@ -75,8 +75,9 @@ def test_may_execute_exit_stoploss_on_exchange_multi(default_conf, ticker, fee,
_notify_exit=MagicMock(),
)
mocker.patch("freqtrade.strategy.interface.IStrategy.should_exit", should_sell_mock)
wallets_mock = mocker.patch("freqtrade.wallets.Wallets.update", MagicMock())
mocker.patch("freqtrade.wallets.Wallets.get_free", MagicMock(return_value=1000))
wallets_mock = mocker.patch("freqtrade.wallets.Wallets.update")
mocker.patch("freqtrade.wallets.Wallets.get_free", return_value=1000)
mocker.patch("freqtrade.wallets.Wallets.check_exit_amount", return_value=True)
freqtrade = get_patched_freqtradebot(mocker, default_conf)
freqtrade.strategy.order_types['stoploss_on_exchange'] = True

7
tests/test_plotting.py
View File

@@ -1,5 +1,4 @@
from copy import deepcopy
from pathlib import Path
from unittest.mock import MagicMock
import pandas as pd
@@ -282,13 +281,13 @@ def test_generate_Plot_filename():
assert fn == "freqtrade-plot-UNITTEST_BTC-5m.html"
def test_generate_plot_file(mocker, caplog):
def test_generate_plot_file(mocker, caplog, user_dir):
fig = generate_empty_figure()
plot_mock = mocker.patch("freqtrade.plot.plotting.plot", MagicMock())
store_plot_file(fig, filename="freqtrade-plot-UNITTEST_BTC-5m.html",
directory=Path("user_data/plot"))
directory=user_dir / "plot")
expected_fn = str(Path("user_data/plot/freqtrade-plot-UNITTEST_BTC-5m.html"))
expected_fn = str(user_dir / "plot/freqtrade-plot-UNITTEST_BTC-5m.html")
assert plot_mock.call_count == 1
assert plot_mock.call_args[0][0] == fig
assert (plot_mock.call_args_list[0][1]['filename']

12
tests/test_strategy_updater.py
View File

@@ -16,18 +16,18 @@ if sys.version_info < (3, 9):
pytest.skip("StrategyUpdater is not compatible with Python 3.8", allow_module_level=True)
def test_strategy_updater_start(tmpdir, capsys) -> None:
def test_strategy_updater_start(user_dir, capsys) -> None:
# Effective test without mocks.
teststrats = Path(__file__).parent / 'strategy/strats'
tmpdirp = Path(tmpdir) / "strategies"
tmpdirp.mkdir()
tmpdirp = Path(user_dir) / "strategies"
tmpdirp.mkdir(parents=True, exist_ok=True)
shutil.copy(teststrats / 'strategy_test_v2.py', tmpdirp)
old_code = (teststrats / 'strategy_test_v2.py').read_text()
args = [
"strategy-updater",
"--userdir",
str(tmpdir),
str(user_dir),
"--strategy-list",
"StrategyTestV2"
]
@@ -36,9 +36,9 @@ def test_strategy_updater_start(tmpdir, capsys) -> None:
start_strategy_update(pargs)
assert Path(tmpdir / "strategies_orig_updater").exists()
assert Path(user_dir / "strategies_orig_updater").exists()
# Backup file exists
assert Path(tmpdir / "strategies_orig_updater" / 'strategy_test_v2.py').exists()
assert Path(user_dir / "strategies_orig_updater" / 'strategy_test_v2.py').exists()
# updated file exists
new_file = Path(tmpdirp / 'strategy_test_v2.py')
assert new_file.exists()

47
tests/test_wallets.py
View File

@@ -3,9 +3,11 @@ from copy import deepcopy
from unittest.mock import MagicMock
import pytest
from sqlalchemy import select
from freqtrade.constants import UNLIMITED_STAKE_AMOUNT
from freqtrade.exceptions import DependencyException
from freqtrade.persistence import Trade
from tests.conftest import EXMS, create_mock_trades, get_patched_freqtradebot, patch_wallet
@@ -364,3 +366,48 @@ def test_sync_wallet_futures_dry(mocker, default_conf, fee):
free = freqtrade.wallets.get_free('BTC')
used = freqtrade.wallets.get_used('BTC')
assert free + used == total
def test_check_exit_amount(mocker, default_conf, fee):
freqtrade = get_patched_freqtradebot(mocker, default_conf)
update_mock = mocker.patch("freqtrade.wallets.Wallets.update")
total_mock = mocker.patch("freqtrade.wallets.Wallets.get_total", return_value=123)
create_mock_trades(fee, is_short=None)
trade = Trade.session.scalars(select(Trade)).first()
assert trade.amount == 123
assert freqtrade.wallets.check_exit_amount(trade) is True
assert update_mock.call_count == 0
assert total_mock.call_count == 1
update_mock.reset_mock()
# Reduce returned amount to below the trade amount - which should
# trigger a wallet update and return False, triggering "order refinding"
total_mock = mocker.patch("freqtrade.wallets.Wallets.get_total", return_value=100)
assert freqtrade.wallets.check_exit_amount(trade) is False
assert update_mock.call_count == 1
assert total_mock.call_count == 2
def test_check_exit_amount_futures(mocker, default_conf, fee):
default_conf['trading_mode'] = 'futures'
default_conf['margin_mode'] = 'isolated'
freqtrade = get_patched_freqtradebot(mocker, default_conf)
total_mock = mocker.patch("freqtrade.wallets.Wallets.get_total", return_value=123)
create_mock_trades(fee, is_short=None)
trade = Trade.session.scalars(select(Trade)).first()
trade.trading_mode = 'futures'
assert trade.amount == 123
assert freqtrade.wallets.check_exit_amount(trade) is True
assert total_mock.call_count == 0
update_mock = mocker.patch("freqtrade.wallets.Wallets.update")
trade.amount = 150
# Reduce returned amount to below the trade amount - which should
# trigger a wallet update and return False, triggering "order refinding"
assert freqtrade.wallets.check_exit_amount(trade) is False
assert total_mock.call_count == 0
assert update_mock.call_count == 1