CTYPTO/freqtrade
1
0
Fork 0
mirror of https://github.com/freqtrade/freqtrade.git synced 2026-01-26 08:50:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/develop' into feature/fetch-public-trades

Browse Source
This commit is contained in:
Joe Schr
2024-03-11 12:29:00 +01:00
parent 6827e17d17 ca4fb98775
commit c12e203054
32 changed files with 729 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
.github/workflows/ci.yml vendored
View File

@@ -124,8 +124,11 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ "macos-latest", "macos-13" ]
os: [ "macos-latest", "macos-13", "macos-14" ]
python-version: ["3.9", "3.10", "3.11", "3.12"]
exclude:
- os: "macos-14"
python-version: "3.9"
steps:
- uses: actions/checkout@v4
@@ -154,7 +157,7 @@ jobs:
run: |
cd build_helpers && ./install_ta-lib.sh ${HOME}/dependencies/; cd ..
- name: Installation - macOS
- name: Installation - macOS (Brew)
run: |
# brew update
# TODO: Should be the brew upgrade
@@ -177,6 +180,9 @@ jobs:
rm /usr/local/bin/python3.12-config || true
brew install hdf5 c-blosc libomp
- name: Installation (python)
run: |
python -m pip install --upgrade pip wheel
export LD_LIBRARY_PATH=${HOME}/dependencies/lib:$LD_LIBRARY_PATH
export TA_LIBRARY_PATH=${HOME}/dependencies/lib
@@ -482,12 +488,12 @@ jobs:
path: dist
- name: Publish to PyPI (Test)
uses: pypa/gh-action-pypi-publish@v1.8.12
uses: pypa/gh-action-pypi-publish@v1.8.14
with:
repository-url: https://test.pypi.org/legacy/
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@v1.8.12
uses: pypa/gh-action-pypi-publish@v1.8.14
deploy-docker:

6
.pre-commit-config.yaml
View File

@@ -16,9 +16,9 @@ repos:
additional_dependencies:
- types-cachetools==5.3.0.7
- types-filelock==3.2.7
- types-requests==2.31.0.20240218
- types-requests==2.31.0.20240311
- types-tabulate==0.9.0.20240106
- types-python-dateutil==2.8.19.20240106
- types-python-dateutil==2.8.19.20240311
- SQLAlchemy==2.0.27
# stages: [push]
@@ -31,7 +31,7 @@ repos:
- repo: https://github.com/charliermarsh/ruff-pre-commit
# Ruff version.
rev: 'v0.2.2'
rev: 'v0.3.0'
hooks:
- id: ruff

4
docs/advanced-backtesting.md
View File

@@ -109,12 +109,12 @@ automatically accessible by including them on the indicator-list, and these incl
- **open_date :** trade open datetime
- **close_date :** trade close datetime
- **min_rate :** minimum price seen throughout the position
- **max_rate :** maxiumum price seen throughout the position
- **max_rate :** maximum price seen throughout the position
- **open :** signal candle open price
- **close :** signal candle close price
- **high :** signal candle high price
- **low :** signal candle low price
- **volume :** signal candle volumne
- **volume :** signal candle volume
- **profit_ratio :** trade profit ratio
- **profit_abs :** absolute profit return of the trade

2
docs/freqai-parameter-table.md
View File

@@ -75,7 +75,7 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| `rl_config` | A dictionary containing the control parameters for a Reinforcement Learning model. <br> **Datatype:** Dictionary.
| `train_cycles` | Training time steps will be set based on the `train_cycles * number of training data points. <br> **Datatype:** Integer.
| `max_trade_duration_candles`| Guides the agent training to keep trades below desired length. Example usage shown in `prediction_models/ReinforcementLearner.py` within the customizable `calculate_reward()` function. <br> **Datatype:** int.
| `model_type` | Model string from stable_baselines3 or SBcontrib. Available strings include: `'TRPO', 'ARS', 'RecurrentPPO', 'MaskablePPO', 'PPO', 'A2C', 'DQN'`. User should ensure that `model_training_parameters` match those available to the corresponding stable_baselines3 model by visiting their documentaiton. [PPO doc](https://stable-baselines3.readthedocs.io/en/master/modules/ppo.html) (external website) <br> **Datatype:** string.
| `model_type` | Model string from stable_baselines3 or SBcontrib. Available strings include: `'TRPO', 'ARS', 'RecurrentPPO', 'MaskablePPO', 'PPO', 'A2C', 'DQN'`. User should ensure that `model_training_parameters` match those available to the corresponding stable_baselines3 model by visiting their documentation. [PPO doc](https://stable-baselines3.readthedocs.io/en/master/modules/ppo.html) (external website) <br> **Datatype:** string.
| `policy_type` | One of the available policy types from stable_baselines3 <br> **Datatype:** string.
| `max_training_drawdown_pct` | The maximum drawdown that the agent is allowed to experience during training. <br> **Datatype:** float. <br> Default: 0.8
| `cpu_count` | Number of threads/cpus to dedicate to the Reinforcement Learning training process (depending on if `ReinforcementLearning_multiproc` is selected or not). Recommended to leave this untouched, by default, this value is set to the total number of physical cores minus 1. <br> **Datatype:** int.

2
docs/freqai-reinforcement-learning.md
View File

@@ -142,7 +142,7 @@ Parameter details can be found [here](freqai-parameter-table.md), but in general
As you begin to modify the strategy and the prediction model, you will quickly realize some important differences between the Reinforcement Learner and the Regressors/Classifiers. Firstly, the strategy does not set a target value (no labels!). Instead, you set the `calculate_reward()` function inside the `MyRLEnv` class (see below). A default `calculate_reward()` is provided inside `prediction_models/ReinforcementLearner.py` to demonstrate the necessary building blocks for creating rewards, but this is *not* designed for production. Users *must* create their own custom reinforcement learning model class or use a pre-built one from outside the Freqtrade source code and save it to `user_data/freqaimodels`. It is inside the `calculate_reward()` where creative theories about the market can be expressed. For example, you can reward your agent when it makes a winning trade, and penalize the agent when it makes a losing trade. Or perhaps, you wish to reward the agent for entering trades, and penalize the agent for sitting in trades too long. Below we show examples of how these rewards are all calculated:
!!! note "Hint"
The best reward functions are ones that are continuously differentiable, and well scaled. In other words, adding a single large negative penalty to a rare event is not a good idea, and the neural net will not be able to learn that function. Instead, it is better to add a small negative penalty to a common event. This will help the agent learn faster. Not only this, but you can help improve the continuity of your rewards/penalties by having them scale with severity according to some linear/exponential functions. In other words, you'd slowly scale the penalty as the duration of the trade increases. This is better than a single large penalty occuring at a single point in time.
The best reward functions are ones that are continuously differentiable, and well scaled. In other words, adding a single large negative penalty to a rare event is not a good idea, and the neural net will not be able to learn that function. Instead, it is better to add a small negative penalty to a common event. This will help the agent learn faster. Not only this, but you can help improve the continuity of your rewards/penalties by having them scale with severity according to some linear/exponential functions. In other words, you'd slowly scale the penalty as the duration of the trade increases. This is better than a single large penalty occurring at a single point in time.
```python
from freqtrade.freqai.prediction_models.ReinforcementLearner import ReinforcementLearner

4
docs/requirements-docs.txt
View File

@@ -1,6 +1,6 @@
markdown==3.5.2
mkdocs==1.5.3
mkdocs-material==9.5.12
mkdocs-material==9.5.13
mdx_truly_sane_lists==1.3
pymdown-extensions==10.7
pymdown-extensions==10.7.1
jinja2==3.1.3

129
docs/strategy-advanced.md
View File

@@ -11,34 +11,129 @@ The call sequence of the methods described here is covered under [bot execution
!!! Tip
Start off with a strategy template containing all available callback methods by running `freqtrade new-strategy --strategy MyAwesomeStrategy --template advanced`
## Storing information
## Storing information (Persistent)
Storing information can be accomplished by creating a new dictionary within the strategy class.
Freqtrade allows storing/retrieving user custom information associated with a specific trade in the database.
The name of the variable can be chosen at will, but should be prefixed with `custom_` to avoid naming collisions with predefined strategy variables.
Using a trade object, information can be stored using `trade.set_custom_data(key='my_key', value=my_value)` and retrieved using `trade.get_custom_data(key='my_key')`. Each data entry is associated with a trade and a user supplied key (of type `string`). This means that this can only be used in callbacks that also provide a trade object.
For the data to be able to be stored within the database, freqtrade must serialized the data. This is done by converting the data to a JSON formatted string.
Freqtrade will attempt to reverse this action on retrieval, so from a strategy perspective, this should not be relevant.
```python
from freqtrade.persistence import Trade
from datetime import timedelta
class AwesomeStrategy(IStrategy):
# Create custom dictionary
custom_info = {}
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
# Check if the entry already exists
if not metadata["pair"] in self.custom_info:
# Create empty entry for this pair
self.custom_info[metadata["pair"]] = {}
def bot_loop_start(self, **kwargs) -> None:
for trade in Trade.get_open_order_trades():
fills = trade.select_filled_orders(trade.entry_side)
if trade.pair == 'ETH/USDT':
trade_entry_type = trade.get_custom_data(key='entry_type')
if trade_entry_type is None:
trade_entry_type = 'breakout' if 'entry_1' in trade.enter_tag else 'dip'
elif fills > 1:
trade_entry_type = 'buy_up'
trade.set_custom_data(key='entry_type', value=trade_entry_type)
return super().bot_loop_start(**kwargs)
if "crosstime" in self.custom_info[metadata["pair"]]:
self.custom_info[metadata["pair"]]["crosstime"] += 1
else:
self.custom_info[metadata["pair"]]["crosstime"] = 1
def adjust_entry_price(self, trade: Trade, order: Optional[Order], pair: str,
current_time: datetime, proposed_rate: float, current_order_rate: float,
entry_tag: Optional[str], side: str, **kwargs) -> float:
# Limit orders to use and follow SMA200 as price target for the first 10 minutes since entry trigger for BTC/USDT pair.
if (
pair == 'BTC/USDT'
and entry_tag == 'long_sma200'
and side == 'long'
and (current_time - timedelta(minutes=10)) > trade.open_date_utc
and order.filled == 0.0
):
dataframe, _ = self.dp.get_analyzed_dataframe(pair=pair, timeframe=self.timeframe)
current_candle = dataframe.iloc[-1].squeeze()
# store information about entry adjustment
existing_count = trade.get_custom_data('num_entry_adjustments', default=0)
if not existing_count:
existing_count = 1
else:
existing_count += 1
trade.set_custom_data(key='num_entry_adjustments', value=existing_count)
# adjust order price
return current_candle['sma_200']
# default: maintain existing order
return current_order_rate
def custom_exit(self, pair: str, trade: Trade, current_time: datetime, current_rate: float, current_profit: float, **kwargs):
entry_adjustment_count = trade.get_custom_data(key='num_entry_adjustments')
trade_entry_type = trade.get_custom_data(key='entry_type')
if entry_adjustment_count is None:
if current_profit > 0.01 and (current_time - timedelta(minutes=100) > trade.open_date_utc):
return True, 'exit_1'
else
if entry_adjustment_count > 0 and if current_profit > 0.05:
return True, 'exit_2'
if trade_entry_type == 'breakout' and current_profit > 0.1:
return True, 'exit_3
return False, None
```
!!! Warning
The data is not persisted after a bot-restart (or config-reload). Also, the amount of data should be kept smallish (no DataFrames and such), otherwise the bot will start to consume a lot of memory and eventually run out of memory and crash.
The above is a simple example - there are simpler ways to retrieve trade data like entry-adjustments.
!!! Note
If the data is pair-specific, make sure to use pair as one of the keys in the dictionary.
It is recommended that simple data types are used `[bool, int, float, str]` to ensure no issues when serializing the data that needs to be stored.
Storing big junks of data may lead to unintended side-effects, like a database becoming big (and as a consequence, also slow).
!!! Warning "Non-serializable data"
If supplied data cannot be serialized a warning is logged and the entry for the specified `key` will contain `None` as data.
??? Note "All attributes"
custom-data has the following accessors through the Trade object (assumed as `trade` below):
* `trade.get_custom_data(key='something', default=0)` - Returns the actual value given in the type provided.
* `trade.get_custom_data_entry(key='something')` - Returns the entry - including metadata. The value is accessible via `.value` property.
* `trade.set_custom_data(key='something', value={'some': 'value'})` - set or update the corresponding key for this trade. Value must be serializable - and we recommend to keep the stored data relatively small.
"value" can be any type (both in setting and receiving) - but must be json serializable.
## Storing information (Non-Persistent)
!!! Warning "Deprecated"
This method of storing information is deprecated and we do advise against using non-persistent storage.
Please use [Persistent Storage](#storing-information-persistent) instead.
It's content has therefore been collapsed.
??? Abstract "Storing information"
Storing information can be accomplished by creating a new dictionary within the strategy class.
The name of the variable can be chosen at will, but should be prefixed with `custom_` to avoid naming collisions with predefined strategy variables.
```python
class AwesomeStrategy(IStrategy):
# Create custom dictionary
custom_info = {}
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
# Check if the entry already exists
if not metadata["pair"] in self.custom_info:
# Create empty entry for this pair
self.custom_info[metadata["pair"]] = {}
if "crosstime" in self.custom_info[metadata["pair"]]:
self.custom_info[metadata["pair"]]["crosstime"] += 1
else:
self.custom_info[metadata["pair"]]["crosstime"] = 1
```
!!! Warning
The data is not persisted after a bot-restart (or config-reload). Also, the amount of data should be kept smallish (no DataFrames and such), otherwise the bot will start to consume a lot of memory and eventually run out of memory and crash.
!!! Note
If the data is pair-specific, make sure to use pair as one of the keys in the dictionary.
## Dataframe access

2
docs/strategy_analysis_example.md
View File

@@ -19,7 +19,7 @@ from pathlib import Path
project_root = "somedir/freqtrade"
i=0
try:
os.chdirdir(project_root)
os.chdir(project_root)
assert Path('LICENSE').is_file()
except:
while i<4 and (not Path('LICENSE').is_file()):

3
docs/telegram-usage.md
View File

@@ -59,7 +59,7 @@ For the Freqtrade configuration, you can then use the the full value (including
"chat_id": "-1001332619709"
```
!!! Warning "Using telegram groups"
When using telegram groups, you're giving every member of the telegram group access to your freqtrade bot and to all commands possible via telegram. Please make sure that you can trust everyone in the telegram group to avoid unpleasent surprises.
When using telegram groups, you're giving every member of the telegram group access to your freqtrade bot and to all commands possible via telegram. Please make sure that you can trust everyone in the telegram group to avoid unpleasant surprises.
## Control telegram noise
@@ -181,6 +181,7 @@ official commands. You can ask at any moment for help with `/help`.
| `/locks` | Show currently locked pairs.
| `/unlock <pair or lock_id>` | Remove the lock for this pair (or for this lock id).
| `/marketdir [long | short | even | none]` | Updates the user managed variable that represents the current market direction. If no direction is provided, the currently set direction will be displayed.
| `/list_custom_data <trade_id> [key]` | List custom_data for Trade ID & Key combination. If no Key is supplied it will list all key-value pairs found for that Trade ID.
| **Modify Trade states** |
| `/forceexit <trade_id> | /fx <tradeid>` | Instantly exits the given trade (Ignoring `minimum_roi`).
| `/forceexit all | /fx all` | Instantly exits all open trades (Ignoring `minimum_roi`).

2
docs/webhook-config.md
View File

@@ -65,7 +65,7 @@ You can set the POST body format to Form-Encoded (default), JSON-Encoded, or raw
The result would be a POST request with e.g. `{"text":"Status: running"}` body and `Content-Type: application/json` header which results `Status: running` message in the Mattermost channel.
When using the Form-Encoded or JSON-Encoded configuration you can configure any number of payload values, and both the key and value will be ouput in the POST request. However, when using the raw data format you can only configure one value and it **must** be named `"data"`. In this instance the data key will not be output in the POST request, only the value. For example:
When using the Form-Encoded or JSON-Encoded configuration you can configure any number of payload values, and both the key and value will be output in the POST request. However, when using the raw data format you can only configure one value and it **must** be named `"data"`. In this instance the data key will not be output in the POST request, only the value. For example:
```json
"webhook": {

12
freqtrade/exchange/exchange.py
View File

@@ -680,7 +680,7 @@ class Exchange:
candle_limit = self.ohlcv_candle_limit(
timeframe, self._config['candle_type_def'],
int(date_minus_candles(timeframe, startup_candles).timestamp() * 1000)
dt_ts(date_minus_candles(timeframe, startup_candles))
if timeframe else None)
# Require one more candle - to account for the still open candle.
candle_count = startup_candles + 1
@@ -2067,7 +2067,11 @@ class Exchange:
if (not since_ms and (self._ft_has["ohlcv_require_since"] or not_all_data)):
# Multiple calls for one pair - to get more history
since_ms = self.needed_candle_ms(timeframe, candle_type)
one_call = timeframe_to_msecs(timeframe) * self.ohlcv_candle_limit(
timeframe, candle_type, since_ms)
move_to = one_call * self.required_candle_call_count
now = timeframe_to_next_date(timeframe)
since_ms = dt_ts(now - timedelta(seconds=move_to // 1000))
if since_ms:
return self._async_get_historic_ohlcv(
@@ -2669,7 +2673,7 @@ class Exchange:
)
if type(since) is datetime:
since = int(since.timestamp()) * 1000 # * 1000 for ms
since = dt_ts(since)
try:
funding_history = self._api.fetch_funding_history(
@@ -2999,7 +3003,7 @@ class Exchange:
if not close_date:
close_date = datetime.now(timezone.utc)
since_ms = int(timeframe_to_prev_date(timeframe, open_date).timestamp()) * 1000
since_ms = dt_ts(timeframe_to_prev_date(timeframe, open_date))
mark_comb: PairWithTimeframe = (pair, timeframe, mark_price_type)
funding_comb: PairWithTimeframe = (pair, timeframe_ff, CandleType.FUNDING_RATE)

2
freqtrade/freqtradebot.py
View File

@@ -962,7 +962,7 @@ class FreqtradeBot(LoggingMixin):
# edge-case for now.
min_stake_amount = self.exchange.get_min_pair_stake_amount(
pair, enter_limit_requested,
self.strategy.stoploss if not mode != 'pos_adjust' else 0.0,
self.strategy.stoploss if not mode == 'pos_adjust' else 0.0,
leverage)
max_stake_amount = self.exchange.get_max_pair_stake_amount(
pair, enter_limit_requested, leverage)

5
freqtrade/optimize/backtesting.py
View File

@@ -33,8 +33,8 @@ from freqtrade.optimize.optimize_reports import (generate_backtest_stats, genera
show_backtest_results,
store_backtest_analysis_results,
store_backtest_stats)
from freqtrade.persistence import (LocalTrade, Order, PairLocks, Trade, disable_database_use,
enable_database_use)
from freqtrade.persistence import (CustomDataWrapper, LocalTrade, Order, PairLocks, Trade,
disable_database_use, enable_database_use)
from freqtrade.plugins.pairlistmanager import PairListManager
from freqtrade.plugins.protectionmanager import ProtectionManager
from freqtrade.resolvers import ExchangeResolver, StrategyResolver
@@ -337,6 +337,7 @@ class Backtesting:
self.disable_database_use()
PairLocks.reset_locks()
Trade.reset_trades()
CustomDataWrapper.reset_custom_data()
self.rejected_trades = 0
self.timedout_entry_orders = 0
self.timedout_exit_orders = 0

1
freqtrade/persistence/__init__.py
View File

@@ -1,5 +1,6 @@
# flake8: noqa: F401
from freqtrade.persistence.custom_data import CustomDataWrapper
from freqtrade.persistence.key_value_store import KeyStoreKeys, KeyValueStore
from freqtrade.persistence.models import init_db
from freqtrade.persistence.pairlock_middleware import PairLocks

174
freqtrade/persistence/custom_data.py Normal file
View File

@@ -0,0 +1,174 @@
import json
import logging
from datetime import datetime
from typing import Any, ClassVar, List, Optional, Sequence
from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, select
from sqlalchemy.orm import Mapped, mapped_column, relationship
from freqtrade.constants import DATETIME_PRINT_FORMAT
from freqtrade.persistence.base import ModelBase, SessionType
from freqtrade.util import dt_now
logger = logging.getLogger(__name__)
class _CustomData(ModelBase):
"""
CustomData database model
Keeps records of metadata as key/value store
for trades or global persistant values
One to many relationship with Trades:
- One trade can have many metadata entries
- One metadata entry can only be associated with one Trade
"""
__tablename__ = 'trade_custom_data'
__allow_unmapped__ = True
session: ClassVar[SessionType]
# Uniqueness should be ensured over pair, order_id
# its likely that order_id is unique per Pair on some exchanges.
__table_args__ = (UniqueConstraint('ft_trade_id', 'cd_key', name="_trade_id_cd_key"),)
id = mapped_column(Integer, primary_key=True)
ft_trade_id = mapped_column(Integer, ForeignKey('trades.id'), index=True)
trade = relationship("Trade", back_populates="custom_data")
cd_key: Mapped[str] = mapped_column(String(255), nullable=False)
cd_type: Mapped[str] = mapped_column(String(25), nullable=False)
cd_value: Mapped[str] = mapped_column(Text, nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime, nullable=False, default=dt_now)
updated_at: Mapped[Optional[datetime]] = mapped_column(DateTime, nullable=True)
# Empty container value - not persisted, but filled with cd_value on query
value: Any = None
def __repr__(self):
create_time = (self.created_at.strftime(DATETIME_PRINT_FORMAT)
if self.created_at is not None else None)
update_time = (self.updated_at.strftime(DATETIME_PRINT_FORMAT)
if self.updated_at is not None else None)
return (f'CustomData(id={self.id}, key={self.cd_key}, type={self.cd_type}, ' +
f'value={self.cd_value}, trade_id={self.ft_trade_id}, created={create_time}, ' +
f'updated={update_time})')
@classmethod
def query_cd(cls, key: Optional[str] = None,
trade_id: Optional[int] = None) -> Sequence['_CustomData']:
"""
Get all CustomData, if trade_id is not specified
return will be for generic values not tied to a trade
:param trade_id: id of the Trade
"""
filters = []
if trade_id is not None:
filters.append(_CustomData.ft_trade_id == trade_id)
if key is not None:
filters.append(_CustomData.cd_key.ilike(key))
return _CustomData.session.scalars(select(_CustomData).filter(*filters)).all()
class CustomDataWrapper:
"""
CustomData middleware class
Abstracts the database layer away so it becomes optional - which will be necessary to support
backtesting and hyperopt in the future.
"""
use_db = True
custom_data: List[_CustomData] = []
unserialized_types = ['bool', 'float', 'int', 'str']
@staticmethod
def _convert_custom_data(data: _CustomData) -> _CustomData:
if data.cd_type in CustomDataWrapper.unserialized_types:
data.value = data.cd_value
if data.cd_type == 'bool':
data.value = data.cd_value.lower() == 'true'
elif data.cd_type == 'int':
data.value = int(data.cd_value)
elif data.cd_type == 'float':
data.value = float(data.cd_value)
else:
data.value = json.loads(data.cd_value)
return data
@staticmethod
def reset_custom_data() -> None:
"""
Resets all key-value pairs. Only active for backtesting mode.
"""
if not CustomDataWrapper.use_db:
CustomDataWrapper.custom_data = []
@staticmethod
def delete_custom_data(trade_id: int) -> None:
_CustomData.session.query(_CustomData).filter(_CustomData.ft_trade_id == trade_id).delete()
_CustomData.session.commit()
@staticmethod
def get_custom_data(*, trade_id: int, key: Optional[str] = None) -> List[_CustomData]:
if CustomDataWrapper.use_db:
filters = [
_CustomData.ft_trade_id == trade_id,
]
if key is not None:
filters.append(_CustomData.cd_key.ilike(key))
filtered_custom_data = _CustomData.session.scalars(select(_CustomData).filter(
*filters)).all()
else:
filtered_custom_data = [
data_entry for data_entry in CustomDataWrapper.custom_data
if (data_entry.ft_trade_id == trade_id)
]
if key is not None:
filtered_custom_data = [
data_entry for data_entry in filtered_custom_data
if (data_entry.cd_key.casefold() == key.casefold())
]
return [CustomDataWrapper._convert_custom_data(d) for d in filtered_custom_data]
@staticmethod
def set_custom_data(trade_id: int, key: str, value: Any) -> None:
value_type = type(value).__name__
if value_type not in CustomDataWrapper.unserialized_types:
try:
value_db = json.dumps(value)
except TypeError as e:
logger.warning(f"could not serialize {key} value due to {e}")
return
else:
value_db = str(value)
if trade_id is None:
trade_id = 0
custom_data = CustomDataWrapper.get_custom_data(trade_id=trade_id, key=key)
if custom_data:
data_entry = custom_data[0]
data_entry.cd_value = value_db
data_entry.updated_at = dt_now()
else:
data_entry = _CustomData(
ft_trade_id=trade_id,
cd_key=key,
cd_type=value_type,
cd_value=value_db,
created_at=dt_now(),
)
data_entry.value = value
if CustomDataWrapper.use_db and value_db is not None:
_CustomData.session.add(data_entry)
_CustomData.session.commit()
else:
if not custom_data:
CustomDataWrapper.custom_data.append(data_entry)
# Existing data will have updated interactively.

3
freqtrade/persistence/models.py
View File

@@ -13,6 +13,7 @@ from sqlalchemy.pool import StaticPool
from freqtrade.exceptions import OperationalException
from freqtrade.persistence.base import ModelBase
from freqtrade.persistence.custom_data import _CustomData
from freqtrade.persistence.key_value_store import _KeyValueStoreModel
from freqtrade.persistence.migrations import check_migrate
from freqtrade.persistence.pairlock import PairLock
@@ -78,6 +79,8 @@ def init_db(db_url: str) -> None:
Order.session = Trade.session
PairLock.session = Trade.session
_KeyValueStoreModel.session = Trade.session
_CustomData.session = scoped_session(sessionmaker(bind=engine, autoflush=True),
scopefunc=get_request_or_thread_id)
previous_tables = inspect(engine).get_table_names()
ModelBase.metadata.create_all(engine)

40
freqtrade/persistence/trade_model.py
View File

@@ -23,6 +23,7 @@ from freqtrade.exchange import (ROUND_DOWN, ROUND_UP, amount_to_contract_precisi
from freqtrade.leverage import interest
from freqtrade.misc import safe_value_fallback
from freqtrade.persistence.base import ModelBase, SessionType
from freqtrade.persistence.custom_data import CustomDataWrapper, _CustomData
from freqtrade.util import FtPrecise, dt_from_ts, dt_now, dt_ts, dt_ts_none
@@ -1214,6 +1215,40 @@ class LocalTrade:
or (o.ft_is_open is True and o.status is not None)
]
def set_custom_data(self, key: str, value: Any) -> None:
"""
Set custom data for this trade
:param key: key of the custom data
:param value: value of the custom data (must be JSON serializable)
"""
CustomDataWrapper.set_custom_data(trade_id=self.id, key=key, value=value)
def get_custom_data(self, key: str, default: Any = None) -> Any:
"""
Get custom data for this trade
:param key: key of the custom data
"""
data = CustomDataWrapper.get_custom_data(trade_id=self.id, key=key)
if data:
return data[0].value
return default
def get_custom_data_entry(self, key: str) -> Optional[_CustomData]:
"""
Get custom data for this trade
:param key: key of the custom data
"""
data = CustomDataWrapper.get_custom_data(trade_id=self.id, key=key)
if data:
return data[0]
return None
def get_all_custom_data(self) -> List[_CustomData]:
"""
Get all custom data for this trade
"""
return CustomDataWrapper.get_custom_data(trade_id=self.id)
@property
def nr_of_successful_entries(self) -> int:
"""
@@ -1469,6 +1504,9 @@ class Trade(ModelBase, LocalTrade):
orders: Mapped[List[Order]] = relationship(
"Order", order_by="Order.id", cascade="all, delete-orphan", lazy="selectin",
innerjoin=True) # type: ignore
custom_data: Mapped[List[_CustomData]] = relationship(
"_CustomData", cascade="all, delete-orphan",
lazy="raise")
exchange: Mapped[str] = mapped_column(String(25), nullable=False) # type: ignore
pair: Mapped[str] = mapped_column(String(25), nullable=False, index=True) # type: ignore
@@ -1572,6 +1610,8 @@ class Trade(ModelBase, LocalTrade):
for order in self.orders:
Order.session.delete(order)
CustomDataWrapper.delete_custom_data(trade_id=self.id)
Trade.session.delete(self)
Trade.commit()

3
freqtrade/persistence/usedb_context.py
View File

@@ -1,4 +1,5 @@
from freqtrade.persistence.custom_data import CustomDataWrapper
from freqtrade.persistence.pairlock_middleware import PairLocks
from freqtrade.persistence.trade_model import Trade
@@ -11,6 +12,7 @@ def disable_database_use(timeframe: str) -> None:
PairLocks.use_db = False
PairLocks.timeframe = timeframe
Trade.use_db = False
CustomDataWrapper.use_db = False
def enable_database_use() -> None:
@@ -20,6 +22,7 @@ def enable_database_use() -> None:
PairLocks.use_db = True
PairLocks.timeframe = ''
Trade.use_db = True
CustomDataWrapper.use_db = True
class FtNoDBContext:

4
freqtrade/rpc/api_server/api_schemas.py
View File

@@ -559,3 +559,7 @@ class SysInfo(BaseModel):
class Health(BaseModel):
last_process: Optional[datetime] = None
last_process_ts: Optional[int] = None
bot_start: Optional[datetime] = None
bot_start_ts: Optional[int] = None
bot_startup: Optional[datetime] = None
bot_startup_ts: Optional[int] = None

76
freqtrade/rpc/rpc.py
View File

@@ -291,6 +291,10 @@ class RPC:
profit_str += f" ({fiat_profit:.2f})"
fiat_profit_sum = fiat_profit if isnan(fiat_profit_sum) \
else fiat_profit_sum + fiat_profit
else:
profit_str += f" ({trade_profit:.2f})"
fiat_profit_sum = trade_profit if isnan(fiat_profit_sum) \
else fiat_profit_sum + trade_profit
active_attempt_side_symbols = [
'*' if (oo and oo.ft_order_side == trade.entry_side) else '**'
@@ -317,6 +321,8 @@ class RPC:
profitcol = "Profit"
if self._fiat_converter:
profitcol += " (" + fiat_display_currency + ")"
else:
profitcol += " (" + stake_currency + ")"
columns = [
'ID L/S' if nonspot else 'ID',
@@ -927,6 +933,7 @@ class RPC:
is_short=is_short,
enter_tag=enter_tag,
leverage_=leverage,
mode='pos_adjust' if trade else 'initial'
):
Trade.commit()
trade = Trade.get_trades([Trade.is_open.is_(True), Trade.pair == pair]).first()
@@ -999,6 +1006,32 @@ class RPC:
'cancel_order_count': c_count,
}
def _rpc_list_custom_data(self, trade_id: int, key: Optional[str]) -> List[Dict[str, Any]]:
# Query for trade
trade = Trade.get_trades(trade_filter=[Trade.id == trade_id]).first()
if trade is None:
return []
# Query custom_data
custom_data = []
if key:
data = trade.get_custom_data(key=key)
if data:
custom_data = [data]
else:
custom_data = trade.get_all_custom_data()
return [
{
'id': data_entry.id,
'ft_trade_id': data_entry.ft_trade_id,
'cd_key': data_entry.cd_key,
'cd_type': data_entry.cd_type,
'cd_value': data_entry.cd_value,
'created_at': data_entry.created_at,
'updated_at': data_entry.updated_at
}
for data_entry in custom_data
]
def _rpc_performance(self) -> List[Dict[str, Any]]:
"""
Handler for performance.
@@ -1333,19 +1366,40 @@ class RPC:
def health(self) -> Dict[str, Optional[Union[str, int]]]:
last_p = self._freqtrade.last_process
if last_p is None:
return {
"last_process": None,
"last_process_loc": None,
"last_process_ts": None,
}
return {
"last_process": str(last_p),
"last_process_loc": format_date(last_p.astimezone(tzlocal())),
"last_process_ts": int(last_p.timestamp()),
res: Dict[str, Union[None, str, int]] = {
"last_process": None,
"last_process_loc": None,
"last_process_ts": None,
"bot_start": None,
"bot_start_loc": None,
"bot_start_ts": None,
"bot_startup": None,
"bot_startup_loc": None,
"bot_startup_ts": None,
}
if last_p is not None:
res.update({
"last_process": str(last_p),
"last_process_loc": format_date(last_p.astimezone(tzlocal())),
"last_process_ts": int(last_p.timestamp()),
})
if (bot_start := KeyValueStore.get_datetime_value(KeyStoreKeys.BOT_START_TIME)):
res.update({
"bot_start": str(bot_start),
"bot_start_loc": format_date(bot_start.astimezone(tzlocal())),
"bot_start_ts": int(bot_start.timestamp()),
})
if (bot_startup := KeyValueStore.get_datetime_value(KeyStoreKeys.STARTUP_TIME)):
res.update({
"bot_startup": str(bot_startup),
"bot_startup_loc": format_date(bot_startup.astimezone(tzlocal())),
"bot_startup_ts": int(bot_startup.timestamp()),
})
return res
def _update_market_direction(self, direction: MarketDirection) -> None:
self._freqtrade.strategy.market_direction = direction

58
freqtrade/rpc/telegram.py
View File

@@ -33,7 +33,7 @@ from freqtrade.misc import chunks, plural
from freqtrade.persistence import Trade
from freqtrade.rpc import RPC, RPCException, RPCHandler
from freqtrade.rpc.rpc_types import RPCEntryMsg, RPCExitMsg, RPCOrderMsg, RPCSendMsg
from freqtrade.util import dt_humanize, fmt_coin, round_value
from freqtrade.util import dt_humanize, fmt_coin, format_date, round_value
MAX_MESSAGE_LENGTH = MessageLimit.MAX_TEXT_LENGTH
@@ -243,6 +243,7 @@ class Telegram(RPCHandler):
CommandHandler('version', self._version),
CommandHandler('marketdir', self._changemarketdir),
CommandHandler('order', self._order),
CommandHandler('list_custom_data', self._list_custom_data),
]
callbacks = [
CallbackQueryHandler(self._status_table, pattern='update_status_table'),
@@ -1667,6 +1668,8 @@ class Telegram(RPCHandler):
"*/marketdir [long | short | even | none]:* `Updates the user managed variable "
"that represents the current market direction. If no direction is provided `"
"`the currently set market direction will be output.` \n"
"*/list_custom_data <trade_id> <key>:* `List custom_data for Trade ID & Key combo.`\n"
"`If no Key is supplied it will list all key-value pairs found for that Trade ID.`"
"_Statistics_\n"
"------------\n"
@@ -1689,7 +1692,7 @@ class Telegram(RPCHandler):
"*/stats:* `Shows Wins / losses by Sell reason as well as "
"Avg. holding durations for buys and sells.`\n"
"*/help:* `This help message`\n"
"*/version:* `Show version`"
"*/version:* `Show version`\n"
)
await self._send_msg(message, parse_mode=ParseMode.MARKDOWN)
@@ -1701,7 +1704,9 @@ class Telegram(RPCHandler):
Shows the last process timestamp
"""
health = self._rpc.health()
message = f"Last process: `{health['last_process_loc']}`"
message = f"Last process: `{health['last_process_loc']}`\n"
message += f"Initial bot start: `{health['bot_start_loc']}`\n"
message += f"Last bot restart: `{health['bot_startup_loc']}`"
await self._send_msg(message)
@authorized_only
@@ -1766,6 +1771,53 @@ class Telegram(RPCHandler):
f"*Current state:* `{val['state']}`"
)
@authorized_only
async def _list_custom_data(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /list_custom_data <id> <key>.
List custom_data for specified trade (and key if supplied).
:param bot: telegram bot
:param update: message update
:return: None
"""
try:
if not context.args or len(context.args) == 0:
raise RPCException("Trade-id not set.")
trade_id = int(context.args[0])
key = None if len(context.args) < 2 else str(context.args[1])
results = self._rpc._rpc_list_custom_data(trade_id, key)
messages = []
if len(results) > 0:
messages.append(
'Found custom-data entr' + ('ies: ' if len(results) > 1 else 'y: ')
)
for result in results:
lines = [
f"*Key:* `{result['cd_key']}`",
f"*ID:* `{result['id']}`",
f"*Trade ID:* `{result['ft_trade_id']}`",
f"*Type:* `{result['cd_type']}`",
f"*Value:* `{result['cd_value']}`",
f"*Create Date:* `{format_date(result['created_at'])}`",
f"*Update Date:* `{format_date(result['updated_at'])}`"
]
# Filter empty lines using list-comprehension
messages.append("\n".join([line for line in lines if line]))
for msg in messages:
if len(msg) > MAX_MESSAGE_LENGTH:
msg = "Message dropped because length exceeds "
msg += f"maximum allowed characters: {MAX_MESSAGE_LENGTH}"
logger.warning(msg)
await self._send_msg(msg)
else:
message = f"Didn't find any custom-data entries for Trade ID: `{trade_id}`"
message += f" and Key: `{key}`." if key is not None else ""
await self._send_msg(message)
except RPCException as e:
await self._send_msg(str(e))
async def _update_msg(self, query: CallbackQuery, msg: str, callback_path: str = "",
reload_able: bool = False, parse_mode: str = ParseMode.MARKDOWN) -> None:
if reload_able:

4
freqtrade/templates/strategy_analysis_example.ipynb
View File

@@ -35,7 +35,7 @@
"project_root = \"somedir/freqtrade\"\n",
"i=0\n",
"try:\n",
" os.chdirdir(project_root)\n",
" os.chdir(project_root)\n",
" assert Path('LICENSE').is_file()\n",
"except:\n",
" while i<4 and (not Path('LICENSE').is_file()):\n",
@@ -181,7 +181,7 @@
"\n",
"# if backtest_dir points to a directory, it'll automatically load the last backtest file.\n",
"backtest_dir = config[\"user_data_dir\"] / \"backtest_results\"\n",
"# backtest_dir can also point to a specific file \n",
"# backtest_dir can also point to a specific file\n",
"# backtest_dir = config[\"user_data_dir\"] / \"backtest_results/backtest-result-2020-07-01_20-04-22.json\""
]
},

12
requirements-dev.txt
View File

@@ -8,10 +8,10 @@
coveralls==3.3.1
ruff==0.3.0
mypy==1.8.0
mypy==1.9.0
pre-commit==3.6.2
pytest==8.1.0
pytest-asyncio==0.23.5
pytest==8.1.1
pytest-asyncio==0.23.5.post1
pytest-cov==4.1.0
pytest-mock==3.12.0
pytest-random-order==1.1.1
@@ -21,11 +21,11 @@ isort==5.13.2
time-machine==2.14.0
# Convert jupyter notebooks to markdown documents
nbconvert==7.16.1
nbconvert==7.16.2
# mypy types
types-cachetools==5.3.0.7
types-filelock==3.2.7
types-requests==2.31.0.20240218
types-requests==2.31.0.20240311
types-tabulate==0.9.0.20240106
types-python-dateutil==2.8.19.20240106
types-python-dateutil==2.8.19.20240311

2
requirements-freqai.txt
View File

@@ -5,7 +5,7 @@
# Required for freqai
scikit-learn==1.4.1.post1
joblib==1.3.2
catboost==1.2.2; 'arm' not in platform_machine and python_version < '3.12'
catboost==1.2.3; 'arm' not in platform_machine
lightgbm==4.3.0
xgboost==2.0.3
tensorboard==2.16.2

6
requirements.txt
View File

@@ -2,11 +2,11 @@ numpy==1.26.4
pandas==2.2.1
pandas-ta==0.3.14b
ccxt==4.2.58
ccxt==4.2.66
cryptography==42.0.5
aiohttp==3.9.3
SQLAlchemy==2.0.27
python-telegram-bot==20.8
python-telegram-bot==21.0.1
# can't be hard-pinned due to telegram-bot pinning httpx with ~
httpx>=0.24.1
arrow==1.3.0
@@ -60,4 +60,4 @@ websockets==12.0
janus==1.0.0
ast-comments==1.2.1
packaging==23.2
packaging==24.0

2
setup.sh
View File

@@ -161,7 +161,7 @@ function install_macos() {
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
fi
brew install gettext
brew install gettext libomp
#Gets number after decimal in python version
version=$(egrep -o 3.\[0-9\]+ <<< $PYTHON | sed 's/3.//g')

7
tests/freqai/conftest.py
View File

@@ -25,10 +25,15 @@ def is_mac() -> bool:
return "Darwin" in machine
def is_arm() -> bool:
machine = platform.machine()
return "arm" in machine or "aarch64" in machine
@pytest.fixture(autouse=True)
def patch_torch_initlogs(mocker) -> None:
if is_mac():
if is_mac() and not is_arm():
# Mock torch import completely
import sys
import types

14
tests/freqai/test_freqai_interface.py
View File

@@ -1,5 +1,4 @@
import logging
import platform
import shutil
from pathlib import Path
from unittest.mock import MagicMock
@@ -15,19 +14,14 @@ from freqtrade.optimize.backtesting import Backtesting
from freqtrade.persistence import Trade
from freqtrade.plugins.pairlistmanager import PairListManager
from tests.conftest import EXMS, create_mock_trades, get_patched_exchange, log_has_re
from tests.freqai.conftest import (get_patched_freqai_strategy, is_mac, is_py12, make_rl_config,
mock_pytorch_mlp_model_training_parameters)
def is_arm() -> bool:
machine = platform.machine()
return "arm" in machine or "aarch64" in machine
from tests.freqai.conftest import (get_patched_freqai_strategy, is_arm, is_mac, is_py12,
make_rl_config, mock_pytorch_mlp_model_training_parameters)
def can_run_model(model: str) -> None:
is_pytorch_model = 'Reinforcement' in model or 'PyTorch' in model
if is_py12() and ("Catboost" in model or is_pytorch_model):
if is_py12() and is_pytorch_model:
pytest.skip("Model not supported on python 3.12 yet.")
if is_arm() and "Catboost" in model:
@@ -243,7 +237,7 @@ def test_extract_data_and_train_model_Classifiers(mocker, freqai_conf, model):
def test_start_backtesting(mocker, freqai_conf, model, num_files, strat, caplog):
can_run_model(model)
test_tb = True
if is_mac():
if is_mac() and not is_arm():
test_tb = False
freqai_conf.get("freqai", {}).update({"save_backtest_models": True})

1
tests/persistence/test_persistence.py
View File

@@ -2099,6 +2099,7 @@ def test_Trade_object_idem():
'get_mix_tag_performance',
'get_trading_volume',
'validate_string_len',
'custom_data'
)
EXCLUDES2 = ('trades', 'trades_open', 'bt_trades_open_pp', 'bt_open_open_trade_count',
'total_profit', 'from_json',)

160
tests/persistence/test_trade_custom_data.py Normal file
View File

@@ -0,0 +1,160 @@
from copy import deepcopy
from unittest.mock import MagicMock
import pytest
from freqtrade.data.history.history_utils import get_timerange
from freqtrade.optimize.backtesting import Backtesting
from freqtrade.persistence import Trade, disable_database_use, enable_database_use
from freqtrade.persistence.custom_data import CustomDataWrapper
from tests.conftest import (EXMS, create_mock_trades_usdt, generate_test_data,
get_patched_freqtradebot, patch_exchange)
@pytest.mark.usefixtures("init_persistence")
@pytest.mark.parametrize("use_db", [True, False])
def test_trade_custom_data(fee, use_db):
if not use_db:
disable_database_use('5m')
Trade.reset_trades()
CustomDataWrapper.reset_custom_data()
create_mock_trades_usdt(fee, use_db=use_db)
trade1 = Trade.get_trades_proxy()[0]
if not use_db:
trade1.id = 1
assert trade1.get_all_custom_data() == []
trade1.set_custom_data('test_str', 'test_value')
trade1.set_custom_data('test_int', 1)
trade1.set_custom_data('test_float', 1.55)
trade1.set_custom_data('test_bool', True)
trade1.set_custom_data('test_dict', {'test': 'dict'})
assert len(trade1.get_all_custom_data()) == 5
assert trade1.get_custom_data('test_str') == 'test_value'
trade1.set_custom_data('test_str', 'test_value_updated')
assert trade1.get_custom_data('test_str') == 'test_value_updated'
assert trade1.get_custom_data('test_int') == 1
assert isinstance(trade1.get_custom_data('test_int'), int)
assert trade1.get_custom_data('test_float') == 1.55
assert isinstance(trade1.get_custom_data('test_float'), float)
assert trade1.get_custom_data('test_bool') is True
assert isinstance(trade1.get_custom_data('test_bool'), bool)
assert trade1.get_custom_data('test_dict') == {'test': 'dict'}
assert isinstance(trade1.get_custom_data('test_dict'), dict)
if not use_db:
enable_database_use()
def test_trade_custom_data_strategy_compat(mocker, default_conf_usdt, fee):
mocker.patch(f'{EXMS}.get_rate', return_value=0.50)
mocker.patch('freqtrade.freqtradebot.FreqtradeBot.get_real_amount', return_value=None)
default_conf_usdt["minimal_roi"] = {"0": 100}
freqtrade = get_patched_freqtradebot(mocker, default_conf_usdt)
create_mock_trades_usdt(fee)
trade1 = Trade.get_trades_proxy(pair='ADA/USDT')[0]
trade1.set_custom_data('test_str', 'test_value')
trade1.set_custom_data('test_int', 1)
def custom_exit(pair, trade, **kwargs):
if pair == 'ADA/USDT':
custom_val = trade.get_custom_data('test_str')
custom_val_i = trade.get_custom_data('test_int')
return f"{custom_val}_{custom_val_i}"
freqtrade.strategy.custom_exit = custom_exit
ff_spy = mocker.spy(freqtrade.strategy, 'custom_exit')
trades = Trade.get_open_trades()
freqtrade.exit_positions(trades)
Trade.commit()
trade_after = Trade.get_trades_proxy(pair='ADA/USDT')[0]
assert trade_after.get_custom_data('test_str') == 'test_value'
assert trade_after.get_custom_data('test_int') == 1
# 2 open pairs eligible for exit
assert ff_spy.call_count == 2
assert trade_after.exit_reason == 'test_value_1'
def test_trade_custom_data_strategy_backtest_compat(mocker, default_conf_usdt, fee):
mocker.patch(f'{EXMS}.get_fee', fee)
mocker.patch(f"{EXMS}.get_min_pair_stake_amount", return_value=10)
mocker.patch(f"{EXMS}.get_max_pair_stake_amount", return_value=float('inf'))
mocker.patch(f"{EXMS}.get_max_leverage", return_value=10)
mocker.patch(f"{EXMS}.get_maintenance_ratio_and_amt", return_value=(0.1, 0.1))
mocker.patch('freqtrade.optimize.backtesting.Backtesting._run_funding_fees')
patch_exchange(mocker)
default_conf_usdt.update({
"stake_amount": 100.0,
"max_open_trades": 2,
"dry_run_wallet": 1000.0,
"strategy": "StrategyTestV3",
"trading_mode": "futures",
"margin_mode": "isolated",
"stoploss": -2,
"minimal_roi": {"0": 100},
})
default_conf_usdt['pairlists'] = [{'method': 'StaticPairList', 'allow_inactive': True}]
backtesting = Backtesting(default_conf_usdt)
df = generate_test_data(default_conf_usdt['timeframe'], 100, '2022-01-01 00:00:00+00:00')
pair_exp = 'XRP/USDT:USDT'
def custom_exit(pair, trade, **kwargs):
custom_val = trade.get_custom_data('test_str')
custom_val_i = trade.get_custom_data('test_int', 0)
if pair == pair_exp:
trade.set_custom_data('test_str', 'test_value')
trade.set_custom_data('test_int', custom_val_i + 1)
if custom_val_i >= 2:
return f"{custom_val}_{custom_val_i}"
backtesting._set_strategy(backtesting.strategylist[0])
processed = backtesting.strategy.advise_all_indicators({
pair_exp: df,
'BTC/USDT:USDT': df,
})
def fun(dataframe, *args, **kwargs):
dataframe.loc[dataframe.index == 50, 'enter_long'] = 1
return dataframe
backtesting.strategy.advise_entry = fun
backtesting.strategy.leverage = MagicMock(return_value=1)
backtesting.strategy.custom_exit = custom_exit
ff_spy = mocker.spy(backtesting.strategy, 'custom_exit')
min_date, max_date = get_timerange(processed)
result = backtesting.backtest(
processed=deepcopy(processed),
start_date=min_date,
end_date=max_date,
)
results = result['results']
assert not results.empty
assert len(results) == 2
assert results['pair'][0] == pair_exp
assert results['pair'][1] == 'BTC/USDT:USDT'
assert results['exit_reason'][0] == 'test_value_2'
assert results['exit_reason'][1] == 'exit_signal'
assert ff_spy.call_count == 7
Backtesting.cleanup()

10
tests/rpc/test_rpc.py
View File

@@ -10,6 +10,7 @@ from freqtrade.edge import PairInfo
from freqtrade.enums import SignalDirection, State, TradingMode
from freqtrade.exceptions import ExchangeError, InvalidOrderException, TemporaryError
from freqtrade.persistence import Order, Trade
from freqtrade.persistence.key_value_store import set_startup_time
from freqtrade.persistence.pairlock_middleware import PairLocks
from freqtrade.rpc import RPC, RPCException
from freqtrade.rpc.fiat_convert import CryptoToFiatConverter
@@ -223,8 +224,8 @@ def test_rpc_status_table(default_conf, ticker, fee, mocker) -> None:
assert "Pair" in headers
assert 'instantly' == result[0][2]
assert 'ETH/BTC' in result[0][1]
assert '0.00' == result[0][3]
assert isnan(fiat_profit_sum)
assert '0.00 (0.00)' == result[0][3]
assert '0.00' == f'{fiat_profit_sum:.2f}'
mocker.patch(f'{EXMS}._dry_is_price_crossed', return_value=True)
freqtradebot.process()
@@ -234,8 +235,8 @@ def test_rpc_status_table(default_conf, ticker, fee, mocker) -> None:
assert "Pair" in headers
assert 'instantly' == result[0][2]
assert 'ETH/BTC' in result[0][1]
assert '-0.41%' == result[0][3]
assert isnan(fiat_profit_sum)
assert '-0.41% (-0.00)' == result[0][3]
assert '-0.00' == f'{fiat_profit_sum:.2f}'
# Test with fiat convert
rpc._fiat_converter = CryptoToFiatConverter()
@@ -1298,6 +1299,7 @@ def test_rpc_health(mocker, default_conf) -> None:
mocker.patch('freqtrade.rpc.telegram.Telegram', MagicMock())
freqtradebot = get_patched_freqtradebot(mocker, default_conf)
set_startup_time()
rpc = RPC(freqtradebot)
result = rpc.health()
assert result['last_process'] is None

48
tests/rpc/test_rpc_telegram.py
View File

@@ -150,7 +150,7 @@ def test_telegram_init(default_conf, mocker, caplog) -> None:
"['stopbuy', 'stopentry'], ['whitelist'], ['blacklist'], "
"['bl_delete', 'blacklist_delete'], "
"['logs'], ['edge'], ['health'], ['help'], ['version'], ['marketdir'], "
"['order']]")
"['order'], ['list_custom_data']]")
assert log_has(message_str, caplog)
@@ -2657,3 +2657,49 @@ async def test_change_market_direction(default_conf, mocker, update) -> None:
context.args = ["invalid"]
await telegram._changemarketdir(update, context)
assert telegram._rpc._freqtrade.strategy.market_direction == MarketDirection.LONG
async def test_telegram_list_custom_data(default_conf_usdt, update, ticker, fee, mocker) -> None:
mocker.patch.multiple(
EXMS,
fetch_ticker=ticker,
get_fee=fee,
)
telegram, _freqtradebot, msg_mock = get_telegram_testobject(mocker, default_conf_usdt)
# Create some test data
create_mock_trades_usdt(fee)
# No trade id
context = MagicMock()
await telegram._list_custom_data(update=update, context=context)
assert msg_mock.call_count == 1
assert 'Trade-id not set.' in msg_mock.call_args_list[0][0][0]
msg_mock.reset_mock()
#
context.args = ['1']
await telegram._list_custom_data(update=update, context=context)
assert msg_mock.call_count == 1
assert (
"Didn't find any custom-data entries for Trade ID: `1`" in msg_mock.call_args_list[0][0][0]
)
msg_mock.reset_mock()
# Add some custom data
trade1 = Trade.get_trades_proxy()[0]
trade1.set_custom_data('test_int', 1)
trade1.set_custom_data('test_dict', {'test': 'dict'})
Trade.commit()
context.args = [f"{trade1.id}"]
await telegram._list_custom_data(update=update, context=context)
assert msg_mock.call_count == 3
assert "Found custom-data entries: " in msg_mock.call_args_list[0][0][0]
assert (
"*Key:* `test_int`\n*ID:* `1`\n*Trade ID:* `1`\n*Type:* `int`\n"
"*Value:* `1`\n*Create Date:*") in msg_mock.call_args_list[1][0][0]
assert (
'*Key:* `test_dict`\n*ID:* `2`\n*Trade ID:* `1`\n*Type:* `dict`\n'
'*Value:* `{"test": "dict"}`\n*Create Date:* `') in msg_mock.call_args_list[2][0][0]
msg_mock.reset_mock()