Merge branch 'develop' into feature/fetch-public-trades

.github/workflows/ci.yml

@@ -420,7 +420,7 @@ jobs:
     ]
     runs-on: ubuntu-22.04
     # Discord notification can't handle schedule events
-    if: (github.event_name != 'schedule')
+    if: github.event_name != 'schedule' && github.repository == 'freqtrade/freqtrade'
     permissions:
       repository-projects: read
     steps:

.pre-commit-config.yaml

@@ -9,7 +9,7 @@ repos:
         # stages: [push]
 
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: "v1.8.0"
+    rev: "v1.9.0"
     hooks:
       - id: mypy
         exclude: build_helpers
@@ -19,7 +19,7 @@ repos:
           - types-requests==2.31.0.20240311
           - types-tabulate==0.9.0.20240106
           - types-python-dateutil==2.8.19.20240311
-          - SQLAlchemy==2.0.27
+          - SQLAlchemy==2.0.28
         # stages: [push]
 
   - repo: https://github.com/pycqa/isort
@@ -31,7 +31,7 @@ repos:
 
   - repo: https://github.com/charliermarsh/ruff-pre-commit
     # Ruff version.
-    rev: 'v0.3.0'
+    rev: 'v0.3.2'
     hooks:
       - id: ruff
 

CONTRIBUTING.md

@@ -48,7 +48,7 @@ pytest tests/test_<file_name>.py::test_<method_name>
 #### Run Ruff
 
 ```bash
-ruff .
+ruff check .
 ```
 
 We receive a lot of code that fails the `ruff` checks.

Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.11.8-slim-bookworm as base
+FROM python:3.12.2-slim-bookworm as base
 
 # Setup env
 ENV LANG C.UTF-8

docs/configuration.md

@@ -326,6 +326,8 @@ You'd set `available_capital=5000` - granting each bot an initial capital of 500
 The bot will then split this starting balance equally into `max_open_trades` buckets.
 Profitable trades will result in increased stake-sizes for this bot - without affecting the stake-sizes of the other bot.
 
+Adjusting `available_capital` requires reloading the configuration to take effect. Adjusting the `available_capital` adds the difference between the previous `available_capital` and the new `available_capital`. Decreasing the available capital when trades are open doesn't exit the trades. The difference is returned to the wallet when the trades conclude. The outcome of this differs depending on the price movement between the adjustment and exiting the trades.
+
 !!! Warning "Incompatible with `tradable_balance_ratio`"
     Setting this option will replace any configuration of `tradable_balance_ratio`.
 
@@ -503,13 +505,13 @@ Configuration:
     Please carefully read the section [Market order pricing](#market-order-pricing) section when using market orders.
 
 !!! Note "Stoploss on exchange"
-    `stoploss_on_exchange_interval` is not mandatory. Do not change its value if you are
+    `order_types.stoploss_on_exchange_interval` is not mandatory. Do not change its value if you are
     unsure of what you are doing. For more information about how stoploss works please
     refer to [the stoploss documentation](stoploss.md).
 
-    If `stoploss_on_exchange` is enabled and the stoploss is cancelled manually on the exchange, then the bot will create a new stoploss order.
+    If `order_types.stoploss_on_exchange` is enabled and the stoploss is cancelled manually on the exchange, then the bot will create a new stoploss order.
 
-!!! Warning "Warning: stoploss_on_exchange failures"
+!!! Warning "Warning: order_types.stoploss_on_exchange failures"
     If stoploss on exchange creation fails for some reason, then an "emergency exit" is initiated. By default, this will exit the trade using a market order. The order-type for the emergency-exit can be changed by setting the `emergency_exit` value in the `order_types` dictionary - however, this is not advised.
 
 ### Understand order_time_in_force

docs/exchanges.md

@@ -68,6 +68,8 @@ Binance supports [time_in_force](configuration.md#understand-order_time_in_force
 For Binance, it is suggested to add `"BNB/<STAKE>"` to your blacklist to avoid issues, unless you are willing to maintain enough extra `BNB` on the account or unless you're willing to disable using `BNB` for fees.
 Binance accounts may use `BNB` for fees, and if a trade happens to be on `BNB`, further trades may consume this position and make the initial BNB trade unsellable as the expected amount is not there anymore.
 
+If not enough `BNB` is available to cover transaction fees, then fees will not be covered by `BNB` and no fee reduction will occur. Freqtrade will never buy BNB to cover for fees. BNB needs to be bought and monitored manually to this end.
+
 ### Binance sites
 
 Binance has been split into 2, and users must use the correct ccxt exchange ID for their exchange, otherwise API keys are not recognized.

docs/includes/pairlists.md

@@ -369,6 +369,11 @@ As this Filter uses past performance of the bot, it'll have some startup-period
 
 Filters low-value coins which would not allow setting stoplosses.
 
+Namely, pairs are blacklisted if a variance of one percent or more in the stop price would be caused by precision rounding on the exchange, i.e. `rounded(stop_price) <= rounded(stop_price * 0.99)`. The idea is to avoid coins with a value VERY close to their lower trading boundary, not allowing setting of proper stoploss.
+
+!!! Tip "PerformanceFilter is pointless for futures trading"
+    The above does not apply to shorts. And for longs, in theory the trade will be liquidated first.
+
 !!! Warning "Backtesting"
     `PrecisionFilter` does not support backtesting mode using multiple strategies.
 

freqtrade/commands/data_commands.py

@@ -107,7 +107,7 @@ def start_list_data(args: Dict[str, Any]) -> None:
 
     from tabulate import tabulate
 
-    from freqtrade.data.history.idatahandler import get_datahandler
+    from freqtrade.data.history import get_datahandler
     dhc = get_datahandler(config['datadir'], config['dataformat_ohlcv'])
 
     paircombs = dhc.ohlcv_get_available_data(

freqtrade/configuration/load_config.py

@@ -46,7 +46,7 @@ def load_file(path: Path) -> Dict[str, Any]:
         with path.open('r') as file:
             config = rapidjson.load(file, parse_mode=CONFIG_PARSE_MODE)
     except FileNotFoundError:
-        raise OperationalException(f'File "{path}" not found!')
+        raise OperationalException(f'File "{path}" not found!') from None
     return config
 
 
@@ -63,7 +63,7 @@ def load_config_file(path: str) -> Dict[str, Any]:
     except FileNotFoundError:
         raise OperationalException(
             f'Config file "{path}" not found!'
-            ' Please create a config file or check whether it exists.')
+            ' Please create a config file or check whether it exists.') from None
     except rapidjson.JSONDecodeError as e:
         err_range = log_config_error_range(path, str(e))
         raise OperationalException(

freqtrade/data/converter/converter.py

@@ -200,7 +200,7 @@ def convert_ohlcv_format(
     :param convert_to: Target format
     :param erase: Erase source data (does not apply if source and target format are identical)
     """
-    from freqtrade.data.history.idatahandler import get_datahandler
+    from freqtrade.data.history import get_datahandler
     src = get_datahandler(config['datadir'], convert_from)
     trg = get_datahandler(config['datadir'], convert_to)
     timeframes = config.get('timeframes', [config.get('timeframe')])

freqtrade/data/converter/trade_converter.py

@@ -96,7 +96,7 @@ def convert_trades_to_ohlcv(
     """
     Convert stored trades data to ohlcv data
     """
-    from freqtrade.data.history.idatahandler import get_datahandler
+    from freqtrade.data.history import get_datahandler
     data_handler_trades = get_datahandler(datadir, data_format=data_format_trades)
     data_handler_ohlcv = get_datahandler(datadir, data_format=data_format_ohlcv)
 
@@ -135,7 +135,7 @@ def convert_trades_format(config: Config, convert_from: str, convert_to: str, er
         import_kraken_trades_from_csv(config, convert_to)
         return
 
-    from freqtrade.data.history.idatahandler import get_datahandler
+    from freqtrade.data.history import get_datahandler
     src = get_datahandler(config['datadir'], convert_from)
     trg = get_datahandler(config['datadir'], convert_to)
 

freqtrade/data/converter/trade_converter_kraken.py

@@ -6,7 +6,7 @@ import pandas as pd
 from freqtrade.constants import DATETIME_PRINT_FORMAT, DEFAULT_TRADES_COLUMNS, Config
 from freqtrade.data.converter.trade_converter import (trades_convert_types,
                                                       trades_df_remove_duplicates)
-from freqtrade.data.history.idatahandler import get_datahandler
+from freqtrade.data.history import get_datahandler
 from freqtrade.enums import TradingMode
 from freqtrade.exceptions import OperationalException
 from freqtrade.plugins.pairlist.pairlist_helpers import expand_pairlist

freqtrade/data/history/__init__.py

@@ -6,7 +6,7 @@ Includes:
 * download data from exchange and store to disk
 """
 # flake8: noqa: F401
+from .datahandlers import get_datahandler
 from .history_utils import (convert_trades_to_ohlcv, download_data_main, get_timerange, load_data,
                             load_pair_history, refresh_backtest_ohlcv_data,
                             refresh_backtest_trades_data, refresh_data, validate_backtest_data)
-from .idatahandler import get_datahandler

freqtrade/data/history/datahandlers/__init__.py

@@ -0,0 +1,2 @@
+# flake8: noqa: F401
+from .idatahandler import IDataHandler, get_datahandler

freqtrade/data/history/history_utils.py

@@ -12,7 +12,7 @@ from freqtrade.constants import (DATETIME_PRINT_FORMAT, DEFAULT_DATAFRAME_COLUMN
 from freqtrade.data.converter import (clean_ohlcv_dataframe, convert_trades_to_ohlcv,
                                       ohlcv_to_dataframe, trades_df_remove_duplicates,
                                       trades_list_to_df)
-from freqtrade.data.history.idatahandler import IDataHandler, get_datahandler
+from freqtrade.data.history.datahandlers import IDataHandler, get_datahandler
 from freqtrade.enums import CandleType, TradingMode
 from freqtrade.exceptions import OperationalException
 from freqtrade.exchange import Exchange

freqtrade/exchange/__init__.py

@@ -15,10 +15,12 @@ from freqtrade.exchange.exchange_utils import (ROUND_DOWN, ROUND_UP, amount_to_c
                                                contracts_to_amount, date_minus_candles,
                                                is_exchange_known_ccxt, list_available_exchanges,
                                                market_is_active, price_to_precision,
-                                               timeframe_to_minutes, timeframe_to_msecs,
-                                               timeframe_to_next_date, timeframe_to_prev_date,
-                                               timeframe_to_resample_freq, timeframe_to_seconds,
                                                validate_exchange)
+from freqtrade.exchange.exchange_utils_timeframe import (timeframe_to_minutes, timeframe_to_msecs,
+                                                         timeframe_to_next_date,
+                                                         timeframe_to_prev_date,
+                                                         timeframe_to_resample_freq,
+                                                         timeframe_to_seconds)
 from freqtrade.exchange.gate import Gate
 from freqtrade.exchange.hitbtc import Hitbtc
 from freqtrade.exchange.htx import Htx

freqtrade/exchange/exchange.py

@@ -37,9 +37,10 @@ from freqtrade.exchange.exchange_utils import (ROUND, ROUND_DOWN, ROUND_UP, Ccxt
                                                amount_to_contract_precision, amount_to_contracts,
                                                amount_to_precision, contracts_to_amount,
                                                date_minus_candles, is_exchange_known_ccxt,
-                                               market_is_active, price_to_precision,
-                                               timeframe_to_minutes, timeframe_to_msecs,
-                                               timeframe_to_next_date, timeframe_to_prev_date,
+                                               market_is_active, price_to_precision)
+from freqtrade.exchange.exchange_utils_timeframe import (timeframe_to_minutes, timeframe_to_msecs,
+                                                         timeframe_to_next_date,
+                                                         timeframe_to_prev_date,
                                                          timeframe_to_seconds)
 from freqtrade.exchange.types import OHLCVResponse, OrderBook, Ticker, Tickers
 from freqtrade.misc import (chunks, deep_merge_dicts, file_dump_json, file_load_json,

freqtrade/exchange/exchange_utils.py

@@ -11,9 +11,9 @@ from ccxt import (DECIMAL_PLACES, ROUND, ROUND_DOWN, ROUND_UP, SIGNIFICANT_DIGIT
 
 from freqtrade.exchange.common import (BAD_EXCHANGES, EXCHANGE_HAS_OPTIONAL, EXCHANGE_HAS_REQUIRED,
                                        SUPPORTED_EXCHANGES)
+from freqtrade.exchange.exchange_utils_timeframe import timeframe_to_minutes, timeframe_to_prev_date
 from freqtrade.types import ValidExchangesType
 from freqtrade.util import FtPrecise
-from freqtrade.util.datetime_helpers import dt_from_ts, dt_ts
 
 
 CcxtModuleType = Any
@@ -108,78 +108,6 @@ def list_available_exchanges(all_exchanges: bool) -> List[ValidExchangesType]:
     return exchanges_valid
 
 
-def timeframe_to_seconds(timeframe: str) -> int:
-    """
-    Translates the timeframe interval value written in the human readable
-    form ('1m', '5m', '1h', '1d', '1w', etc.) to the number
-    of seconds for one timeframe interval.
-    """
-    return ccxt.Exchange.parse_timeframe(timeframe)
-
-
-def timeframe_to_minutes(timeframe: str) -> int:
-    """
-    Same as timeframe_to_seconds, but returns minutes.
-    """
-    return ccxt.Exchange.parse_timeframe(timeframe) // 60
-
-
-def timeframe_to_msecs(timeframe: str) -> int:
-    """
-    Same as timeframe_to_seconds, but returns milliseconds.
-    """
-    return ccxt.Exchange.parse_timeframe(timeframe) * 1000
-
-
-def timeframe_to_resample_freq(timeframe: str) -> str:
-    """
-    Translates the timeframe interval value written in the human readable
-    form ('1m', '5m', '1h', '1d', '1w', etc.) to the resample frequency
-    used by pandas ('1T', '5T', '1H', '1D', '1W', etc.)
-    """
-    if timeframe == '1y':
-        return '1YS'
-    timeframe_seconds = timeframe_to_seconds(timeframe)
-    timeframe_minutes = timeframe_seconds // 60
-    resample_interval = f'{timeframe_seconds}s'
-    if 10000 < timeframe_minutes < 43200:
-        resample_interval = '1W-MON'
-    elif timeframe_minutes >= 43200 and timeframe_minutes < 525600:
-        # Monthly candles need special treatment to stick to the 1st of the month
-        resample_interval = f'{timeframe}S'
-    elif timeframe_minutes > 43200:
-        resample_interval = timeframe
-    return resample_interval
-
-
-def timeframe_to_prev_date(timeframe: str, date: Optional[datetime] = None) -> datetime:
-    """
-    Use Timeframe and determine the candle start date for this date.
-    Does not round when given a candle start date.
-    :param timeframe: timeframe in string format (e.g. "5m")
-    :param date: date to use. Defaults to now(utc)
-    :returns: date of previous candle (with utc timezone)
-    """
-    if not date:
-        date = datetime.now(timezone.utc)
-
-    new_timestamp = ccxt.Exchange.round_timeframe(timeframe, dt_ts(date), ROUND_DOWN) // 1000
-    return dt_from_ts(new_timestamp)
-
-
-def timeframe_to_next_date(timeframe: str, date: Optional[datetime] = None) -> datetime:
-    """
-    Use Timeframe and determine next candle.
-    :param timeframe: timeframe in string format (e.g. "5m")
-    :param date: date to use. Defaults to now(utc)
-    :returns: date of next candle (with utc timezone)
-    """
-    if not date:
-        date = datetime.now(timezone.utc)
-    new_timestamp = ccxt.Exchange.round_timeframe(timeframe, dt_ts(date), ROUND_UP) // 1000
-    return dt_from_ts(new_timestamp)
-
-
 def date_minus_candles(
         timeframe: str, candle_count: int, date: Optional[datetime] = None) -> datetime:
     """

freqtrade/exchange/exchange_utils_timeframe.py

@@ -0,0 +1,81 @@
+from datetime import datetime, timezone
+from typing import Optional
+
+import ccxt
+from ccxt import ROUND_DOWN, ROUND_UP
+
+from freqtrade.util.datetime_helpers import dt_from_ts, dt_ts
+
+
+def timeframe_to_seconds(timeframe: str) -> int:
+    """
+    Translates the timeframe interval value written in the human readable
+    form ('1m', '5m', '1h', '1d', '1w', etc.) to the number
+    of seconds for one timeframe interval.
+    """
+    return ccxt.Exchange.parse_timeframe(timeframe)
+
+
+def timeframe_to_minutes(timeframe: str) -> int:
+    """
+    Same as timeframe_to_seconds, but returns minutes.
+    """
+    return ccxt.Exchange.parse_timeframe(timeframe) // 60
+
+
+def timeframe_to_msecs(timeframe: str) -> int:
+    """
+    Same as timeframe_to_seconds, but returns milliseconds.
+    """
+    return ccxt.Exchange.parse_timeframe(timeframe) * 1000
+
+
+def timeframe_to_resample_freq(timeframe: str) -> str:
+    """
+    Translates the timeframe interval value written in the human readable
+    form ('1m', '5m', '1h', '1d', '1w', etc.) to the resample frequency
+    used by pandas ('1T', '5T', '1H', '1D', '1W', etc.)
+    """
+    if timeframe == '1y':
+        return '1YS'
+    timeframe_seconds = timeframe_to_seconds(timeframe)
+    timeframe_minutes = timeframe_seconds // 60
+    resample_interval = f'{timeframe_seconds}s'
+    if 10000 < timeframe_minutes < 43200:
+        resample_interval = '1W-MON'
+    elif timeframe_minutes >= 43200 and timeframe_minutes < 525600:
+        # Monthly candles need special treatment to stick to the 1st of the month
+        resample_interval = f'{timeframe}S'
+    elif timeframe_minutes > 43200:
+        resample_interval = timeframe
+    return resample_interval
+
+
+def timeframe_to_prev_date(timeframe: str, date: Optional[datetime] = None) -> datetime:
+    """
+    Use Timeframe and determine the candle start date for this date.
+    Does not round when given a candle start date.
+    :param timeframe: timeframe in string format (e.g. "5m")
+    :param date: date to use. Defaults to now(utc)
+    :returns: date of previous candle (with utc timezone)
+    """
+    if not date:
+        date = datetime.now(timezone.utc)
+
+    new_timestamp = ccxt.Exchange.round_timeframe(
+        timeframe, dt_ts(date), ROUND_DOWN) // 1000
+    return dt_from_ts(new_timestamp)
+
+
+def timeframe_to_next_date(timeframe: str, date: Optional[datetime] = None) -> datetime:
+    """
+    Use Timeframe and determine next candle.
+    :param timeframe: timeframe in string format (e.g. "5m")
+    :param date: date to use. Defaults to now(utc)
+    :returns: date of next candle (with utc timezone)
+    """
+    if not date:
+        date = datetime.now(timezone.utc)
+    new_timestamp = ccxt.Exchange.round_timeframe(
+        timeframe, dt_ts(date), ROUND_UP) // 1000
+    return dt_from_ts(new_timestamp)

freqtrade/freqai/freqai_interface.py

@@ -825,7 +825,7 @@ class IFreqaiModel(ABC):
         """
         if self.config.get("freqai_backtest_live_models", False) and len_dataframe_backtest == 0:
             logger.info(f"No data found for pair {pair} from "
-                        f"from { tr_backtest.start_fmt} to {tr_backtest.stop_fmt}. "
+                        f"from {tr_backtest.start_fmt} to {tr_backtest.stop_fmt}. "
                         "Probably more than one training within the same candle period.")
             return False
         return True

freqtrade/optimize/space/decimalspace.py

@@ -20,8 +20,10 @@ class SKDecimal(Integer):
         super().__init__(_low, _high, prior, base, transform, name, dtype)
 
     def __repr__(self):
-        return "Decimal(low={}, high={}, decimals={}, prior='{}', transform='{}')".format(
-            self.low_orig, self.high_orig, self.decimals, self.prior, self.transform_)
+        return (
+            f"Decimal(low={self.low_orig}, high={self.high_orig}, decimals={self.decimals}, "
+            f"prior='{self.prior}', transform='{self.transform_}')"
+        )
 
     def __contains__(self, point):
         if isinstance(point, list):

freqtrade/rpc/fiat_convert.py

@@ -77,9 +77,8 @@ class CryptoToFiatConverter(LoggingMixin):
                 return
             # If the request is not a 429 error we want to raise the normal error
             logger.error(
-                "Could not load FIAT Cryptocurrency map for the following problem: {}".format(
-                    request_exception
-                )
+                "Could not load FIAT Cryptocurrency map for the following problem: "
+                f"{request_exception}"
             )
         except (Exception) as exception:
             logger.error(

freqtrade/util/migrations/binance_mig.py

@@ -63,7 +63,7 @@ def migrate_binance_futures_data(config: Config):
         # only act on new futures
         return
 
-    from freqtrade.data.history.idatahandler import get_datahandler
+    from freqtrade.data.history import get_datahandler
     dhc = get_datahandler(config['datadir'], config['dataformat_ohlcv'])
 
     paircombs = dhc.ohlcv_get_available_data(

freqtrade/util/migrations/funding_rate_mig.py

@@ -2,7 +2,7 @@ import logging
 from typing import Optional
 
 from freqtrade.constants import Config
-from freqtrade.data.history.idatahandler import get_datahandler
+from freqtrade.data.history import get_datahandler
 from freqtrade.enums import TradingMode
 from freqtrade.exchange import Exchange
 

requirements-dev.txt

@@ -7,7 +7,7 @@
 -r docs/requirements-docs.txt
 
 coveralls==3.3.1
-ruff==0.3.0
+ruff==0.3.2
 mypy==1.9.0
 pre-commit==3.6.2
 pytest==8.1.1

requirements-freqai-rl.txt

@@ -2,10 +2,9 @@
 -r requirements-freqai.txt
 
 # Required for freqai-rl
-torch==2.1.2; python_version < '3.12'
-#until these branches will be released we can use this
-gymnasium==0.29.1; python_version < '3.12'
-stable_baselines3==2.2.1; python_version < '3.12'
-sb3_contrib>=2.0.0a9; python_version < '3.12'
+torch==2.2.1
+gymnasium==0.29.1
+stable_baselines3==2.2.1
+sb3_contrib>=2.2.1
 # Progress bar for stable-baselines3 and sb3-contrib
 tqdm==4.66.2

requirements.txt

@@ -2,10 +2,10 @@ numpy==1.26.4
 pandas==2.2.1
 pandas-ta==0.3.14b
 
-ccxt==4.2.66
+ccxt==4.2.67
 cryptography==42.0.5
 aiohttp==3.9.3
-SQLAlchemy==2.0.27
+SQLAlchemy==2.0.28
 python-telegram-bot==21.0.1
 # can't be hard-pinned due to telegram-bot pinning httpx with ~
 httpx>=0.24.1
@@ -22,7 +22,7 @@ jinja2==3.1.3
 tables==3.9.1
 joblib==1.3.2
 rich==13.7.1
-pyarrow==15.0.0; platform_machine != 'armv7l'
+pyarrow==15.0.1; platform_machine != 'armv7l'
 
 # find first, C search in arrays
 py_find_1st==1.1.6
@@ -38,7 +38,7 @@ sdnotify==0.3.2
 # API Server
 fastapi==0.110.0
 pydantic==2.6.3
-uvicorn==0.27.1
+uvicorn==0.28.0
 pyjwt==2.8.0
 aiofiles==23.2.1
 psutil==5.9.8

tests/data/test_converter.py

@@ -15,7 +15,7 @@ from freqtrade.data.converter import (convert_ohlcv_format, convert_trades_forma
                                       trades_to_ohlcv, trim_dataframe)
 from freqtrade.data.history import (get_timerange, load_data, load_pair_history,
                                     validate_backtest_data)
-from freqtrade.data.history.idatahandler import IDataHandler
+from freqtrade.data.history.datahandlers import IDataHandler
 from freqtrade.enums import CandleType
 from freqtrade.exchange import timeframe_to_minutes, timeframe_to_seconds
 from tests.conftest import generate_test_data, generate_trades_history, log_has, log_has_re

tests/data/test_datahandler.py

@@ -11,11 +11,12 @@ from pandas.testing import assert_frame_equal
 
 from freqtrade.configuration import TimeRange
 from freqtrade.constants import AVAILABLE_DATAHANDLERS
-from freqtrade.data.history.featherdatahandler import FeatherDataHandler
-from freqtrade.data.history.hdf5datahandler import HDF5DataHandler
-from freqtrade.data.history.idatahandler import IDataHandler, get_datahandler, get_datahandlerclass
-from freqtrade.data.history.jsondatahandler import JsonDataHandler, JsonGzDataHandler
-from freqtrade.data.history.parquetdatahandler import ParquetDataHandler
+from freqtrade.data.history.datahandlers.featherdatahandler import FeatherDataHandler
+from freqtrade.data.history.datahandlers.hdf5datahandler import HDF5DataHandler
+from freqtrade.data.history.datahandlers.idatahandler import (IDataHandler, get_datahandler,
+                                                              get_datahandlerclass)
+from freqtrade.data.history.datahandlers.jsondatahandler import JsonDataHandler, JsonGzDataHandler
+from freqtrade.data.history.datahandlers.parquetdatahandler import ParquetDataHandler
 from freqtrade.enums import CandleType, TradingMode
 from tests.conftest import log_has, log_has_re
 

tests/data/test_dataprovider.py

@@ -64,8 +64,11 @@ def test_historic_ohlcv(mocker, default_conf, ohlcv_history):
 def test_historic_ohlcv_dataformat(mocker, default_conf, ohlcv_history):
     hdf5loadmock = MagicMock(return_value=ohlcv_history)
     featherloadmock = MagicMock(return_value=ohlcv_history)
-    mocker.patch("freqtrade.data.history.hdf5datahandler.HDF5DataHandler._ohlcv_load", hdf5loadmock)
-    mocker.patch("freqtrade.data.history.featherdatahandler.FeatherDataHandler._ohlcv_load",
+    mocker.patch(
+        "freqtrade.data.history.datahandlers.hdf5datahandler.HDF5DataHandler._ohlcv_load",
+        hdf5loadmock)
+    mocker.patch(
+        "freqtrade.data.history.datahandlers.featherdatahandler.FeatherDataHandler._ohlcv_load",
         featherloadmock)
 
     default_conf["runmode"] = RunMode.BACKTEST

tests/data/test_history.py

@@ -15,14 +15,14 @@ from pandas.testing import assert_frame_equal
 from freqtrade.configuration import TimeRange
 from freqtrade.constants import DATETIME_PRINT_FORMAT
 from freqtrade.data.converter import ohlcv_to_dataframe
+from freqtrade.data.history import get_datahandler
+from freqtrade.data.history.datahandlers.jsondatahandler import JsonDataHandler, JsonGzDataHandler
 from freqtrade.data.history.history_utils import (_download_pair_history, _download_trades_history,
                                                   _load_cached_data_for_updating, get_timerange,
                                                   load_data, load_pair_history,
                                                   refresh_backtest_ohlcv_data,
                                                   refresh_backtest_trades_data, refresh_data,
                                                   validate_backtest_data)
-from freqtrade.data.history.idatahandler import get_datahandler
-from freqtrade.data.history.jsondatahandler import JsonDataHandler, JsonGzDataHandler
 from freqtrade.enums import CandleType, TradingMode
 from freqtrade.exchange import timeframe_to_minutes
 from freqtrade.misc import file_dump_json
@@ -91,7 +91,7 @@ def test_load_data_mark(ohlcv_history, mocker, caplog, testdatadir) -> None:
 
 def test_load_data_startup_candles(mocker, testdatadir) -> None:
     ltfmock = mocker.patch(
-        'freqtrade.data.history.featherdatahandler.FeatherDataHandler._ohlcv_load',
+        'freqtrade.data.history.datahandlers.featherdatahandler.FeatherDataHandler._ohlcv_load',
         MagicMock(return_value=DataFrame()))
     timerange = TimeRange('date', None, 1510639620, 0)
     load_pair_history(pair='UNITTEST/BTC', timeframe='1m',
@@ -326,7 +326,7 @@ def test_download_pair_history2(mocker, default_conf, testdatadir) -> None:
         [1509836580000, 0.00161, 0.00161, 0.00161, 0.00161, 82.390199]
     ]
     json_dump_mock = mocker.patch(
-        'freqtrade.data.history.featherdatahandler.FeatherDataHandler.ohlcv_store',
+        'freqtrade.data.history.datahandlers.featherdatahandler.FeatherDataHandler.ohlcv_store',
         return_value=None)
     mocker.patch(f'{EXMS}.get_historic_ohlcv', return_value=tick)
     exchange = get_patched_exchange(mocker, default_conf)

tests/data/test_trade_converter_kraken.py

@@ -5,7 +5,7 @@ from unittest.mock import PropertyMock
 import pytest
 
 from freqtrade.data.converter.trade_converter_kraken import import_kraken_trades_from_csv
-from freqtrade.data.history.idatahandler import get_datahandler
+from freqtrade.data.history import get_datahandler
 from freqtrade.enums import TradingMode
 from freqtrade.exceptions import OperationalException
 from tests.conftest import EXMS, log_has, log_has_re, patch_exchange

tests/freqai/conftest.py

@@ -33,7 +33,7 @@ def is_arm() -> bool:
 @pytest.fixture(autouse=True)
 def patch_torch_initlogs(mocker) -> None:
 
-    if is_mac() and not is_arm():
+    if is_mac():
         # Mock torch import completely
         import sys
         import types
@@ -41,7 +41,7 @@ def patch_torch_initlogs(mocker) -> None:
         module_name = 'torch'
         mocked_module = types.ModuleType(module_name)
         sys.modules[module_name] = mocked_module
-    elif not is_py12():
+    else:
         mocker.patch("torch._logging._init_logs")
 
 

tests/freqai/test_freqai_interface.py

@@ -14,20 +14,17 @@ from freqtrade.optimize.backtesting import Backtesting
 from freqtrade.persistence import Trade
 from freqtrade.plugins.pairlistmanager import PairListManager
 from tests.conftest import EXMS, create_mock_trades, get_patched_exchange, log_has_re
-from tests.freqai.conftest import (get_patched_freqai_strategy, is_arm, is_mac, is_py12,
-                                   make_rl_config, mock_pytorch_mlp_model_training_parameters)
+from tests.freqai.conftest import (get_patched_freqai_strategy, is_arm, is_mac, make_rl_config,
+                                   mock_pytorch_mlp_model_training_parameters)
 
 
 def can_run_model(model: str) -> None:
     is_pytorch_model = 'Reinforcement' in model or 'PyTorch' in model
 
-    if is_py12() and is_pytorch_model:
-        pytest.skip("Model not supported on python 3.12 yet.")
-
     if is_arm() and "Catboost" in model:
         pytest.skip("CatBoost is not supported on ARM.")
 
-    if is_pytorch_model and is_mac() and not is_arm():
+    if is_pytorch_model and is_mac():
         pytest.skip("Reinforcement learning / PyTorch module not available on intel based Mac OS.")
 
 
@@ -521,8 +518,6 @@ def test_get_state_info(mocker, freqai_conf, dp_exists, caplog, tickers):
 
     if is_mac():
         pytest.skip("Reinforcement learning module not available on intel based Mac OS")
-    if is_py12():
-        pytest.skip("Reinforcement learning currently not available on python 3.12.")
 
     freqai_conf.update({"freqaimodel": "ReinforcementLearner"})
     freqai_conf.update({"timerange": "20180110-20180130"})