Merge pull request #12479 from freqtrade/feat/hyperopt_custom_spaces

Add support for custom hyperopt spaces
This commit is contained in:
Matthias
2025-11-08 16:01:42 +01:00
committed by GitHub
18 changed files with 332 additions and 284 deletions

View File

@@ -301,18 +301,7 @@
"description": "Hyperopt parameter spaces to optimize. Default is the default set andincludes all spaces except for 'trailing', 'protection', and 'trades'.",
"type": "array",
"items": {
"type": "string",
"enum": [
"all",
"buy",
"sell",
"roi",
"stoploss",
"trailing",
"protection",
"trades",
"default"
]
"type": "string"
},
"default": [
"default"

View File

@@ -55,8 +55,9 @@ options:
-e INT, --epochs INT Specify number of epochs (default: 100).
--spaces SPACES [SPACES ...]
Specify which parameters to hyperopt. Space-separated
list. Available options: all, buy, sell, roi,
stoploss, trailing, protection, trades, default.
list. Available builtin options (custom spaces will
not be listed here): default, all, buy, sell, enter,
exit, roi, stoploss, trailing, protection, trades.
Default: `default` - which includes all spaces except
for 'trailing', 'protection', and 'trades'.
--print-all Print all results, not only the best ones.

View File

@@ -46,10 +46,17 @@ Depending on the space you want to optimize, only some of the below are required
* define parameters with `space='buy'` - for entry signal optimization
* define parameters with `space='sell'` - for exit signal optimization
* define parameters with `space='enter'` - for entry signal optimization
* define parameters with `space='exit'` - for exit signal optimization
* define parameters with `space='protection'` - for protection optimization
* define parameters with `space='random_spacename'` - for better control over which parameters are optimized together
Pick the space name that suits the parameter best. We recommend to use either `buy` / `sell` or `enter` / `exit` for clarity (however there's no technical limitation in this regard).
!!! Note
`populate_indicators` needs to create all indicators any of the spaces may use, otherwise hyperopt will not work.
Rarely you may also need to create a [nested class](advanced-hyperopt.md#overriding-pre-defined-spaces) named `HyperOpt` and implement
* `roi_space` - for custom ROI optimization (if you need the ranges for the ROI parameters in the optimization hyperspace that differ from default)
@@ -79,15 +86,15 @@ Based on the loss function result, hyperopt will determine the next set of param
### Configure your Guards and Triggers
There are two places you need to change in your strategy file to add a new buy hyperopt for testing:
There are two places you need to change in your strategy file to add a new hyperopt parameter for optimization:
* Define the parameters at the class level hyperopt shall be optimizing.
* Within `populate_entry_trend()` - use defined parameter values instead of raw constants.
There you have two different types of indicators: 1. `guards` and 2. `triggers`.
1. Guards are conditions like "never buy if ADX < 10", or never buy if current price is over EMA10.
2. Triggers are ones that actually trigger buy in specific moment, like "buy when EMA5 crosses over EMA10" or "buy when close price touches lower Bollinger band".
1. Guards are conditions like "never enter if ADX < 10", or never enter if current price is over EMA10.
2. Triggers are ones that actually trigger entry in specific moment, like "enter when EMA5 crosses over EMA10" or "enter when close price touches lower Bollinger band".
!!! Hint "Guards and Triggers"
Technically, there is no difference between Guards and Triggers.
@@ -160,9 +167,11 @@ We use these to either enable or disable the ADX and RSI guards.
The last one we call `trigger` and use it to decide which buy trigger we want to use.
!!! Note "Parameter space assignment"
Parameters must either be assigned to a variable named `buy_*` or `sell_*` - or contain `space='buy'` | `space='sell'` to be assigned to a space correctly.
If no parameter is available for a space, you'll receive the error that no space was found when running hyperopt.
- Parameters must either be assigned to a variable named `buy_*`, `sell_*`, `enter_*` or `exit_*` or `protection_*` - or contain have a space assigned explicitly via parameter (`space='buy'`, `space='sell'`, `space='protection'`).
- Parameters with conflicting assignments (e.g. `buy_adx = IntParameter(4, 24, default=14, space='sell')`) will use the explicit space assignment.
- If no parameter is available for a space, you'll receive the error that no space was found when running hyperopt.
Parameters with unclear space (e.g. `adx_period = IntParameter(4, 24, default=14)` - no explicit nor implicit space) will not be detected and will therefore be ignored.
Spaces can also be custom named (e.g. `space='my_custom_space'`), with the only limitation that the space name cannot be `all`, `default` - and must result in a valid python identifier.
So let's write the buy strategy using these values:
@@ -520,21 +529,24 @@ freqtrade hyperopt --strategy <strategyname> --timerange 20210101-20210201
### Running Hyperopt with Smaller Search Space
Use the `--spaces` option to limit the search space used by hyperopt.
Letting Hyperopt optimize everything is a huuuuge search space.
Often it might make more sense to start by just searching for initial buy algorithm.
Or maybe you just want to optimize your stoploss or roi table for that awesome new buy strategy you have.
Letting Hyperopt optimize everything is often a huuuuge search space.
Often it might make more sense to start by just searching for initial entry algorithm.
Or maybe you just want to optimize your stoploss or roi table for that awesome new strategy you have.
Legal values are:
* `all`: optimize everything
* `all`: optimize everything (including custom spaces)
* `buy`: just search for a new buy strategy
* `sell`: just search for a new sell strategy
* `enter`: just search for a new entry logic
* `exit`: just search for a new entry logic
* `roi`: just optimize the minimal profit table for your strategy
* `stoploss`: search for the best stoploss value
* `trailing`: search for the best trailing stop values
* `trades`: search for the best max open trades values
* `protection`: search for the best protection parameters (read the [protections section](#optimizing-protections) on how to properly define these)
* `default`: `all` except `trailing`, `trades` and `protection`
* `custom_space_name`: any custom space used by any parameter in your strategy
* space-separated list of any of the above values for example `--spaces roi stoploss`
The default Hyperopt Search Space, used when no `--space` command line option is specified, does not include the `trailing` hyperspace. We recommend you to run optimization for the `trailing` hyperspace separately, when the best parameters for other hyperspaces were found, validated and pasted into your custom strategy.

View File

@@ -5,7 +5,10 @@ Definition of cli arguments used in arguments.py
from argparse import ArgumentTypeError
from freqtrade import constants
from freqtrade.constants import HYPEROPT_BUILTIN_SPACES, HYPEROPT_LOSS_BUILTIN
from freqtrade.constants import (
HYPEROPT_BUILTIN_SPACE_OPTIONS,
HYPEROPT_LOSS_BUILTIN,
)
from freqtrade.enums import CandleType
@@ -278,9 +281,12 @@ AVAILABLE_CLI_OPTIONS = {
),
"spaces": Arg(
"--spaces",
help="Specify which parameters to hyperopt. Space-separated list. Available options: "
f"{', '.join(HYPEROPT_BUILTIN_SPACES)}. Default: `default` - "
"which includes all spaces except for 'trailing', 'protection', and 'trades'.",
help=(
"Specify which parameters to hyperopt. Space-separated list. "
"Available builtin options (custom spaces will not be listed here): "
f"{', '.join(HYPEROPT_BUILTIN_SPACE_OPTIONS)}. Default: `default` - "
"which includes all spaces except for 'trailing', 'protection', and 'trades'."
),
nargs="+",
),
"analyze_per_epoch": Arg(

View File

@@ -101,7 +101,7 @@ def _print_objs_tabular(objs: list, print_colorized: bool) -> None:
names = [s["name"] for s in objs]
objs_to_print: list[dict[str, Text | str]] = [
{
"name": Text(s["name"] if s["name"] else "--"),
"Strategy name": Text(s["name"] if s["name"] else "--"),
"location": s["location_rel"],
"status": (
Text("LOAD FAILED", style="bold red")
@@ -115,11 +115,19 @@ def _print_objs_tabular(objs: list, print_colorized: bool) -> None:
]
for idx, s in enumerate(objs):
if "hyperoptable" in s:
custom_params = [
f"{space}: {len(params)}"
for space, params in s["hyperoptable"].items()
if space not in ["buy", "sell", "protection"]
]
hyp = s["hyperoptable"]
objs_to_print[idx].update(
{
"hyperoptable": "Yes" if s["hyperoptable"]["count"] > 0 else "No",
"buy-Params": str(len(s["hyperoptable"].get("buy", []))),
"sell-Params": str(len(s["hyperoptable"].get("sell", []))),
"hyperoptable": "Yes" if len(hyp) > 0 else "No",
"buy-Params": str(len(hyp.get("buy", []))),
"sell-Params": str(len(hyp.get("sell", []))),
"protection-Params": str(len(hyp.get("protection", []))),
"custom-Params": ", ".join(custom_params) if custom_params else "",
}
)
table = Table()
@@ -140,6 +148,7 @@ def start_list_strategies(args: dict[str, Any]) -> None:
"""
from freqtrade.configuration import setup_utils_configuration
from freqtrade.resolvers import StrategyResolver
from freqtrade.strategy.hyper import detect_all_parameters
config = setup_utils_configuration(args, RunMode.UTIL_NO_EXCHANGE)
@@ -153,9 +162,9 @@ def start_list_strategies(args: dict[str, Any]) -> None:
strategy_objs = sorted(strategy_objs, key=lambda x: x["name"])
for obj in strategy_objs:
if obj["class"]:
obj["hyperoptable"] = obj["class"].detect_all_parameters()
obj["hyperoptable"] = detect_all_parameters(obj["class"])
else:
obj["hyperoptable"] = {"count": 0}
obj["hyperoptable"] = {}
if args["print_one_column"]:
print("\n".join([s["name"] for s in strategy_objs]))

View File

@@ -8,7 +8,6 @@ from freqtrade.constants import (
BACKTEST_CACHE_AGE,
DRY_RUN_WALLET,
EXPORT_OPTIONS,
HYPEROPT_BUILTIN_SPACES,
HYPEROPT_LOSS_BUILTIN,
MARGIN_MODES,
ORDERTIF_POSSIBILITIES,
@@ -260,7 +259,7 @@ CONF_SCHEMA = {
"includes all spaces except for 'trailing', 'protection', and 'trades'."
),
"type": "array",
"items": {"type": "string", "enum": HYPEROPT_BUILTIN_SPACES},
"items": {"type": "string"},
"default": ["default"],
},
"analyze_per_epoch": {

View File

@@ -42,16 +42,17 @@ HYPEROPT_LOSS_BUILTIN = [
"MultiMetricHyperOptLoss",
]
HYPEROPT_BUILTIN_SPACES = [
"all",
"buy",
"sell",
"enter",
"exit",
"roi",
"stoploss",
"trailing",
"protection",
"trades",
"default",
]
HYPEROPT_BUILTIN_SPACE_OPTIONS = ["default", "all"] + HYPEROPT_BUILTIN_SPACES
AVAILABLE_PAIRLISTS = [
"StaticPairList",

View File

@@ -7,6 +7,7 @@ This module implements a convenience auto-hyperopt class, which can be used toge
import logging
from collections.abc import Callable
from contextlib import suppress
from typing import Literal
from freqtrade.exceptions import OperationalException
@@ -41,6 +42,13 @@ class HyperOptAuto(IHyperOpt):
sell_indicator_space methods, but other hyperopt methods can be overridden as well.
"""
def get_available_spaces(self) -> list[str]:
"""
Get list of available spaces defined in strategy.
:return: list of available spaces.
"""
return list(self.strategy._ft_hyper_params)
def _get_func(self, name) -> Callable:
"""
Return a function defined in Strategy.HyperOpt class, or one defined in super() class.
@@ -59,7 +67,13 @@ class HyperOptAuto(IHyperOpt):
if attr.optimize:
yield attr.get_space(attr_name)
def _get_indicator_space(self, category) -> list:
def get_indicator_space(
self, category: Literal["buy", "sell", "enter", "exit", "protection"] | str
) -> list:
"""
Get indicator space for a given space.
:param category: parameter space to get.
"""
# TODO: is this necessary, or can we call "generate_space" directly?
indicator_space = list(self._generate_indicator_space(category))
if len(indicator_space) > 0:
@@ -70,15 +84,6 @@ class HyperOptAuto(IHyperOpt):
)
return []
def buy_indicator_space(self) -> list["Dimension"]:
return self._get_indicator_space("buy")
def sell_indicator_space(self) -> list["Dimension"]:
return self._get_indicator_space("sell")
def protection_space(self) -> list["Dimension"]:
return self._get_indicator_space("protection")
def generate_roi_table(self, params: dict) -> dict[int, float]:
return self._get_func("generate_roi_table")(params)

View File

@@ -70,13 +70,7 @@ class HyperOptimizer:
"""
def __init__(self, config: Config, data_pickle_file: Path) -> None:
self.buy_space: list[DimensionProtocol] = []
self.sell_space: list[DimensionProtocol] = []
self.protection_space: list[DimensionProtocol] = []
self.roi_space: list[DimensionProtocol] = []
self.stoploss_space: list[DimensionProtocol] = []
self.trailing_space: list[DimensionProtocol] = []
self.max_open_trades_space: list[DimensionProtocol] = []
self.spaces: dict[str, list[DimensionProtocol]] = {}
self.dimensions: list[DimensionProtocol] = []
self.o_dimensions: dict = {}
@@ -167,27 +161,25 @@ class HyperOptimizer:
"""
result: dict = {}
if HyperoptTools.has_space(self.config, "buy"):
result["buy"] = round_dict({p.name: params.get(p.name) for p in self.buy_space}, 13)
if HyperoptTools.has_space(self.config, "sell"):
result["sell"] = round_dict({p.name: params.get(p.name) for p in self.sell_space}, 13)
if HyperoptTools.has_space(self.config, "protection"):
for space in self.spaces.keys():
if space == "protection":
result["protection"] = round_dict(
{p.name: params.get(p.name) for p in self.protection_space}, 13
{p.name: params.get(p.name) for p in self.spaces[space]}, 13
)
if HyperoptTools.has_space(self.config, "roi"):
elif space == "roi":
result["roi"] = round_dict(
{str(k): v for k, v in self.custom_hyperopt.generate_roi_table(params).items()}, 13
{str(k): v for k, v in self.custom_hyperopt.generate_roi_table(params).items()},
13,
)
if HyperoptTools.has_space(self.config, "stoploss"):
elif space == "stoploss":
result["stoploss"] = round_dict(
{p.name: params.get(p.name) for p in self.stoploss_space}, 13
{p.name: params.get(p.name) for p in self.spaces[space]}, 13
)
if HyperoptTools.has_space(self.config, "trailing"):
elif space == "trailing":
result["trailing"] = round_dict(
self.custom_hyperopt.generate_trailing_params(params), 13
)
if HyperoptTools.has_space(self.config, "trades"):
elif space == "trades":
result["max_open_trades"] = round_dict(
{
"max_open_trades": (
@@ -198,6 +190,10 @@ class HyperOptimizer:
},
13,
)
else:
result[space] = round_dict(
{p.name: params.get(p.name) for p in self.spaces[space]}, 13
)
return result
@@ -226,56 +222,39 @@ class HyperOptimizer:
"""
Assign the dimensions in the hyperoptimization space.
"""
if HyperoptTools.has_space(self.config, "protection"):
spaces = ["buy", "sell", "protection", "roi", "stoploss", "trailing", "trades"]
spaces += [s for s in self.custom_hyperopt.get_available_spaces() if s not in spaces]
for space in spaces:
if not HyperoptTools.has_space(self.config, space):
continue
logger.debug(f"Hyperopt has '{space}' space")
if space == "protection":
# Protections can only be optimized when using the Parameter interface
logger.debug("Hyperopt has 'protection' space")
# Enable Protections if protection space is selected.
self.config["enable_protections"] = True
self.backtesting.enable_protections = True
self.protection_space = self.custom_hyperopt.protection_space()
self.spaces[space] = self.custom_hyperopt.get_indicator_space(space)
elif space == "roi":
self.spaces[space] = self.custom_hyperopt.roi_space()
elif space == "stoploss":
self.spaces[space] = self.custom_hyperopt.stoploss_space()
elif space == "trailing":
self.spaces[space] = self.custom_hyperopt.trailing_space()
elif space == "trades":
self.spaces[space] = self.custom_hyperopt.max_open_trades_space()
else:
self.spaces[space] = self.custom_hyperopt.get_indicator_space(space)
if HyperoptTools.has_space(self.config, "buy"):
logger.debug("Hyperopt has 'buy' space")
self.buy_space = self.custom_hyperopt.buy_indicator_space()
if HyperoptTools.has_space(self.config, "sell"):
logger.debug("Hyperopt has 'sell' space")
self.sell_space = self.custom_hyperopt.sell_indicator_space()
if HyperoptTools.has_space(self.config, "roi"):
logger.debug("Hyperopt has 'roi' space")
self.roi_space = self.custom_hyperopt.roi_space()
if HyperoptTools.has_space(self.config, "stoploss"):
logger.debug("Hyperopt has 'stoploss' space")
self.stoploss_space = self.custom_hyperopt.stoploss_space()
if HyperoptTools.has_space(self.config, "trailing"):
logger.debug("Hyperopt has 'trailing' space")
self.trailing_space = self.custom_hyperopt.trailing_space()
if HyperoptTools.has_space(self.config, "trades"):
logger.debug("Hyperopt has 'trades' space")
self.max_open_trades_space = self.custom_hyperopt.max_open_trades_space()
self.dimensions = (
self.buy_space
+ self.sell_space
+ self.protection_space
+ self.roi_space
+ self.stoploss_space
+ self.trailing_space
+ self.max_open_trades_space
self.dimensions = [s for space in self.spaces.values() for s in space]
if len(self.dimensions) == 0:
raise OperationalException(
"No hyperopt parameters found to optimize."
f"Available spaces: {', '.join(spaces)}. "
"Check your strategy's parameter definitions or verify the configured spaces "
"in your command."
)
def assign_params(self, params_dict: dict[str, Any], category: str) -> None:
"""
Assign hyperoptable parameters
"""
for attr_name, attr in self.backtesting.strategy.enumerate_parameters(category):
if attr.optimize:
# noinspection PyProtectedMember
attr.value = params_dict[attr_name]
self.o_dimensions = self.convert_dimensions_to_optuna_space(self.dimensions)
@delayed
@wrap_non_picklable_objects
@@ -292,15 +271,9 @@ class HyperOptimizer:
HyperoptStateContainer.set_state(HyperoptState.OPTIMIZE)
backtest_start_time = datetime.now(UTC)
# Apply parameters
if HyperoptTools.has_space(self.config, "buy"):
self.assign_params(params_dict, "buy")
if HyperoptTools.has_space(self.config, "sell"):
self.assign_params(params_dict, "sell")
if HyperoptTools.has_space(self.config, "protection"):
self.assign_params(params_dict, "protection")
for attr_name, attr in self.backtesting.strategy.enumerate_parameters():
if attr.in_space and attr.optimize:
attr.value = params_dict[attr_name]
if HyperoptTools.has_space(self.config, "roi"):
self.backtesting.strategy.minimal_roi = self.custom_hyperopt.generate_roi_table(
@@ -436,7 +409,6 @@ class HyperOptimizer:
o_sampler = self.custom_hyperopt.generate_estimator(
dimensions=self.dimensions, random_state=random_state
)
self.o_dimensions = self.convert_dimensions_to_optuna_space(self.dimensions)
if isinstance(o_sampler, str):
if o_sampler not in optuna_samplers_dict.keys():

View File

@@ -9,7 +9,7 @@ import numpy as np
import rapidjson
from pandas import isna, json_normalize
from freqtrade.constants import FTHYPT_FILEVERSION, Config
from freqtrade.constants import FTHYPT_FILEVERSION, HYPEROPT_BUILTIN_SPACES, Config
from freqtrade.enums import HyperoptState
from freqtrade.exceptions import OperationalException
from freqtrade.misc import deep_merge_dicts, round_dict, safe_value_fallback2
@@ -219,20 +219,21 @@ class HyperoptTools:
print(rapidjson.dumps(result_dict, default=str, number_mode=HYPER_PARAMS_FILE_FORMAT))
else:
HyperoptTools._params_pretty_print(
params, "buy", "Buy hyperspace params:", non_optimized
all_spaces = list(params.keys() | non_optimized.keys())
# Explicitly listed to keep original sort order
spaces = ["buy", "sell", "protection", "roi", "stoploss", "trailing", "max_open_trades"]
spaces += [s for s in all_spaces if s not in spaces]
lookup = {
"roi": "ROI",
"trailing": "Trailing stop",
}
for space in spaces:
name = lookup.get(
space, space.capitalize() if space in HYPEROPT_BUILTIN_SPACES else space
)
HyperoptTools._params_pretty_print(
params, "sell", "Sell hyperspace params:", non_optimized
)
HyperoptTools._params_pretty_print(
params, "protection", "Protection hyperspace params:", non_optimized
)
HyperoptTools._params_pretty_print(params, "roi", "ROI table:", non_optimized)
HyperoptTools._params_pretty_print(params, "stoploss", "Stoploss:", non_optimized)
HyperoptTools._params_pretty_print(params, "trailing", "Trailing stop:", non_optimized)
HyperoptTools._params_pretty_print(
params, "max_open_trades", "Max Open Trades:", non_optimized
params, space, f"{name} parameters:", non_optimized
)
@staticmethod

View File

@@ -4,9 +4,9 @@ This module defines a base class for auto-hyperoptable strategies.
"""
import logging
from collections import defaultdict
from collections.abc import Iterator
from pathlib import Path
from typing import Any
from freqtrade.constants import Config
from freqtrade.exceptions import OperationalException
@@ -18,6 +18,11 @@ from freqtrade.strategy.parameters import BaseParameter
logger = logging.getLogger(__name__)
# Type aliases
SpaceParams = dict[str, BaseParameter]
AllSpaceParams = dict[str, SpaceParams]
class HyperStrategyMixin:
"""
A helper base class which allows HyperOptAuto class to reuse implementations of buy/sell
@@ -29,9 +34,7 @@ class HyperStrategyMixin:
Initialize hyperoptable strategy mixin.
"""
self.config = config
self.ft_buy_params: list[BaseParameter] = []
self.ft_sell_params: list[BaseParameter] = []
self.ft_protection_params: list[BaseParameter] = []
self._ft_hyper_params: AllSpaceParams = {}
params = self.load_params_from_file()
params = params.get("params", {})
@@ -46,31 +49,10 @@ class HyperStrategyMixin:
:param category:
:return:
"""
if category not in ("buy", "sell", "protection", None):
raise OperationalException(
'Category must be one of: "buy", "sell", "protection", None.'
)
if category is None:
params = self.ft_buy_params + self.ft_sell_params + self.ft_protection_params
else:
params = getattr(self, f"ft_{category}_params")
for par in params:
for category in [c for c in self._ft_hyper_params if category is None or c == category]:
for par in self._ft_hyper_params[category].values():
yield par.name, par
@classmethod
def detect_all_parameters(cls) -> dict:
"""Detect all parameters and return them as a list"""
params: dict[str, Any] = {
"buy": list(detect_parameters(cls, "buy")),
"sell": list(detect_parameters(cls, "sell")),
"protection": list(detect_parameters(cls, "protection")),
}
params.update({"count": len(params["buy"] + params["sell"] + params["protection"])})
return params
def ft_load_params_from_file(self) -> None:
"""
Load Parameters from parameter file
@@ -110,20 +92,13 @@ class HyperStrategyMixin:
* Parameters defined in parameters objects (buy_params, sell_params, ...)
* Parameter defaults
"""
self._ft_hyper_params = detect_all_parameters(self)
buy_params = deep_merge_dicts(
self._ft_params_from_file.get("buy", {}), getattr(self, "buy_params", {})
for space in self._ft_hyper_params.keys():
params_values = deep_merge_dicts(
self._ft_params_from_file.get(space, {}), getattr(self, f"{space}_params", {})
)
sell_params = deep_merge_dicts(
self._ft_params_from_file.get("sell", {}), getattr(self, "sell_params", {})
)
protection_params = deep_merge_dicts(
self._ft_params_from_file.get("protection", {}), getattr(self, "protection_params", {})
)
self._ft_load_params(buy_params, "buy", hyperopt)
self._ft_load_params(sell_params, "sell", hyperopt)
self._ft_load_params(protection_params, "protection", hyperopt)
self._ft_load_params(self._ft_hyper_params[space], params_values, space, hyperopt)
def load_params_from_file(self) -> dict:
filename_str = getattr(self, "__file__", "")
@@ -145,72 +120,73 @@ class HyperStrategyMixin:
return {}
def _ft_load_params(self, params: dict, space: str, hyperopt: bool = False) -> None:
def _ft_load_params(
self, params: SpaceParams, param_values: dict, space: str, hyperopt: bool = False
) -> None:
"""
Set optimizable parameter values.
:param params: Dictionary with new parameter values.
"""
if not params:
if not param_values:
logger.info(f"No params for {space} found, using default values.")
param_container: list[BaseParameter] = getattr(self, f"ft_{space}_params")
for attr_name, attr in detect_parameters(self, space):
attr.name = attr_name
attr.in_space = hyperopt and HyperoptTools.has_space(self.config, space)
if not attr.category:
attr.category = space
for param_name, param in params.items():
param.in_space = hyperopt and HyperoptTools.has_space(self.config, space)
if not param.category:
param.category = space
param_container.append(attr)
if params and attr_name in params:
if attr.load:
attr.value = params[attr_name]
logger.info(f"Strategy Parameter: {attr_name} = {attr.value}")
if param_values and param_name in param_values:
if param.load:
param.value = param_values[param_name]
logger.info(f"Strategy Parameter: {param_name} = {param.value}")
else:
logger.warning(
f'Parameter "{attr_name}" exists, but is disabled. '
f'Default value "{attr.value}" used.'
f'Parameter "{param_name}" exists, but is disabled. '
f'Default value "{param.value}" used.'
)
else:
logger.info(f"Strategy Parameter(default): {attr_name} = {attr.value}")
logger.info(f"Strategy Parameter(default): {param_name} = {param.value}")
def get_no_optimize_params(self) -> dict[str, dict]:
"""
Returns list of Parameters that are not part of the current optimize job
"""
params: dict[str, dict] = {
"buy": {},
"sell": {},
"protection": {},
}
params: dict[str, dict] = defaultdict(dict)
for name, p in self.enumerate_parameters():
if p.category and (not p.optimize or not p.in_space):
params[p.category][name] = p.value
return params
def detect_parameters(
obj: HyperStrategyMixin | type[HyperStrategyMixin], category: str
) -> Iterator[tuple[str, BaseParameter]]:
def detect_all_parameters(
obj: HyperStrategyMixin | type[HyperStrategyMixin],
) -> AllSpaceParams:
"""
Detect all parameters for 'category' for "obj"
Detect all hyperoptable parameters for this object.
:param obj: Strategy object or class
:param category: category - usually `'buy', 'sell', 'protection',...
:return: Dictionary of detected parameters by space
"""
auto_categories = ["buy", "sell", "enter", "exit", "protection"]
result: AllSpaceParams = defaultdict(dict)
for attr_name in dir(obj):
if not attr_name.startswith("__"): # Ignore internals, not strictly necessary.
if attr_name.startswith("__"): # Ignore internals
continue
attr = getattr(obj, attr_name)
if issubclass(attr.__class__, BaseParameter):
if (
attr_name.startswith(category + "_")
and attr.category is not None
and attr.category != category
):
raise OperationalException(
f"Inconclusive parameter name {attr_name}, category: {attr.category}."
)
if not issubclass(attr.__class__, BaseParameter):
continue
if not attr.category:
# Category auto detection
for category in auto_categories:
if attr_name.startswith(category + "_"):
attr.category = category
break
if attr.category is None:
raise OperationalException(f"Cannot determine parameter space for {attr_name}.")
if category == attr.category or (
attr_name.startswith(category + "_") and attr.category is None
):
yield attr_name, attr
if attr.category in ("all", "default") or attr.category.isidentifier() is False:
raise OperationalException(
f"'{attr.category}' is not a valid space. Parameter: {attr_name}."
)
attr.name = attr_name
result[attr.category][attr_name] = attr
return result

View File

@@ -49,9 +49,9 @@ class BaseParameter(ABC):
):
"""
Initialize hyperopt-optimizable parameter.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter field
name is prefixed with 'buy_' or 'sell_'.
:param space: The parameter space. Can be 'buy', 'sell', or a string that's also a
valid python identifier.
This parameter is optional if parameter name is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to optuna.distributions.
@@ -109,8 +109,9 @@ class NumericParameter(BaseParameter):
:param high: Upper end (inclusive) of optimization space.
Must be none of entire range is passed first parameter.
:param default: A default value.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter fieldname is prefixed with 'buy_' or 'sell_'.
:param space: The parameter space. Can be 'buy', 'sell', or a string that's also a
valid python identifier.
This parameter is optional if parameter name is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to optuna.distributions.*.
@@ -151,8 +152,9 @@ class IntParameter(NumericParameter):
:param high: Upper end (inclusive) of optimization space.
Must be none of entire range is passed first parameter.
:param default: A default value.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter fieldname is prefixed with 'buy_' or 'sell_'.
:param space: The parameter space. Can be 'buy', 'sell', or a string that's also a
valid python identifier.
This parameter is optional if parameter name is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to optuna.distributions.IntDistribution.
@@ -205,8 +207,9 @@ class RealParameter(NumericParameter):
:param high: Upper end (inclusive) of optimization space.
Must be none if entire range is passed first parameter.
:param default: A default value.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter fieldname is prefixed with 'buy_' or 'sell_'.
:param space: The parameter space. Can be 'buy', 'sell', or a string that's also a
valid python identifier.
This parameter is optional if parameter name is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to optuna.distributions.FloatDistribution.
@@ -245,8 +248,9 @@ class DecimalParameter(NumericParameter):
Must be none if entire range is passed first parameter.
:param default: A default value.
:param decimals: A number of decimals after floating point to be included in testing.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter fieldname is prefixed with 'buy_' or 'sell_'.
:param space: The parameter space. Can be 'buy', 'sell', or a string that's also a
valid python identifier.
This parameter is optional if parameter name is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to optuna's NumericParameter.
@@ -311,9 +315,9 @@ class CategoricalParameter(BaseParameter):
:param categories: Optimization space, [a, b, ...].
:param default: A default value. If not specified, first item from specified space will be
used.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter field
name is prefixed with 'buy_' or 'sell_'.
:param space: The parameter space. Can be 'buy', 'sell', or a string that's also a
valid python identifier.
This parameter is optional if parameter name is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Compatibility. Optuna's CategoricalDistribution does not
@@ -362,9 +366,9 @@ class BooleanParameter(CategoricalParameter):
It's a shortcut to `CategoricalParameter([True, False])`.
:param default: A default value. If not specified, first item from specified space will be
used.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter field
name is prefixed with 'buy_' or 'sell_'.
:param space: The parameter space. Can be 'buy', 'sell', or a string that's also a
valid python identifier.
This parameter is optional if parameter name is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to optuna.distributions.CategoricalDistribution.

View File

@@ -96,7 +96,9 @@ class SampleStrategy(IStrategy):
buy_rsi = IntParameter(low=1, high=50, default=30, space="buy", optimize=True, load=True)
sell_rsi = IntParameter(low=50, high=100, default=70, space="sell", optimize=True, load=True)
short_rsi = IntParameter(low=51, high=100, default=70, space="sell", optimize=True, load=True)
exit_short_rsi = IntParameter(low=1, high=50, default=30, space="buy", optimize=True, load=True)
exit_short_rsi = IntParameter(
low=1, high=50, default=30, space="exit", optimize=True, load=True
)
# Number of candles the strategy requires before producing valid signals
startup_candle_count: int = 200

View File

@@ -1319,10 +1319,10 @@ def test_hyperopt_list(mocker, capsys, caplog, tmp_path):
" 2/12",
" 10/12",
"Best result:",
"Buy hyperspace params",
"Sell hyperspace params",
"ROI table",
"Stoploss",
"Buy parameters",
"Sell parameters",
"ROI parameters",
"Stoploss parameters",
]
)
assert all(

View File

@@ -501,7 +501,7 @@ def test_populate_indicators(hyperopt, testdatadir) -> None:
def test_generate_optimizer(mocker, hyperopt_conf) -> None:
hyperopt_conf.update(
{
"spaces": "all",
"spaces": ["all"],
"hyperopt_min_trades": 1,
}
)
@@ -569,6 +569,8 @@ def test_generate_optimizer(mocker, hyperopt_conf) -> None:
"buy_rsi": 35,
"sell_minusdi": 0.02,
"sell_rsi": 75,
"exit_rsi": 7,
"exitaaa": 7,
"protection_cooldown_lookback": 20,
"protection_enabled": True,
"roi_t1": 60.0,
@@ -597,6 +599,12 @@ def test_generate_optimizer(mocker, hyperopt_conf) -> None:
"buy_plusdi": 0.02,
"buy_rsi": 35,
},
"exitaspace": {
"exitaaa": 7,
},
"exit": {
"exit_rsi": 7,
},
"roi": {"0": 0.12, "20.0": 0.02, "50.0": 0.01, "110.0": 0},
"protection": {
"protection_cooldown_lookback": 20,
@@ -616,7 +624,7 @@ def test_generate_optimizer(mocker, hyperopt_conf) -> None:
"max_open_trades": {"max_open_trades": 3},
},
"params_dict": optimizer_param,
"params_not_optimized": {"buy": {}, "protection": {}, "sell": {}},
"params_not_optimized": {},
"results_metrics": ANY,
"total_profit": 3.1e-08,
}
@@ -906,14 +914,43 @@ def test_simplified_interface_all_failed(mocker, hyperopt_conf, caplog) -> None:
hyperopt.hyperopter.backtesting.strategy.advise_all_indicators = MagicMock()
hyperopt.hyperopter.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
with pytest.raises(OperationalException, match=r"The 'protection' space is included into *"):
# The first one to fail raises the exception
with pytest.raises(OperationalException, match=r"The 'buy' space is included into *"):
hyperopt.hyperopter.init_spaces()
hyperopt.config["hyperopt_ignore_missing_space"] = True
caplog.clear()
hyperopt.hyperopter.init_spaces()
assert log_has_re(r"The 'protection' space is included into *", caplog)
assert hyperopt.hyperopter.protection_space == []
assert hyperopt.hyperopter.spaces["protection"] == []
def test_simplified_interface_none_selected(mocker, hyperopt_conf, caplog) -> None:
mocker.patch("freqtrade.optimize.hyperopt.hyperopt_optimizer.dump", MagicMock())
mocker.patch("freqtrade.optimize.hyperopt.hyperopt.file_dump_json")
mocker.patch(
"freqtrade.optimize.backtesting.Backtesting.load_bt_data",
MagicMock(return_value=(MagicMock(), None)),
)
mocker.patch(
"freqtrade.optimize.hyperopt.hyperopt_optimizer.get_timerange",
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13))),
)
patch_exchange(mocker)
hyperopt_conf.update(
{
"spaces": [],
}
)
hyperopt = Hyperopt(hyperopt_conf)
hyperopt.hyperopter.backtesting.strategy.advise_all_indicators = MagicMock()
hyperopt.hyperopter.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
with pytest.raises(OperationalException, match=r"No hyperopt parameters found to optimize\..*"):
hyperopt.hyperopter.init_spaces()
def test_simplified_interface_buy(mocker, hyperopt_conf, capsys) -> None:

View File

@@ -296,14 +296,14 @@ def test_show_epoch_details(capsys):
HyperoptTools.show_epoch_details(test_result, 5, False, no_header=True)
captured = capsys.readouterr()
assert "# Trailing stop:" in captured.out
assert "# Trailing stop parameters:" in captured.out
# re.match(r"Pairs for .*", captured.out)
assert re.search(r"^\s+trailing_stop = True$", captured.out, re.MULTILINE)
assert re.search(r"^\s+trailing_stop_positive = 0.02$", captured.out, re.MULTILINE)
assert re.search(r"^\s+trailing_stop_positive_offset = 0.04$", captured.out, re.MULTILINE)
assert re.search(r"^\s+trailing_only_offset_is_reached = True$", captured.out, re.MULTILINE)
assert "# ROI table:" in captured.out
assert "# ROI parameters:" in captured.out
assert re.search(r"^\s+minimal_roi = \{$", captured.out, re.MULTILINE)
assert re.search(r"^\s+\"90\"\:\s0.14,\s*$", captured.out, re.MULTILINE)

View File

@@ -35,6 +35,9 @@ class HyperoptableStrategy(StrategyTestV3):
sell_minusdi = DecimalParameter(
low=0, high=1, default=0.5001, decimals=3, space="sell", load=False
)
exitaaa = IntParameter(low=0, high=10, default=5, space="exitaspace")
exit_rsi = IntParameter(low=0, high=10, default=5)
protection_enabled = BooleanParameter(default=True)
protection_cooldown_lookback = IntParameter([0, 50], default=30)

View File

@@ -16,7 +16,7 @@ from freqtrade.enums import ExitCheckTuple, ExitType, SignalDirection
from freqtrade.exceptions import OperationalException, StrategyError
from freqtrade.persistence import PairLocks, Trade
from freqtrade.resolvers import StrategyResolver
from freqtrade.strategy.hyper import detect_parameters
from freqtrade.strategy.hyper import detect_all_parameters
from freqtrade.strategy.parameters import (
IntParameter,
)
@@ -928,8 +928,7 @@ def test_auto_hyperopt_interface(default_conf):
PairLocks.timeframe = default_conf["timeframe"]
strategy = StrategyResolver.load_strategy(default_conf)
strategy.ft_bot_start()
with pytest.raises(OperationalException):
next(strategy.enumerate_parameters("deadBeef"))
assert list(strategy.enumerate_parameters("deadBeef")) == []
assert strategy.buy_rsi.value == strategy.buy_params["buy_rsi"]
# PlusDI is NOT in the buy-params, so default should be used
@@ -940,20 +939,52 @@ def test_auto_hyperopt_interface(default_conf):
# Parameter is disabled - so value from sell_param dict will NOT be used.
assert strategy.sell_minusdi.value == 0.5
all_params = strategy.detect_all_parameters()
all_params = detect_all_parameters(strategy.__class__)
assert isinstance(all_params, dict)
# Only one buy param at class level
assert len(all_params["buy"]) == 1
# Running detect params at instance level reveals both parameters.
assert len(list(detect_parameters(strategy, "buy"))) == 2
assert len(all_params["sell"]) == 2
# Number of Hyperoptable parameters
assert all_params["count"] == 5
params_inst = detect_all_parameters(strategy)
assert len(params_inst["buy"]) == 2
assert len(params_inst["sell"]) == 2
strategy.__class__.sell_rsi = IntParameter([0, 10], default=5, space="buy")
with pytest.raises(OperationalException, match=r"Inconclusive parameter.*"):
[x for x in detect_parameters(strategy, "sell")]
spaces = detect_all_parameters(strategy.__class__)
assert "buy" in spaces
assert spaces["buy"]["sell_rsi"] == strategy.sell_rsi
del strategy.__class__.sell_rsi
strategy.__class__.exit22_rsi = IntParameter([0, 10], default=5)
with pytest.raises(
OperationalException, match=r"Cannot determine parameter space for exit22_rsi\."
):
detect_all_parameters(strategy.__class__)
# Invalid parameter space
strategy.__class__.exit22_rsi = IntParameter([0, 10], default=5, space="all")
with pytest.raises(
OperationalException, match=r"'all' is not a valid space\. Parameter: exit22_rsi\."
):
detect_all_parameters(strategy.__class__)
strategy.__class__.exit22_rsi = IntParameter([0, 10], default=5, space="hello:world:22")
with pytest.raises(
OperationalException,
match=r"'hello:world:22' is not a valid space\. Parameter: exit22_rsi\.",
):
detect_all_parameters(strategy.__class__)
del strategy.__class__.exit22_rsi
# Valid exit parameter
strategy.__class__.exit_rsi = IntParameter([0, 10], default=5)
strategy.__class__.enter_rsi = IntParameter([0, 10], default=5)
spaces = detect_all_parameters(strategy.__class__)
assert "exit" in spaces
assert "enter" in spaces
del strategy.__class__.exit_rsi
del strategy.__class__.enter_rsi
def test_auto_hyperopt_interface_loadparams(default_conf, mocker, caplog):