+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ This page explains some advanced Hyperopt topics that may require higher +coding skills and Python knowledge than creation of an ordinal hyperoptimization +class.
+To use a custom loss function class, make sure that the function hyperopt_loss_function is defined in your custom hyperopt loss class.
+For the sample below, you then need to add the command line parameter --hyperopt-loss SuperDuperHyperOptLoss to your hyperopt call so this function is being used.
A sample of this can be found below, which is identical to the Default Hyperopt loss implementation. A full sample can be found in userdata/hyperopts.
+``` python +from datetime import datetime +from typing import Dict
+from pandas import DataFrame
+from freqtrade.optimize.hyperopt import IHyperOptLoss
+TARGET_TRADES = 600 +EXPECTED_MAX_PROFIT = 3.0 +MAX_ACCEPTED_TRADE_DURATION = 300
+class SuperDuperHyperOptLoss(IHyperOptLoss): + """ + Defines the default loss function for hyperopt + """
+@staticmethod
+def hyperopt_loss_function(results: DataFrame, trade_count: int,
+ min_date: datetime, max_date: datetime,
+ config: Dict, processed: Dict[str, DataFrame],
+ *args, **kwargs) -> float:
+ """
+ Objective function, returns smaller number for better results
+ This is the legacy algorithm (used until now in freqtrade).
+ Weights are distributed as follows:
+ * 0.4 to trade duration
+ * 0.25: Avoiding trade loss
+ * 1.0 to total profit, compared to the expected value (`EXPECTED_MAX_PROFIT`) defined above
+ """
+ total_profit = results['profit_ratio'].sum()
+ trade_duration = results['trade_duration'].mean()
+
+ trade_loss = 1 - 0.25 * exp(-(trade_count - TARGET_TRADES) ** 2 / 10 ** 5.8)
+ profit_loss = max(0, 1 - total_profit / EXPECTED_MAX_PROFIT)
+ duration_loss = 0.4 * min(trade_duration / MAX_ACCEPTED_TRADE_DURATION, 1)
+ result = trade_loss + profit_loss + duration_loss
+ return result
+```
+Currently, the arguments are:
+results: DataFrame containing the result--export trades):pair, profit_ratio, profit_abs, open_date, open_rate, fee_open, close_date, close_rate, fee_close, amount, trade_duration, is_open, sell_reason, stake_amount, min_rate, max_rate, stop_loss_ratio, stop_loss_abstrade_count: Amount of trades (identical to len(results))min_date: Start date of the timerange usedmin_date: End date of the timerange usedconfig: Config object used (Note: Not all strategy-related parameters will be updated here if they are part of a hyperopt space).processed: Dict of Dataframes with the pair as keys containing the data used for backtesting.This function needs to return a floating point number (float). Smaller numbers will be interpreted as better results. The parameters and balancing for this is up to you.
Note
+This function is called once per iteration - so please make sure to have this as optimized as possible to not slow hyperopt down unnecessarily.
+Note
+Please keep the arguments *args and **kwargs in the interface to allow us to extend this interface later.
To override a pre-defined space (roi_space, generate_roi_table, stoploss_space, trailing_space), define a nested class called Hyperopt and define the required spaces as follows:
python
+class MyAwesomeStrategy(IStrategy):
+ class HyperOpt:
+ # Define a custom stoploss space.
+ def stoploss_space(self):
+ return [SKDecimal(-0.05, -0.01, decimals=3, name='stoploss')]
For the additional spaces, scikit-optimize (in combination with Freqtrade) provides the following space types:
+Categorical - Pick from a list of categories (e.g. Categorical(['a', 'b', 'c'], name="cat"))Integer - Pick from a range of whole numbers (e.g. Integer(1, 10, name='rsi'))SKDecimal - Pick from a range of decimal numbers with limited precision (e.g. SKDecimal(0.1, 0.5, decimals=3, name='adx')). Available only with freqtrade.Real - Pick from a range of decimal numbers with full precision (e.g. Real(0.1, 0.5, name='adx')You can import all of these from freqtrade.optimize.space, although Categorical, Integer and Real are only aliases for their corresponding scikit-optimize Spaces. SKDecimal is provided by freqtrade for faster optimizations.
python
+from freqtrade.optimize.space import Categorical, Dimension, Integer, SKDecimal, Real # noqa
SKDecimal vs. Real
+We recommend to use SKDecimal instead of the Real space in almost all cases. While the Real space provides full accuracy (up to ~16 decimal places) - this precision is rarely needed, and leads to unnecessary long hyperopt times.
Assuming the definition of a rather small space (SKDecimal(0.10, 0.15, decimals=2, name='xxx')) - SKDecimal will have 5 possibilities ([0.10, 0.11, 0.12, 0.13, 0.14, 0.15]).
A corresponding real space Real(0.10, 0.15 name='xxx') on the other hand has an almost unlimited number of possibilities ([0.10, 0.010000000001, 0.010000000002, ... 0.014999999999, 0.01500000000]).
This Section explains the configuration of an explicit Hyperopt file (separate to the strategy).
+Deprecated / legacy mode
+Since the 2021.4 release you no longer have to write a separate hyperopt class, but all strategies can be hyperopted. +Please read the main hyperopt page for more details.
+Configuring an explicit hyperopt file is similar to writing your own strategy, and many tasks will be similar.
+About this page
+For this page, we will be using a fictional strategy called AwesomeStrategy - which will be optimized using the AwesomeHyperopt class.
The simplest way to get started is to use the following command, which will create a new hyperopt file from a template, which will be located under user_data/hyperopts/AwesomeHyperopt.py.
Let assume you want a hyperopt file AwesomeHyperopt.py:
bash
+freqtrade new-hyperopt --hyperopt AwesomeHyperopt
Checklist on all tasks / possibilities in hyperopt
+Depending on the space you want to optimize, only some of the below are required:
+buy_strategy_generator - for buy signal optimizationindicator_space - for buy signal optimizationsell_strategy_generator - for sell signal optimizationsell_indicator_space - for sell signal optimizationNote
+populate_indicators needs to create all indicators any of thee spaces may use, otherwise hyperopt will not work.
Optional in hyperopt - can also be loaded from a strategy (recommended):
+populate_indicators - fallback to create indicatorspopulate_buy_trend - fallback if not optimizing for buy space. should come from strategypopulate_sell_trend - fallback if not optimizing for sell space. should come from strategyNote
+You always have to provide a strategy to Hyperopt, even if your custom Hyperopt class contains all methods.
+Assuming the optional methods are not in your hyperopt file, please use --strategy AweSomeStrategy which contains these methods so hyperopt can use these methods instead.
Rarely you may also need to override:
+roi_space - for custom ROI optimization (if you need the ranges for the ROI parameters in the optimization hyperspace that differ from default)generate_roi_table - for custom ROI optimization (if you need the ranges for the values in the ROI table that differ from default or the number of entries (steps) in the ROI table which differs from the default 4 steps)stoploss_space - for custom stoploss optimization (if you need the range for the stoploss parameter in the optimization hyperspace that differs from default)trailing_space - for custom trailing stop optimization (if you need the ranges for the trailing stop parameters in the optimization hyperspace that differ from default)Let's say you are curious: should you use MACD crossings or lower Bollinger +Bands to trigger your buys. And you also wonder should you use RSI or ADX to +help with those buy decisions. If you decide to use RSI or ADX, which values +should I use for them? So let's use hyperparameter optimization to solve this +mystery.
+We will start by defining a search space:
+python
+ def indicator_space() -> List[Dimension]:
+ """
+ Define your Hyperopt space for searching strategy parameters
+ """
+ return [
+ Integer(20, 40, name='adx-value'),
+ Integer(20, 40, name='rsi-value'),
+ Categorical([True, False], name='adx-enabled'),
+ Categorical([True, False], name='rsi-enabled'),
+ Categorical(['bb_lower', 'macd_cross_signal'], name='trigger')
+ ]
Above definition says: I have five parameters I want you to randomly combine
+to find the best combination. Two of them are integer values (adx-value and rsi-value) and I want you test in the range of values 20 to 40.
+Then we have three category variables. First two are either True or False.
+We use these to either enable or disable the ADX and RSI guards.
+The last one we call trigger and use it to decide which buy trigger we want to use.
So let's write the buy strategy generator using these values:
+```python + @staticmethod + def buy_strategy_generator(params: Dict[str, Any]) -> Callable: + """ + Define the buy strategy parameters to be used by Hyperopt. + """ + def populate_buy_trend(dataframe: DataFrame, metadata: dict) -> DataFrame: + conditions = [] + # GUARDS AND TRENDS + if 'adx-enabled' in params and params['adx-enabled']: + conditions.append(dataframe['adx'] > params['adx-value']) + if 'rsi-enabled' in params and params['rsi-enabled']: + conditions.append(dataframe['rsi'] < params['rsi-value'])
+ # TRIGGERS
+ if 'trigger' in params:
+ if params['trigger'] == 'bb_lower':
+ conditions.append(dataframe['close'] < dataframe['bb_lowerband'])
+ if params['trigger'] == 'macd_cross_signal':
+ conditions.append(qtpylib.crossed_above(
+ dataframe['macd'], dataframe['macdsignal']
+ ))
+
+ # Check that volume is not 0
+ conditions.append(dataframe['volume'] > 0)
+
+ if conditions:
+ dataframe.loc[
+ reduce(lambda x, y: x & y, conditions),
+ 'buy'] = 1
+
+ return dataframe
+
+ return populate_buy_trend
+```
+Hyperopt will now call populate_buy_trend() many times (epochs) with different value combinations.
+It will use the given historical data and make buys based on the buy signals generated with the above function.
+Based on the results, hyperopt will tell you which parameter combination produced the best results (based on the configured loss function).
Note
+The above setup expects to find ADX, RSI and Bollinger Bands in the populated indicators.
+When you want to test an indicator that isn't used by the bot currently, remember to
+add it to the populate_indicators() method in your strategy or hyperopt file.
Similar to the buy-signal above, sell-signals can also be optimized. +Place the corresponding settings into the following methods
+sell_indicator_space() - the parameters hyperopt shall be optimizing.sell_strategy_generator() - populate the nested method populate_sell_trend() to apply the parameters.The configuration and rules are the same than for buy signals.
+To avoid naming collisions in the search-space, please prefix all sell-spaces with sell-.
Once you have updated your hyperopt configuration you can run it. +Because hyperopt tries a lot of combinations to find the best parameters it will take time to get a good result. More time usually results in better results.
+We strongly recommend to use screen or tmux to prevent any connection loss.
bash
+freqtrade hyperopt --config config.json --hyperopt <hyperoptname> --hyperopt-loss <hyperoptlossname> --strategy <strategyname> -e 500 --spaces all
Use <hyperoptname> as the name of the custom hyperopt used.
The -e option will set how many evaluations hyperopt will do. Since hyperopt uses Bayesian search, running too many epochs at once may not produce greater results. Experience has shown that best results are usually not improving much after 500-1000 epochs.
+Doing multiple runs (executions) with a few 1000 epochs and different random state will most likely produce different results.
The --spaces all option determines that all possible parameters should be optimized. Possibilities are listed below.
Note
+Hyperopt will store hyperopt results with the timestamp of the hyperopt start time.
+Reading commands (hyperopt-list, hyperopt-show) can use --hyperopt-filename <filename> to read and display older hyperopt results.
+You can find a list of filenames with ls -l user_data/hyperopt_results/.
Hyperopt can reuse populate_indicators, populate_buy_trend, populate_sell_trend from your strategy, assuming these methods are not in your custom hyperopt file, and a strategy is provided.
bash
+freqtrade hyperopt --hyperopt AwesomeHyperopt --hyperopt-loss SharpeHyperOptLossDaily --strategy AwesomeStrategy
Once Hyperopt is completed you can use the result to create a new strategy. +Given the following result from hyperopt:
+``` +Best result:
+44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722%). Avg duration 180.4 mins. Objective: 1.94367
+Buy hyperspace params: +{ 'adx-value': 44, + 'rsi-value': 29, + 'adx-enabled': False, + 'rsi-enabled': True, + 'trigger': 'bb_lower'} +```
+You should understand this result like:
+bb_lower.adx-enabled: False)rsi-enabled: True and the best value is 29.0 (rsi-value: 29.0)You have to look inside your strategy file into buy_strategy_generator()
+method, what those values match to.
So for example you had rsi-value: 29.0 so we would look at rsi-block, that translates to the following code block:
python
+(dataframe['rsi'] < 29.0)
Translating your whole hyperopt result as the new buy-signal would then look like:
+python
+def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
+ dataframe.loc[
+ (
+ (dataframe['rsi'] < 29.0) & # rsi-value
+ dataframe['close'] < dataframe['bb_lowerband'] # trigger
+ ),
+ 'buy'] = 1
+ return dataframe
Once the optimized parameters and conditions have been implemented into your strategy, you should backtest the strategy to make sure everything is working as expected.
+To achieve same results (number of trades, their durations, profit, etc.) than during Hyperopt, please use same configuration and parameters (timerange, timeframe, ...) used for hyperopt --dmmp/--disable-max-market-positions and --eps/--enable-position-stacking for Backtesting.
Should results don't match, please double-check to make sure you transferred all conditions correctly.
+Pay special care to the stoploss (and trailing stoploss) parameters, as these are often set in configuration files, which override changes to the strategy.
+You should also carefully review the log of your backtest to ensure that there were no parameters inadvertently set by the configuration (like stoploss or trailing_stop).
Hyperopt classes provide access to the Strategy via the strategy class attribute.
+This can be a great way to reduce code duplication if used correctly, but will also complicate usage for inexperienced users.
``` python +from pandas import DataFrame +from freqtrade.strategy.interface import IStrategy +import freqtrade.vendor.qtpylib.indicators as qtpylib
+class MyAwesomeStrategy(IStrategy):
+buy_params = {
+ 'rsi-value': 30,
+ 'adx-value': 35,
+}
+
+def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
+ return self.buy_strategy_generator(self.buy_params, dataframe, metadata)
+
+@staticmethod
+def buy_strategy_generator(params, dataframe: DataFrame, metadata: dict) -> DataFrame:
+ dataframe.loc[
+ (
+ qtpylib.crossed_above(dataframe['rsi'], params['rsi-value']) &
+ dataframe['adx'] > params['adx-value']) &
+ dataframe['volume'] > 0
+ )
+ , 'buy'] = 1
+ return dataframe
+class MyAwesomeHyperOpt(IHyperOpt): + ... + @staticmethod + def buy_strategy_generator(params: Dict[str, Any]) -> Callable: + """ + Define the buy strategy parameters to be used by Hyperopt. + """ + def populate_buy_trend(dataframe: DataFrame, metadata: dict) -> DataFrame: + # Call strategy's buy strategy generator + return self.StrategyClass.buy_strategy_generator(params, dataframe, metadata)
+ return populate_buy_trend
+```
+ + + + + + + +This page explains some advanced tasks and configuration options that can be performed after the bot installation and may be uselful in some environments.
+If you do not know what things mentioned here mean, you probably do not need it.
+This section will show you how to run multiple bots at the same time, on the same machine.
+In order to keep track of your trades, profits, etc., freqtrade is using a SQLite database where it stores various types of information such as the trades you performed in the past and the current position(s) you are holding at any time. This allows you to keep track of your profits, but most importantly, keep track of ongoing activity if the bot process would be restarted or would be terminated unexpectedly.
+Freqtrade will, by default, use separate database files for dry-run and live bots (this assumes no database-url is given in either configuration nor via command line argument).
+For live trading mode, the default database will be tradesv3.sqlite and for dry-run it will be tradesv3.dryrun.sqlite.
The optional argument to the trade command used to specify the path of these files is --db-url, which requires a valid SQLAlchemy url.
+So when you are starting a bot with only the config and strategy arguments in dry-run mode, the following 2 commands would have the same outcome.
``` bash +freqtrade trade -c MyConfig.json -s MyStrategy
+freqtrade trade -c MyConfig.json -s MyStrategy --db-url sqlite:///tradesv3.dryrun.sqlite +```
+It means that if you are running the trade command in two different terminals, for example to test your strategy both for trades in USDT and in another instance for trades in BTC, you will have to run them with different databases.
+If you specify the URL of a database which does not exist, freqtrade will create one with the name you specified. So to test your custom strategy with BTC and USDT stake currencies, you could use the following commands (in 2 separate terminals):
+``` bash
+freqtrade trade -c MyConfigBTC.json -s MyCustomStrategy --db-url sqlite:///user_data/tradesBTC.dryrun.sqlite
+freqtrade trade -c MyConfigUSDT.json -s MyCustomStrategy --db-url sqlite:///user_data/tradesUSDT.dryrun.sqlite +```
+Conversely, if you wish to do the same thing in production mode, you will also have to create at least one new database (in addition to the default one) and specify the path to the "live" databases, for example:
+``` bash
+freqtrade trade -c MyConfigBTC.json -s MyCustomStrategy --db-url sqlite:///user_data/tradesBTC.live.sqlite
+freqtrade trade -c MyConfigUSDT.json -s MyCustomStrategy --db-url sqlite:///user_data/tradesUSDT.live.sqlite +```
+For more information regarding usage of the sqlite databases, for example to manually enter or remove trades, please refer to the SQL Cheatsheet.
+Copy the freqtrade.service file to your systemd user directory (usually ~/.config/systemd/user) and update WorkingDirectory and ExecStart to match your setup.
Note
+Certain systems (like Raspbian) don't load service unit files from the user directory. In this case, copy freqtrade.service into /etc/systemd/user/ (requires superuser permissions).
After that you can start the daemon with:
+bash
+systemctl --user start freqtrade
For this to be persistent (run when user is logged out) you'll need to enable linger for your freqtrade user.
bash
+sudo loginctl enable-linger "$USER"
If you run the bot as a service, you can use systemd service manager as a software watchdog monitoring freqtrade bot
+state and restarting it in the case of failures. If the internals.sd_notify parameter is set to true in the
+configuration or the --sd-notify command line option is used, the bot will send keep-alive ping messages to systemd
+using the sd_notify (systemd notifications) protocol and will also tell systemd its current state (Running or Stopped)
+when it changes.
The freqtrade.service.watchdog file contains an example of the service unit configuration file which uses systemd
+as the watchdog.
Note
+The sd_notify communication between the bot and the systemd service manager will not work if the bot runs in a Docker container.
+On many Linux systems the bot can be configured to send its log messages to syslog or journald system services. Logging to a remote syslog server is also available on Windows. The special values for the --logfile command line option can be used for this.
To send Freqtrade log messages to a local or remote syslog service use the --logfile command line option with the value in the following format:
--logfile syslog:<syslog_address> -- send log messages to syslog service using the <syslog_address> as the syslog address.The syslog address can be either a Unix domain socket (socket filename) or a UDP socket specification, consisting of IP address and UDP port, separated by the : character.
So, the following are the examples of possible usages:
+--logfile syslog:/dev/log -- log to syslog (rsyslog) using the /dev/log socket, suitable for most systems.--logfile syslog -- same as above, the shortcut for /dev/log.--logfile syslog:/var/run/syslog -- log to syslog (rsyslog) using the /var/run/syslog socket. Use this on MacOS.--logfile syslog:localhost:514 -- log to local syslog using UDP socket, if it listens on port 514.--logfile syslog:<ip>:514 -- log to remote syslog at IP address and port 514. This may be used on Windows for remote logging to an external syslog server.Log messages are send to syslog with the user facility. So you can see them with the following commands:
tail -f /var/log/user, or On many systems syslog (rsyslog) fetches data from journald (and vice versa), so both --logfile syslog or --logfile journald can be used and the messages be viewed with both journalctl and a syslog viewer utility. You can combine this in any way which suites you better.
For rsyslog the messages from the bot can be redirected into a separate dedicated log file. To achieve this, add
+if $programname startswith "freqtrade" then -/var/log/freqtrade.log
+to one of the rsyslog configuration files, for example at the end of the /etc/rsyslog.d/50-default.conf.
For syslog (rsyslog), the reduction mode can be switched on. This will reduce the number of repeating messages. For instance, multiple bot Heartbeat messages will be reduced to a single message when nothing else happens with the bot. To achieve this, set in /etc/rsyslog.conf:
+```
$RepeatedMsgReduction on +```
+This needs the systemd python package installed as the dependency, which is not available on Windows. Hence, the whole journald logging functionality is not available for a bot running on Windows.
To send Freqtrade log messages to journald system service use the --logfile command line option with the value in the following format:
--logfile journald -- send log messages to journald.Log messages are send to journald with the user facility. So you can see them with the following commands:
journalctl -f -- shows Freqtrade log messages sent to journald along with other log messages fetched by journald.journalctl -f -u freqtrade.service -- this command can be used when the bot is run as a systemd service.There are many other options in the journalctl utility to filter the messages, see manual pages for this utility.
On many systems syslog (rsyslog) fetches data from journald (and vice versa), so both --logfile syslog or --logfile journald can be used and the messages be viewed with both journalctl and a syslog viewer utility. You can combine this in any way which suites you better.
\n {translation(\"search.result.term.missing\")}: {...missing}\n
\n }\n \n