diff --git a/docs/freqai-configuration.md b/docs/freqai-configuration.md index 6e2ed8379..090fa8415 100644 --- a/docs/freqai-configuration.md +++ b/docs/freqai-configuration.md @@ -160,7 +160,7 @@ Below are the values you can expect to include/use inside a typical strategy dat |------------|-------------| | `df['&*']` | Any dataframe column prepended with `&` in `set_freqai_targets()` is treated as a training target (label) inside FreqAI (typically following the naming convention `&-s*`). For example, to predict the close price 40 candles into the future, you would set `df['&-s_close'] = df['close'].shift(-self.freqai_info["feature_parameters"]["label_period_candles"])` with `"label_period_candles": 40` in the config. FreqAI makes the predictions and gives them back under the same key (`df['&-s_close']`) to be used in `populate_entry/exit_trend()`.
**Datatype:** Depends on the output of the model. | `df['&*_std/mean']` | Standard deviation and mean values of the defined labels during training (or live tracking with `fit_live_predictions_candles`). Commonly used to understand the rarity of a prediction (use the z-score as shown in `templates/FreqaiExampleStrategy.py` and explained [here](#creating-a-dynamic-target-threshold) to evaluate how often a particular prediction was observed during training or historically with `fit_live_predictions_candles`).
**Datatype:** Float. -| `df['do_predict']` | Indication of an outlier data point. The return value is integer between -2 and 2, which lets you know if the prediction is trustworthy or not. `do_predict==1` means that the prediction is trustworthy. If the Dissimilarity Index (DI, see details [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di)) of the input data point is above the threshold defined in the config, FreqAI will subtract 1 from `do_predict`, resulting in `do_predict==0`. If `use_SVM_to_remove_outliers()` is active, the Support Vector Machine (SVM, see details [here](freqai-feature-engineering.md#identifying-outliers-using-a-support-vector-machine-svm)) may also detect outliers in training and prediction data. In this case, the SVM will also subtract 1 from `do_predict`. If the input data point was considered an outlier by the SVM but not by the DI, or vice versa, the result will be `do_predict==0`. If both the DI and the SVM considers the input data point to be an outlier, the result will be `do_predict==-1`. As with the SVM, if `use_DBSCAN_to_remove_outliers` is active, DBSCAN (see details [here](freqai-feature-engineering.md#identifying-outliers-with-dbscan)) may also detect outliers and subtract 1 from `do_predict`. Hence, if both the SVM and DBSCAN are active and identify a datapoint that was above the DI threshold as an outlier, the result will be `do_predict==-2`. A particular case is when `do_predict == 2`, which means that the model has expired due to exceeding `expired_hours`.
**Datatype:** Integer between -2 and 2. +| `df['do_predict']` | Indication of an outlier data point. The return value is integer between -2 and 2, which lets you know if the prediction is trustworthy or not. `do_predict==1` means that the prediction is trustworthy. If the Dissimilarity Index (DI, see details [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di)) of the input data point is above the threshold defined in the config, FreqAI will subtract 1 from `do_predict`, resulting in `do_predict==0`. If `use_SVM_to_remove_outliers` is active, the Support Vector Machine (SVM, see details [here](freqai-feature-engineering.md#identifying-outliers-using-a-support-vector-machine-svm)) may also detect outliers in training and prediction data. In this case, the SVM will also subtract 1 from `do_predict`. If the input data point was considered an outlier by the SVM but not by the DI, or vice versa, the result will be `do_predict==0`. If both the DI and the SVM considers the input data point to be an outlier, the result will be `do_predict==-1`. As with the SVM, if `use_DBSCAN_to_remove_outliers` is active, DBSCAN (see details [here](freqai-feature-engineering.md#identifying-outliers-with-dbscan)) may also detect outliers and subtract 1 from `do_predict`. Hence, if both the SVM and DBSCAN are active and identify a datapoint that was above the DI threshold as an outlier, the result will be `do_predict==-2`. A particular case is when `do_predict == 2`, which means that the model has expired due to exceeding `expired_hours`.
**Datatype:** Integer between -2 and 2. | `df['DI_values']` | Dissimilarity Index (DI) values are proxies for the level of confidence FreqAI has in the prediction. A lower DI means the prediction is close to the training data, i.e., higher prediction confidence. See details about the DI [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di).
**Datatype:** Float. | `df['%*']` | Any dataframe column prepended with `%` in `feature_engineering_*()` is treated as a training feature. For example, you can include the RSI in the training feature set (similar to in `templates/FreqaiExampleStrategy.py`) by setting `df['%-rsi']`. See more details on how this is done [here](freqai-feature-engineering.md).
**Note:** Since the number of features prepended with `%` can multiply very quickly (10s of thousands of features are easily engineered using the multiplictative functionality of, e.g., `include_shifted_candles` and `include_timeframes` as described in the [parameter table](freqai-parameter-table.md)), these features are removed from the dataframe that is returned from FreqAI to the strategy. To keep a particular type of feature for plotting purposes, you would prepend it with `%%`.
**Datatype:** Depends on the output of the model. diff --git a/docs/strategy_migration.md b/docs/strategy_migration.md index 353da0ccb..d00349d1d 100644 --- a/docs/strategy_migration.md +++ b/docs/strategy_migration.md @@ -800,8 +800,8 @@ class MyCoolFreqaiModel(BaseRegressionModel): if self.freqai_info.get("DI_threshold", 0) > 0: dk.DI_values = dk.feature_pipeline["di"].di_values else: - dk.DI_values = np.zeros(len(outliers.index)) - dk.do_predict = outliers.to_numpy() + dk.DI_values = np.zeros(outliers.shape[0]) + dk.do_predict = outliers # ... your custom code return (pred_df, dk.do_predict) diff --git a/freqtrade/freqai/base_models/BaseClassifierModel.py b/freqtrade/freqai/base_models/BaseClassifierModel.py index f35b07e66..42b5c1a0e 100644 --- a/freqtrade/freqai/base_models/BaseClassifierModel.py +++ b/freqtrade/freqai/base_models/BaseClassifierModel.py @@ -120,7 +120,7 @@ class BaseClassifierModel(IFreqaiModel): if dk.feature_pipeline["di"]: dk.DI_values = dk.feature_pipeline["di"].di_values else: - dk.DI_values = np.zeros(len(outliers.index)) - dk.do_predict = outliers.to_numpy() + dk.DI_values = np.zeros(outliers.shape[0]) + dk.do_predict = outliers return (pred_df, dk.do_predict) diff --git a/freqtrade/freqai/base_models/BasePyTorchClassifier.py b/freqtrade/freqai/base_models/BasePyTorchClassifier.py index c47c5069a..4780af818 100644 --- a/freqtrade/freqai/base_models/BasePyTorchClassifier.py +++ b/freqtrade/freqai/base_models/BasePyTorchClassifier.py @@ -94,8 +94,8 @@ class BasePyTorchClassifier(BasePyTorchModel): if dk.feature_pipeline["di"]: dk.DI_values = dk.feature_pipeline["di"].di_values else: - dk.DI_values = np.zeros(len(outliers.index)) - dk.do_predict = outliers.to_numpy() + dk.DI_values = np.zeros(outliers.shape[0]) + dk.do_predict = outliers return (pred_df, dk.do_predict) diff --git a/freqtrade/freqai/base_models/BasePyTorchRegressor.py b/freqtrade/freqai/base_models/BasePyTorchRegressor.py index 325743134..83fea4ef9 100644 --- a/freqtrade/freqai/base_models/BasePyTorchRegressor.py +++ b/freqtrade/freqai/base_models/BasePyTorchRegressor.py @@ -55,8 +55,8 @@ class BasePyTorchRegressor(BasePyTorchModel): if dk.feature_pipeline["di"]: dk.DI_values = dk.feature_pipeline["di"].di_values else: - dk.DI_values = np.zeros(len(outliers.index)) - dk.do_predict = outliers.to_numpy() + dk.DI_values = np.zeros(outliers.shape[0]) + dk.do_predict = outliers return (pred_df, dk.do_predict) def train( diff --git a/freqtrade/freqai/base_models/BaseRegressionModel.py b/freqtrade/freqai/base_models/BaseRegressionModel.py index 2e07d3fb7..179e4be87 100644 --- a/freqtrade/freqai/base_models/BaseRegressionModel.py +++ b/freqtrade/freqai/base_models/BaseRegressionModel.py @@ -114,7 +114,7 @@ class BaseRegressionModel(IFreqaiModel): if dk.feature_pipeline["di"]: dk.DI_values = dk.feature_pipeline["di"].di_values else: - dk.DI_values = np.zeros(len(outliers.index)) - dk.do_predict = outliers.to_numpy() + dk.DI_values = np.zeros(outliers.shape[0]) + dk.do_predict = outliers return (pred_df, dk.do_predict) diff --git a/freqtrade/freqai/freqai_interface.py b/freqtrade/freqai/freqai_interface.py index 4ca5467b6..9fe8bd194 100644 --- a/freqtrade/freqai/freqai_interface.py +++ b/freqtrade/freqai/freqai_interface.py @@ -1012,6 +1012,6 @@ class IFreqaiModel(ABC): if self.freqai_info.get("DI_threshold", 0) > 0: dk.DI_values = dk.feature_pipeline["di"].di_values else: - dk.DI_values = np.zeros(len(outliers.index)) - dk.do_predict = outliers.to_numpy() + dk.DI_values = np.zeros(outliers.shape[0]) + dk.do_predict = outliers return diff --git a/freqtrade/freqai/prediction_models/PyTorchTransformerRegressor.py b/freqtrade/freqai/prediction_models/PyTorchTransformerRegressor.py index bf78488ff..a76bab05c 100644 --- a/freqtrade/freqai/prediction_models/PyTorchTransformerRegressor.py +++ b/freqtrade/freqai/prediction_models/PyTorchTransformerRegressor.py @@ -136,8 +136,8 @@ class PyTorchTransformerRegressor(BasePyTorchRegressor): if self.freqai_info.get("DI_threshold", 0) > 0: dk.DI_values = dk.feature_pipeline["di"].di_values else: - dk.DI_values = np.zeros(len(outliers.index)) - dk.do_predict = outliers.to_numpy() + dk.DI_values = np.zeros(outliers.shape[0]) + dk.do_predict = outliers if x.shape[1] > 1: zeros_df = pd.DataFrame(np.zeros((x.shape[1] - len(pred_df), len(pred_df.columns))), diff --git a/requirements-freqai.txt b/requirements-freqai.txt index 2eacbaffb..e8e3b9334 100644 --- a/requirements-freqai.txt +++ b/requirements-freqai.txt @@ -9,4 +9,4 @@ catboost==1.2; 'arm' not in platform_machine lightgbm==3.3.5 xgboost==1.7.6 tensorboard==2.13.0 -datasieve==0.1.5 +datasieve==0.1.6