CTYPTO/freqtrade
1
0
Fork 0
mirror of https://github.com/freqtrade/freqtrade.git synced 2026-01-20 05:50:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #12649 from freqtrade/maint/remove_catboost

Browse Source 
Remove catboost dependency
This commit is contained in:
Matthias
2025-12-23 06:32:30 +01:00
committed by GitHub
parent 0feec257e3 d2c4bd1b50
commit 9baa6eebbe
15 changed files with 26 additions and 303 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/deprecated.md
View File

@@ -135,3 +135,13 @@ you can verify this with `freqtrade list-data --exchange <yourexchange> --show`.
Additional arguments to the above commands may be necessary, like configuration files or explicit user_data if they deviate from the default.
**Hyperliquid** is a special case now - which will no longer require 1h mark data - but will use regular candles instead (this data never existed and is identical to 1h futures candles). As we don't support download-data for hyperliquid (they don't provide historic data) - there won't be actions necessary for hyperliquid users.
## Catboost models in freqAI
CatBoost models have been removed with version 2025.12 and are no longer actively supported.
If you have existing bots using CatBoost models, you can still use them in your custom models by copy/pasting them from the git history (as linked below) and installing the Catboost library manually.
We do however recommend switching to other supported model libraries like LightGBM or XGBoost for better support and future compatibility.
* [CatboostRegressor](https://github.com/freqtrade/freqtrade/blob/c6f3b0081927e161a16b116cc47fb663f7831d30/freqtrade/freqai/prediction_models/CatboostRegressor.py)
* [CatboostClassifier](https://github.com/freqtrade/freqtrade/blob/c6f3b0081927e161a16b116cc47fb663f7831d30/freqtrade/freqai/prediction_models/CatboostClassifier.py)
* [CatboostClassifierMultiTarget](https://github.com/freqtrade/freqtrade/blob/c6f3b0081927e161a16b116cc47fb663f7831d30/freqtrade/freqai/prediction_models/CatboostClassifierMultiTarget.py)

12
docs/freqai-configuration.md
View File

@@ -200,15 +200,15 @@ If this value is set, FreqAI will initially use the predictions from the trainin
## Using different prediction models
FreqAI has multiple example prediction model libraries that are ready to be used as is via the flag `--freqaimodel`. These libraries include `CatBoost`, `LightGBM`, and `XGBoost` regression, classification, and multi-target models, and can be found in `freqai/prediction_models/`.
FreqAI has multiple example prediction model libraries that are ready to be used as is via the flag `--freqaimodel`. These libraries include `LightGBM`, and `XGBoost` regression, classification, and multi-target models, and can be found in `freqai/prediction_models/`.
Regression and classification models differ in what targets they predict - a regression model will predict a target of continuous values, for example what price BTC will be at tomorrow, whilst a classifier will predict a target of discrete values, for example if the price of BTC will go up tomorrow or not. This means that you have to specify your targets differently depending on which model type you are using (see details [below](#setting-model-targets)).
All of the aforementioned model libraries implement gradient boosted decision tree algorithms. They all work on the principle of ensemble learning, where predictions from multiple simple learners are combined to get a final prediction that is more stable and generalized. The simple learners in this case are decision trees. Gradient boosting refers to the method of learning, where each simple learner is built in sequence - the subsequent learner is used to improve on the error from the previous learner. If you want to learn more about the different model libraries you can find the information in their respective docs:
* CatBoost: https://catboost.ai/en/docs/
* LightGBM: https://lightgbm.readthedocs.io/en/v3.3.2/#
* XGBoost: https://xgboost.readthedocs.io/en/stable/#
* LightGBM: <https://lightgbm.readthedocs.io/en/v3.3.2/#>
* XGBoost: <https://xgboost.readthedocs.io/en/stable/#>
* CatBoost: <https://catboost.ai/en/docs/> (No longer actively supported since 2025.12)
There are also numerous online articles describing and comparing the algorithms. Some relatively lightweight examples would be [CatBoost vs. LightGBM vs. XGBoost — Which is the best algorithm?](https://towardsdatascience.com/catboost-vs-lightgbm-vs-xgboost-c80f40662924#:~:text=In%20CatBoost%2C%20symmetric%20trees%2C%20or,the%20same%20depth%20can%20differ.) and [XGBoost, LightGBM or CatBoost — which boosting algorithm should I use?](https://medium.com/riskified-technology/xgboost-lightgbm-or-catboost-which-boosting-algorithm-should-i-use-e7fda7bb36bc). Keep in mind that the performance of each model is highly dependent on the application and so any reported metrics might not be true for your particular use of the model.
@@ -219,7 +219,7 @@ Make sure to use unique names to avoid overriding built-in models.
#### Regressors
If you are using a regressor, you need to specify a target that has continuous values. FreqAI includes a variety of regressors, such as the `CatboostRegressor`via the flag `--freqaimodel CatboostRegressor`. An example of how you could set a regression target for predicting the price 100 candles into the future would be
If you are using a regressor, you need to specify a target that has continuous values. FreqAI includes a variety of regressors, such as the `LightGBMRegressor`via the flag `--freqaimodel LightGBMRegressor`. An example of how you could set a regression target for predicting the price 100 candles into the future would be
```python
df['&s-close_price'] = df['close'].shift(-100)
@@ -229,7 +229,7 @@ If you want to predict multiple targets, you need to define multiple labels usin
#### Classifiers
If you are using a classifier, you need to specify a target that has discrete values. FreqAI includes a variety of classifiers, such as the `CatboostClassifier` via the flag `--freqaimodel CatboostClassifier`. If you elects to use a classifier, the classes need to be set using strings. For example, if you want to predict if the price 100 candles into the future goes up or down you would set
If you are using a classifier, you need to specify a target that has discrete values. FreqAI includes a variety of classifiers, such as the `LightGBMClassifier` via the flag `--freqaimodel LightGBMClassifier`. If you elects to use a classifier, the classes need to be set using strings. For example, if you want to predict if the price 100 candles into the future goes up or down you would set
```python
df['&s-up_or_down'] = np.where( df["close"].shift(-100) > df["close"], 'up', 'down')

2
freqtrade/freqai/base_models/BaseClassifierModel.py
View File

@@ -18,7 +18,7 @@ class BaseClassifierModel(IFreqaiModel):
"""
Base class for regression type models (e.g. Catboost, LightGBM, XGboost etc.).
User *must* inherit from this class and set fit(). See example scripts
such as prediction_models/CatboostClassifier.py for guidance.
such as prediction_models/XGBoostClassifier.py for guidance.
"""
def train(self, unfiltered_df: DataFrame, pair: str, dk: FreqaiDataKitchen, **kwargs) -> Any:

2
freqtrade/freqai/base_models/BaseRegressionModel.py
View File

@@ -18,7 +18,7 @@ class BaseRegressionModel(IFreqaiModel):
"""
Base class for regression type models (e.g. Catboost, LightGBM, XGboost etc.).
User *must* inherit from this class and set fit(). See example scripts
such as prediction_models/CatboostRegressor.py for guidance.
such as prediction_models/XGBoostRegressor.py for guidance.
"""
def train(self, unfiltered_df: DataFrame, pair: str, dk: FreqaiDataKitchen, **kwargs) -> Any:

4
freqtrade/freqai/freqai_interface.py
View File

@@ -948,7 +948,7 @@ class IFreqaiModel(ABC):
return dk
# Following methods which are overridden by user made prediction models.
# See freqai/prediction_models/CatboostPredictionModel.py for an example.
# See freqai/prediction_models/XGBoostRegressor.py for an example.
@abstractmethod
def train(self, unfiltered_df: DataFrame, pair: str, dk: FreqaiDataKitchen, **kwargs) -> Any:
@@ -964,7 +964,7 @@ class IFreqaiModel(ABC):
def fit(self, data_dictionary: dict[str, Any], dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
Most regressors use the same function names and arguments e.g. user
can drop in LGBMRegressor in place of CatBoostRegressor and all data
can drop in LGBMRegressor in place of XGBoostRegressor and all data
management will be properly handled by Freqai.
:param data_dictionary: Dict = the dictionary constructed by DataHandler to hold
all the training and test data/labels.

61
freqtrade/freqai/prediction_models/CatboostClassifier.py
View File

@@ -1,61 +0,0 @@
import logging
from pathlib import Path
from typing import Any
from catboost import CatBoostClassifier, Pool
from freqtrade.freqai.base_models.BaseClassifierModel import BaseClassifierModel
from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
logger = logging.getLogger(__name__)
class CatboostClassifier(BaseClassifierModel):
"""
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
train_data = Pool(
data=data_dictionary["train_features"],
label=data_dictionary["train_labels"],
weight=data_dictionary["train_weights"],
)
if self.freqai_info.get("data_split_parameters", {}).get("test_size", 0.1) == 0:
test_data = None
else:
test_data = Pool(
data=data_dictionary["test_features"],
label=data_dictionary["test_labels"],
weight=data_dictionary["test_weights"],
)
cbr = CatBoostClassifier(
allow_writing_files=True,
loss_function="MultiClass",
train_dir=Path(dk.data_path),
**self.model_training_parameters,
)
init_model = self.get_init_model(dk.pair)
cbr.fit(
X=train_data,
eval_set=test_data,
init_model=init_model,
)
return cbr

79
freqtrade/freqai/prediction_models/CatboostClassifierMultiTarget.py
View File

@@ -1,79 +0,0 @@
import logging
from pathlib import Path
from typing import Any
from catboost import CatBoostClassifier, Pool
from freqtrade.freqai.base_models.BaseClassifierModel import BaseClassifierModel
from freqtrade.freqai.base_models.FreqaiMultiOutputClassifier import FreqaiMultiOutputClassifier
from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
logger = logging.getLogger(__name__)
class CatboostClassifierMultiTarget(BaseClassifierModel):
"""
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
cbc = CatBoostClassifier(
allow_writing_files=True,
loss_function="MultiClass",
train_dir=Path(dk.data_path),
**self.model_training_parameters,
)
X = data_dictionary["train_features"]
y = data_dictionary["train_labels"]
sample_weight = data_dictionary["train_weights"]
eval_sets = [None] * y.shape[1]
if self.freqai_info.get("data_split_parameters", {}).get("test_size", 0.1) != 0:
eval_sets = [None] * data_dictionary["test_labels"].shape[1]
for i in range(data_dictionary["test_labels"].shape[1]):
eval_sets[i] = Pool(
data=data_dictionary["test_features"],
label=data_dictionary["test_labels"].iloc[:, i],
weight=data_dictionary["test_weights"],
)
init_model = self.get_init_model(dk.pair)
if init_model:
init_models = init_model.estimators_
else:
init_models = [None] * y.shape[1]
fit_params = []
for i in range(len(eval_sets)):
fit_params.append(
{
"eval_set": eval_sets[i],
"init_model": init_models[i],
}
)
model = FreqaiMultiOutputClassifier(estimator=cbc)
thread_training = self.freqai_info.get("multitarget_parallel_training", False)
if thread_training:
model.n_jobs = y.shape[1]
model.fit(X=X, y=y, sample_weight=sample_weight, fit_params=fit_params)
return model

60
freqtrade/freqai/prediction_models/CatboostRegressor.py
View File

@@ -1,60 +0,0 @@
import logging
from pathlib import Path
from typing import Any
from catboost import CatBoostRegressor, Pool
from freqtrade.freqai.base_models.BaseRegressionModel import BaseRegressionModel
from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
logger = logging.getLogger(__name__)
class CatboostRegressor(BaseRegressionModel):
"""
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
train_data = Pool(
data=data_dictionary["train_features"],
label=data_dictionary["train_labels"],
weight=data_dictionary["train_weights"],
)
if self.freqai_info.get("data_split_parameters", {}).get("test_size", 0.1) == 0:
test_data = None
else:
test_data = Pool(
data=data_dictionary["test_features"],
label=data_dictionary["test_labels"],
weight=data_dictionary["test_weights"],
)
init_model = self.get_init_model(dk.pair)
model = CatBoostRegressor(
allow_writing_files=True,
train_dir=Path(dk.data_path),
**self.model_training_parameters,
)
model.fit(
X=train_data,
eval_set=test_data,
init_model=init_model,
)
return model

78
freqtrade/freqai/prediction_models/CatboostRegressorMultiTarget.py
View File

@@ -1,78 +0,0 @@
import logging
from pathlib import Path
from typing import Any
from catboost import CatBoostRegressor, Pool
from freqtrade.freqai.base_models.BaseRegressionModel import BaseRegressionModel
from freqtrade.freqai.base_models.FreqaiMultiOutputRegressor import FreqaiMultiOutputRegressor
from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
logger = logging.getLogger(__name__)
class CatboostRegressorMultiTarget(BaseRegressionModel):
"""
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
cbr = CatBoostRegressor(
allow_writing_files=True,
train_dir=Path(dk.data_path),
**self.model_training_parameters,
)
X = data_dictionary["train_features"]
y = data_dictionary["train_labels"]
sample_weight = data_dictionary["train_weights"]
eval_sets = [None] * y.shape[1]
if self.freqai_info.get("data_split_parameters", {}).get("test_size", 0.1) != 0:
eval_sets = [None] * data_dictionary["test_labels"].shape[1]
for i in range(data_dictionary["test_labels"].shape[1]):
eval_sets[i] = Pool(
data=data_dictionary["test_features"],
label=data_dictionary["test_labels"].iloc[:, i],
weight=data_dictionary["test_weights"],
)
init_model = self.get_init_model(dk.pair)
if init_model:
init_models = init_model.estimators_
else:
init_models = [None] * y.shape[1]
fit_params = []
for i in range(len(eval_sets)):
fit_params.append(
{
"eval_set": eval_sets[i],
"init_model": init_models[i],
}
)
model = FreqaiMultiOutputRegressor(estimator=cbr)
thread_training = self.freqai_info.get("multitarget_parallel_training", False)
if thread_training:
model.n_jobs = y.shape[1]
model.fit(X=X, y=y, sample_weight=sample_weight, fit_params=fit_params)
return model

4
freqtrade/freqai/utils.py
View File

@@ -97,7 +97,7 @@ def plot_feature_importance(
"""
Plot Best and worst features by importance for a single sub-train.
:param model: Any = A model which was `fit` using a common library
such as catboost or lightgbm
such as XGBoost or lightgbm
:param pair: str = pair e.g. BTC/USD
:param dk: FreqaiDataKitchen = non-persistent data container for current coin/loop
:param count_max: int = the amount of features to be loaded per column
@@ -115,6 +115,8 @@ def plot_feature_importance(
for label in models:
mdl = models[label]
if "catboost.core" in str(mdl.__class__):
# CatBoost is no longer actively supported since 2025.12
# However users can still use it in their custom models
feature_importance = mdl.get_feature_importance()
elif "lightgbm.sklearn" in str(mdl.__class__):
feature_importance = mdl.feature_importances_

2
freqtrade/templates/FreqaiExampleHybridStrategy.py
View File

@@ -20,7 +20,7 @@ class FreqaiExampleHybridStrategy(IStrategy):
Launching this strategy would be:
freqtrade trade --strategy FreqaiExampleHybridStrategy --strategy-path freqtrade/templates
--freqaimodel CatboostClassifier --config config_examples/config_freqai.example.json
--freqaimodel XGBoostClassifier --config config_examples/config_freqai.example.json
or the user simply adds this to their config:

4
freqtrade/templates/FreqaiExampleStrategy.py
View File

@@ -205,8 +205,8 @@ class FreqaiExampleStrategy(IStrategy):
# If user wishes to use multiple targets, they can add more by
# appending more columns with '&'. User should keep in mind that multi targets
# requires a multioutput prediction model such as
# freqai/prediction_models/CatboostRegressorMultiTarget.py,
# freqtrade trade --freqaimodel CatboostRegressorMultiTarget
# freqai/prediction_models/LightGBMClassifierMultiTarget.py,
# freqtrade trade --freqaimodel LightGBMClassifierMultiTarget
# df["&-s_range"] = (
# df["close"]

1
pyproject.toml
View File

@@ -85,7 +85,6 @@ hyperopt = [
freqai = [
"scikit-learn",
"joblib",
"catboost; platform_machine != 'arm'",
"lightgbm",
"xgboost",
"tensorboard",

1
requirements-freqai.txt
View File

@@ -5,7 +5,6 @@
# Required for freqai
scikit-learn==1.7.2
joblib==1.5.2
catboost==1.2.8; 'arm' not in platform_machine
lightgbm==4.6.0
xgboost==3.1.2
tensorboard==2.20.0

9
tests/freqai/test_freqai_interface.py
View File

@@ -31,9 +31,6 @@ from tests.freqai.conftest import (
def can_run_model(model: str) -> None:
is_pytorch_model = "Reinforcement" in model or "PyTorch" in model
if is_arm() and "Catboost" in model:
pytest.skip("CatBoost is not supported on ARM.")
if is_pytorch_model and is_mac():
pytest.skip("Reinforcement learning / PyTorch module not available on intel based Mac OS.")
@@ -44,7 +41,6 @@ def can_run_model(model: str) -> None:
("LightGBMRegressor", True, False, True, True, False, 0, 0),
("XGBoostRegressor", False, True, False, True, False, 10, 0.05),
("XGBoostRFRegressor", False, False, False, True, False, 0, 0),
("CatboostRegressor", False, False, False, True, True, 0, 0),
("PyTorchMLPRegressor", False, False, False, False, False, 0, 0),
("PyTorchTransformerRegressor", False, False, False, False, False, 0, 0),
("ReinforcementLearner", False, True, False, True, False, 0, 0),
@@ -138,9 +134,7 @@ def test_extract_data_and_train_model_Standard(
[
("LightGBMRegressorMultiTarget", "freqai_test_multimodel_strat"),
("XGBoostRegressorMultiTarget", "freqai_test_multimodel_strat"),
("CatboostRegressorMultiTarget", "freqai_test_multimodel_strat"),
("LightGBMClassifierMultiTarget", "freqai_test_multimodel_classifier_strat"),
("CatboostClassifierMultiTarget", "freqai_test_multimodel_classifier_strat"),
],
)
@pytest.mark.filterwarnings(r"ignore:.*__sklearn_tags__.*:DeprecationWarning")
@@ -184,7 +178,6 @@ def test_extract_data_and_train_model_MultiTargets(mocker, freqai_conf, model, s
"model",
[
"LightGBMClassifier",
"CatboostClassifier",
"XGBoostClassifier",
"XGBoostRFClassifier",
"SKLearnRandomForestClassifier",
@@ -246,13 +239,11 @@ def test_extract_data_and_train_model_Classifiers(mocker, freqai_conf, model):
[
("LightGBMRegressor", 2, "freqai_test_strat"),
("XGBoostRegressor", 2, "freqai_test_strat"),
("CatboostRegressor", 2, "freqai_test_strat"),
("PyTorchMLPRegressor", 2, "freqai_test_strat"),
("PyTorchTransformerRegressor", 2, "freqai_test_strat"),
("ReinforcementLearner", 3, "freqai_rl_test_strat"),
("XGBoostClassifier", 2, "freqai_test_classifier"),
("LightGBMClassifier", 2, "freqai_test_classifier"),
("CatboostClassifier", 2, "freqai_test_classifier"),
("PyTorchMLPClassifier", 2, "freqai_test_classifier"),
],
)