CTYPTO/freqtrade
1
0
Fork 0
mirror of https://github.com/freqtrade/freqtrade.git synced 2026-02-10 08:10:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'develop' into feature/stoploss-start-at

Browse Source
This commit is contained in:
Matthias
2024-08-02 19:51:09 +02:00
parent 1760624954 9429657a2b
commit 79d4dc1646
102 changed files with 8637 additions and 2296 deletions
Download Patch File Download Diff File Expand all files Collapse all files

152
docs/advanced-orderflow.md Normal file
View File

@@ -0,0 +1,152 @@
# Orderflow data
This guide walks you through utilizing public trade data for advanced orderflow analysis in Freqtrade.
!!! Warning "Experimental Feature"
The orderflow feature is currently in beta and may be subject to changes in future releases. Please report any issues or feedback on the [Freqtrade GitHub repository](https://github.com/freqtrade/freqtrade/issues).
!!! Warning "Performance"
Orderflow requires raw trades data. This data is rather large, and can cause a slow initial startup, when freqtrade needs to download the trades data for the last X candles. Additionally, enabling this feature will cause increased memory usage. Please ensure to have sufficient resources available.
## Getting Started
### Enable Public Trades
In your `config.json` file, set the `use_public_trades` option to true under the `exchange` section.
```json
"exchange": {
...
"use_public_trades": true,
}
```
### Configure Orderflow Processing
Define your desired settings for orderflow processing within the orderflow section of config.json. Here, you can adjust factors like:
- `cache_size`: How many previous orderflow candles are saved into cache instead of calculated every new candle
- `max_candles`: Filter how many candles would you like to get trades data for.
- `scale`: This controls the price bin size for the footprint chart.
- `stacked_imbalance_range`: Defines the minimum consecutive imbalanced price levels required for consideration.
- `imbalance_volume`: Filters out imbalances with volume below this threshold.
- `imbalance_ratio`: Filters out imbalances with a ratio (difference between ask and bid volume) lower than this value.
```json
"orderflow": {
"cache_size": 1000,
"max_candles": 1500,
"scale": 0.5,
"stacked_imbalance_range": 3, // needs at least this amount of imbalance next to each other
"imbalance_volume": 1, // filters out below
"imbalance_ratio": 3 // filters out ratio lower than
},
```
## Downloading Trade Data for Backtesting
To download historical trade data for backtesting, use the --dl-trades flag with the freqtrade download-data command.
```bash
freqtrade download-data -p BTC/USDT:USDT --timerange 20230101- --trading-mode futures --timeframes 5m --dl-trades
```
!!! Warning "Data availability"
Not all exchanges provide public trade data. For supported exchanges, freqtrade will warn you if public trade data is not available if you start downloading data with the `--dl-trades` flag.
## Accessing Orderflow Data
Once activated, several new columns become available in your dataframe:
``` python
dataframe["trades"] # Contains information about each individual trade.
dataframe["orderflow"] # Represents a footprint chart dict (see below)
dataframe["imbalances"] # Contains information about imbalances in the order flow.
dataframe["bid"] # Total bid volume
dataframe["ask"] # Total ask volume
dataframe["delta"] # Difference between ask and bid volume.
dataframe["min_delta"] # Minimum delta within the candle
dataframe["max_delta"] # Maximum delta within the candle
dataframe["total_trades"] # Total number of trades
dataframe["stacked_imbalances_bid"] # Price level of stacked bid imbalance
dataframe["stacked_imbalances_ask"] # Price level of stacked ask imbalance
```
You can access these columns in your strategy code for further analysis. Here's an example:
``` python
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
# Calculating cumulative delta
dataframe["cum_delta"] = cumulative_delta(dataframe["delta"])
# Accessing total trades
total_trades = dataframe["total_trades"]
...
def cumulative_delta(delta: Series):
cumdelta = delta.cumsum()
return cumdelta
```
### Footprint chart (`dataframe["orderflow"]`)
This column provides a detailed breakdown of buy and sell orders at different price levels, offering valuable insights into order flow dynamics. The `scale` parameter in your configuration determines the price bin size for this representation
The `orderflow` column contains a dict with the following structure:
``` output
{
"price": {
"bid_amount": 0.0,
"ask_amount": 0.0,
"bid": 0,
"ask": 0,
"delta": 0.0,
"total_volume": 0.0,
"total_trades": 0
}
}
```
#### Orderflow column explanation
- key: Price bin - binned at `scale` intervals
- `bid_amount`: Total volume bought at each price level.
- `ask_amount`: Total volume sold at each price level.
- `bid`: Number of buy orders at each price level.
- `ask`: Number of sell orders at each price level.
- `delta`: Difference between ask and bid volume at each price level.
- `total_volume`: Total volume (ask amount + bid amount) at each price level.
- `total_trades`: Total number of trades (ask + bid) at each price level.
By leveraging these features, you can gain valuable insights into market sentiment and potential trading opportunities based on order flow analysis.
### Raw trades data (`dataframe["trades"]`)
List with the individual trades that occurred during the candle. This data can be used for more granular analysis of order flow dynamics.
Each individual entry contains a dict with the following keys:
- `timestamp`: Timestamp of the trade.
- `date`: Date of the trade.
- `price`: Price of the trade.
- `amount`: Volume of the trade.
- `side`: Buy or sell.
- `id`: Unique identifier for the trade.
- `cost`: Total cost of the trade (price * amount).
### Imbalances (`dataframe["imbalances"]`)
This column provides a dict with information about imbalances in the order flow. An imbalance occurs when there is a significant difference between the ask and bid volume at a given price level.
Each row looks as follows - with price as index, and the corresponding bid and ask imbalance values as columns
``` output
{
"price": {
"bid_imbalance": False,
"ask_imbalance": False
}
}
```

38
docs/advanced-setup.md
View File

@@ -114,8 +114,46 @@ services:
--strategy SampleStrategy
```
You can use whatever naming convention you want, freqtrade1 and 2 are arbitrary. Note, that you will need to use different database files, port mappings and telegram configurations for each instance, as mentioned above.
## Use a different database system
Freqtrade is using SQLAlchemy, which supports multiple different database systems. As such, a multitude of database systems should be supported.
Freqtrade does not depend or install any additional database driver. Please refer to the [SQLAlchemy docs](https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls) on installation instructions for the respective database systems.
The following systems have been tested and are known to work with freqtrade:
* sqlite (default)
* PostgreSQL
* MariaDB
!!! Warning
By using one of the below database systems, you acknowledge that you know how to manage such a system. The freqtrade team will not provide any support with setup or maintenance (or backups) of the below database systems.
### PostgreSQL
Installation:
`pip install psycopg2-binary`
Usage:
`... --db-url postgresql+psycopg2://<username>:<password>@localhost:5432/<database>`
Freqtrade will automatically create the tables necessary upon startup.
If you're running different instances of Freqtrade, you must either setup one database per Instance or use different users / schemas for your connections.
### MariaDB / MySQL
Freqtrade supports MariaDB by using SQLAlchemy, which supports multiple different database systems.
Installation:
`pip install pymysql`
Usage:
`... --db-url mysql+pymysql://<username>:<password>@localhost:3306/<database>`
## Configure the bot running as a systemd service

4
docs/freqai-running.md
View File

@@ -83,6 +83,10 @@ To change your **features**, you **must** set a new `identifier` in the config t
To save the models generated during a particular backtest so that you can start a live deployment from one of them instead of training a new model, you must set `save_backtest_models` to `True` in the config.
!!! Note
To ensure that the model can be reused, freqAI will call your strategy with a dataframe of length 1.
If your strategy requires more data than this to generate the same features, you can't reuse backtest predictions for live deployment and need to update your `identifier` for each new backtest.
### Backtest live collected predictions
FreqAI allow you to reuse live historic predictions through the backtest parameter `--freqai-backtest-live-models`. This can be useful when you want to reuse predictions generated in dry/run for comparison or other study.

88
docs/includes/pairlists.md
View File

@@ -2,11 +2,11 @@
Pairlist Handlers define the list of pairs (pairlist) that the bot should trade. They are configured in the `pairlists` section of the configuration settings.
In your configuration, you can use Static Pairlist (defined by the [`StaticPairList`](#static-pair-list) Pairlist Handler) and Dynamic Pairlist (defined by the [`VolumePairList`](#volume-pair-list) Pairlist Handler).
In your configuration, you can use Static Pairlist (defined by the [`StaticPairList`](#static-pair-list) Pairlist Handler) and Dynamic Pairlist (defined by the [`VolumePairList`](#volume-pair-list) and [`PercentChangePairList`](#percent-change-pair-list) Pairlist Handlers).
Additionally, [`AgeFilter`](#agefilter), [`PrecisionFilter`](#precisionfilter), [`PriceFilter`](#pricefilter), [`ShuffleFilter`](#shufflefilter), [`SpreadFilter`](#spreadfilter) and [`VolatilityFilter`](#volatilityfilter) act as Pairlist Filters, removing certain pairs and/or moving their positions in the pairlist.
If multiple Pairlist Handlers are used, they are chained and a combination of all Pairlist Handlers forms the resulting pairlist the bot uses for trading and backtesting. Pairlist Handlers are executed in the sequence they are configured. You can define either `StaticPairList`, `VolumePairList`, `ProducerPairList`, `RemotePairList` or `MarketCapPairList` as the starting Pairlist Handler.
If multiple Pairlist Handlers are used, they are chained and a combination of all Pairlist Handlers forms the resulting pairlist the bot uses for trading and backtesting. Pairlist Handlers are executed in the sequence they are configured. You can define either `StaticPairList`, `VolumePairList`, `ProducerPairList`, `RemotePairList`, `MarketCapPairList` or `PercentChangePairList` as the starting Pairlist Handler.
Inactive markets are always removed from the resulting pairlist. Explicitly blacklisted pairs (those in the `pair_blacklist` configuration setting) are also always removed from the resulting pairlist.
@@ -22,6 +22,7 @@ You may also use something like `.*DOWN/BTC` or `.*UP/BTC` to exclude leveraged
* [`StaticPairList`](#static-pair-list) (default, if not configured differently)
* [`VolumePairList`](#volume-pair-list)
* [`PercentChangePairList`](#percent-change-pair-list)
* [`ProducerPairList`](#producerpairlist)
* [`RemotePairList`](#remotepairlist)
* [`MarketCapPairList`](#marketcappairlist)
@@ -152,6 +153,89 @@ More sophisticated approach can be used, by using `lookback_timeframe` for candl
!!! Note
`VolumePairList` does not support backtesting mode.
#### Percent Change Pair List
`PercentChangePairList` filters and sorts pairs based on the percentage change in their price over the last 24 hours or any defined timeframe as part of advanced options. This allows traders to focus on assets that have experienced significant price movements, either positive or negative.
**Configuration Options**
* `number_assets`: Specifies the number of top pairs to select based on the 24-hour percentage change.
* `min_value`: Sets a minimum percentage change threshold. Pairs with a percentage change below this value will be filtered out.
* `max_value`: Sets a maximum percentage change threshold. Pairs with a percentage change above this value will be filtered out.
* `sort_direction`: Specifies the order in which pairs are sorted based on their percentage change. Accepts two values: `asc` for ascending order and `desc` for descending order.
* `refresh_period`: Defines the interval (in seconds) at which the pairlist will be refreshed. The default is 1800 seconds (30 minutes).
* `lookback_days`: Number of days to look back. When `lookback_days` is selected, the `lookback_timeframe` is defaulted to 1 day.
* `lookback_timeframe`: Timeframe to use for the lookback period.
* `lookback_period`: Number of periods to look back at.
When PercentChangePairList is used after other Pairlist Handlers, it will operate on the outputs of those handlers. If it is the leading Pairlist Handler, it will select pairs from all available markets with the specified stake currency.
`PercentChangePairList` uses ticker data from the exchange, provided via the ccxt library:
The percentage change is calculated as the change in price over the last 24 hours.
??? Note "Unsupported exchanges"
On some exchanges (like HTX), regular PercentChangePairList does not work as the api does not natively provide 24h percent change in price. This can be worked around by using candle data to calculate the percentage change. To roughly simulate 24h percent change, you can use the following configuration. Please note that these pairlists will only refresh once per day.
```json
"pairlists": [
{
"method": "PercentChangePairList",
"number_assets": 20,
"min_value": 0,
"refresh_period": 86400,
"lookback_days": 1
}
],
```
**Example Configuration to Read from Ticker**
```json
"pairlists": [
{
"method": "PercentChangePairList",
"number_assets": 15,
"min_value": -10,
"max_value": 50
}
],
```
In this configuration:
1. The top 15 pairs are selected based on the highest percentage change in price over the last 24 hours.
2. Only pairs with a percentage change between -10% and 50% are considered.
**Example Configuration to Read from Candles**
```json
"pairlists": [
{
"method": "PercentChangePairList",
"number_assets": 15,
"sort_key": "percentage",
"min_value": 0,
"refresh_period": 3600,
"lookback_timeframe": "1h",
"lookback_period": 72
}
],
```
This example builds the percent change pairs based on a rolling period of 3 days of 1-hour candles by using `lookback_timeframe` for candle size and `lookback_period` which specifies the number of candles.
The percent change in price is calculated using the following formula, which expresses the percentage difference between the current candle's close price and the previous candle's close price, as defined by the specified timeframe and lookback period:
$$ Percent Change = (\frac{Current Close - Previous Close}{Previous Close}) * 100 $$
!!! Warning "Range look back and refresh period"
When used in conjunction with `lookback_days` and `lookback_timeframe` the `refresh_period` can not be smaller than the candle size in seconds. As this will result in unnecessary requests to the exchanges API.
!!! Warning "Performance implications when using lookback range"
If used in first position in combination with lookback, the computation of the range-based percent change can be time and resource consuming, as it downloads candles for all tradable pairs. Hence it's highly advised to use the standard approach with `PercentChangePairList` to narrow the pairlist down for further percent-change calculation.
!!! Note "Backtesting"
`PercentChangePairList` does not support backtesting mode.
#### ProducerPairList
With `ProducerPairList`, you can reuse the pairlist from a [Producer](producer-consumer.md) without explicitly defining the pairlist on each consumer.

4
docs/requirements-docs.txt
View File

@@ -1,6 +1,6 @@
markdown==3.6
mkdocs==1.6.0
mkdocs-material==9.5.27
mkdocs-material==9.5.30
mdx_truly_sane_lists==1.3
pymdown-extensions==10.8.1
pymdown-extensions==10.9
jinja2==3.1.4

2
docs/rest-api.md
View File

@@ -2,7 +2,7 @@
## FreqUI
FreqUI now has it's own dedicated [documentation section](frequi.md) - please refer to that section for all information regarding the FreqUI.
FreqUI now has it's own dedicated [documentation section](freq-ui.md) - please refer to that section for all information regarding the FreqUI.
## Configuration

65
docs/sql_cheatsheet.md
View File

@@ -1,6 +1,13 @@
# SQL Helper
This page contains some help if you want to edit your sqlite db.
This page contains some help if you want to query your sqlite db.
!!! Tip "Other Database systems"
To use other Database Systems like PostgreSQL or MariaDB, you can use the same queries, but you need to use the respective client for the database system. [Click here](advanced-setup.md#use-a-different-database-system) to learn how to setup a different database system with freqtrade.
!!! Warning
If you are not familiar with SQL, you should be very careful when running queries on your database.
Always make sure to have a backup of your database before running any queries.
## Install sqlite3
@@ -43,13 +50,25 @@ sqlite3
.schema <table_name>
```
## Get all trades in the table
### Get all trades in the table
```sql
SELECT * FROM trades;
```
## Fix trade still open after a manual exit on the exchange
## Destructive queries
Queries that write to the database.
These queries should usually not be necessary as freqtrade tries to handle all database operations itself - or exposes them via API or telegram commands.
!!! Warning
Please make sure you have a backup of your database before running any of the below queries.
!!! Danger
You should also **never** run any writing query (`update`, `insert`, `delete`) while a bot is connected to the database.
This can and will lead to data corruption - most likely, without the possibility of recovery.
### Fix trade still open after a manual exit on the exchange
!!! Warning
Manually selling a pair on the exchange will not be detected by the bot and it will try to sell anyway. Whenever possible, /forceexit <tradeid> should be used to accomplish the same thing.
@@ -69,7 +88,7 @@ SET is_open=0,
WHERE id=<trade_ID_to_update>;
```
### Example
#### Example
```sql
UPDATE trades
@@ -82,7 +101,7 @@ SET is_open=0,
WHERE id=31;
```
## Remove trade from the database
### Remove trade from the database
!!! Tip "Use RPC Methods to delete trades"
Consider using `/delete <tradeid>` via telegram or rest API. That's the recommended way to deleting trades.
@@ -100,39 +119,3 @@ DELETE FROM trades WHERE id = 31;
!!! Warning
This will remove this trade from the database. Please make sure you got the correct id and **NEVER** run this query without the `where` clause.
## Use a different database system
Freqtrade is using SQLAlchemy, which supports multiple different database systems. As such, a multitude of database systems should be supported.
Freqtrade does not depend or install any additional database driver. Please refer to the [SQLAlchemy docs](https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls) on installation instructions for the respective database systems.
The following systems have been tested and are known to work with freqtrade:
* sqlite (default)
* PostgreSQL
* MariaDB
!!! Warning
By using one of the below database systems, you acknowledge that you know how to manage such a system. The freqtrade team will not provide any support with setup or maintenance (or backups) of the below database systems.
### PostgreSQL
Installation:
`pip install psycopg2-binary`
Usage:
`... --db-url postgresql+psycopg2://<username>:<password>@localhost:5432/<database>`
Freqtrade will automatically create the tables necessary upon startup.
If you're running different instances of Freqtrade, you must either setup one database per Instance or use different users / schemas for your connections.
### MariaDB / MySQL
Freqtrade supports MariaDB by using SQLAlchemy, which supports multiple different database systems.
Installation:
`pip install pymysql`
Usage:
`... --db-url mysql+pymysql://<username>:<password>@localhost:3306/<database>`

2
docs/utils.md
View File

@@ -488,7 +488,7 @@ freqtrade test-pairlist --config config.json --quote USDT BTC
`freqtrade convert-db` can be used to convert your database from one system to another (sqlite -> postgres, postgres -> other postgres), migrating all trades, orders and Pairlocks.
Please refer to the [SQL cheatsheet](sql_cheatsheet.md#use-a-different-database-system) to learn about requirements for different database systems.
Please refer to the [corresponding documentation](advanced-setup.md#use-a-different-database-system) to learn about requirements for different database systems.
```
usage: freqtrade convert-db [-h] [--db-url PATH] [--db-url-from PATH]