Release 1.199.0

See release notes.
nautechsystems · Aug 19, 2024 · 7a496b4 · 7a496b4
2 parents 84442a2 + 05346c4
commit 7a496b4
Show file tree

Hide file tree

Showing 325 changed files with 5,816 additions and 3,183 deletions.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -83,7 +83,7 @@ repos:
         exclude: "docs/_pygments/monokai.py"
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.5.7
+    rev: v0.6.1
     hooks:
       - id: ruff
         args: ["--fix"]

diff --git a/RELEASES.md b/RELEASES.md
@@ -1,12 +1,45 @@
+# NautilusTrader 1.199.0 Beta
+
+Released on 19th August 2024 (UTC).
+
+### Enhancements
+- Added `LiveExecEngineConfig.generate_missing_orders` reconciliation config option to align internal and external position states
+- Added `LogLevel::TRACE` (only available in Rust for debug/development builds)
+- Added `Actor.subscribe_signal(...)` method and `Data.is_signal(...)` class method (#1853), thanks @faysou
+- Added Binance Futures support for `HEDGE` mode (#1846), thanks @DevRoss
+- Overhauled and refined error modeling and handling in Rust (#1849, #1858), thanks @twitu
+- Improved `BinanceExecutionClient` position report requests (can now filter by instrument and includes reporting for flat positions)
+- Improved `BybitExecutionClient` position report requests (can now filter by instrument and includes reporting for flat positions)
+- Improved `LiveExecutionEngine` reconciliation robustness and recovery when internal positions do not match external positions
+- Improved `@customdataclass` decorator constructor to allow more positional arguments (#1850), thanks @faysou
+- Improved `@customdataclass` documentation (#1854), thanks @faysou
+- Upgraded `datafusion` crate to v41.0.0
+- Upgraded `tokio` crate to v1.39.3
+- Upgraded `uvloop` to v0.20.0 (upgrades libuv to v1.48.0)
+
+### Breaking Changes
+- Changed `VolumeWeightedAveragePrice` calculation formula to use each bars "typical" price (#1842), thanks @evgenii-prusov
+- Changed `OptionsContract` constructor parameter ordering and Arrow schema (consistently group option kind and strike price)
+- Renamed `snapshot_positions_interval` to `snapshot_positions_interval_secs` (more explicitly indicates time units)
+- Moved `snapshot_orders` config setting to `ExecEngineConfig` (can now be used for all environment contexts)
+- Moved `snapshot_positions` config setting to `ExecEngineConfig` (can now be used for all environment contexts)
+- Moved `snapshot_positions_interval_secs` config setting to `ExecEngineConfig` (can now be used for all environment contexts)
+
+### Fixes
+- Fixed `Position` exception type on duplicate fill (should be `KeyError` to align with the same error for `Order`)
+- Fixed Bybit position report parsing when position is flat (`BybitPositionSide` now correctly handles the empty string)
+
+---
+
 # NautilusTrader 1.198.0 Beta
 
 Released on 9th August 2024 (UTC).
 
 ### Enhancements
-- Added `@customdata` decorator to reduce need for boiler plate implementing custom data types (#1828), thanks @faysou
+- Added `@customdataclass` decorator to reduce need for boiler plate implementing custom data types (#1828), thanks @faysou
 - Added timeout for HTTP client in Rust (#1835), thanks @davidsblom
 - Added catalog conversion function of streamed data to backtest data (#1834), thanks @faysou
-- Upgraded Cython to 3.0.11
+- Upgraded Cython to v3.0.11
 
 ### Breaking Changes
 None
@@ -38,14 +71,14 @@ Released on 2nd August 2024 (UTC).
 - Refactored order submission error handling for Interactive Brokers (#1783), thanks @rsmb7z
 - Improved live reconciliation robustness (will now generate inferred orders necessary to align external position state)
 - Improved tests for Interactive Brokers (#1776), thanks @mylesgamez
-- Upgraded `tokio` crate to 1.39.2
-- Upgraded `datafusion` crate to 40.0.0
+- Upgraded `tokio` crate to v1.39.2
+- Upgraded `datafusion` crate to v40.0.0
 
 ### Breaking Changes
 - Removed `VenueStatus` and all associated methods and schemas (redundant with `InstrumentStatus`)
 - Renamed `QuoteTick.extract_volume(...)` to `.extract_size(...)` (more accurate terminology)
 - Changed `InstrumentStatus` params (support Databento `status` schema)
-- Changed `InstrumentStatus` arrow schema
+- Changed `InstrumentStatus` Arrow schema
 - Changed `OrderBook` FFI API to take data by reference instead of by value
 
 ### Fixes
@@ -289,7 +322,7 @@ Released on 22nd March 2024 (UTC).
 - Ported `VIDYA` indicator to Rust, thanks @Pushkarm029
 - Refactored `InteractiveBrokersEWrapper`, thanks @rsmb7z
 - Redact Redis passwords in strings and logs
-- Upgraded `redis` crate to 0.25.2 which bumps up TLS dependencies, and turned on `tls-rustls-webpki-roots` feature flag
+- Upgraded `redis` crate to v0.25.2 which bumps up TLS dependencies, and turned on `tls-rustls-webpki-roots` feature flag
 
 ### Breaking Changes
 None
@@ -765,7 +798,7 @@ Released on 31st July 2023 (UTC).
 - Added `USDP` (Pax Dollar) and `TUSD` (TrueUSD) stablecoins
 - Improved `OrderMatchingEngine` handling when no fills (an error is now logged)
 - Improved `Binance` live clients logging
-- Upgraded Cython to 3.0.0 stable
+- Upgraded Cython to v3.0.0 stable
 
 ### Breaking Changes
 - Moved `filter_unclaimed_external_orders` from `ExecEngineConfig` to `LiveExecEngineConfig`

diff --git a/docs/concepts/advanced/custom_data.md b/docs/concepts/advanced/custom_data.md
@@ -101,6 +101,21 @@ def on_data(self, data: Data) -> None:
         # Do something with the data
 ```
 
+### Publishing and receiving signal data
+
+Here is an example of publishing and receiving signal data using the `MessageBus` from an actor or strategy. 
+A signal is an automatically generated custom data identified by a name containing only one value of a basic type 
+(str, float, int, bool or bytes).
+
+```python
+self.publish_signal("signal_name", value, ts_event)
+self.subscribe_signal("signal_name")
+
+def on_data(self, data):
+    if data.is_signal("signal_name"):
+        print("Signal", data)
+```
+
 ## Option Greeks example
 
 This example demonstrates how to create a custom data type for option Greeks, specifically the delta.
@@ -186,7 +201,7 @@ class GreeksData(Data):
 
 ### Publishing and receiving data
 
-Here is an example of publishing and receiving data using the `MessageBus` from an actor (which includes strategies):
+Here is an example of publishing and receiving data using the `MessageBus` from an actor or strategy:
 
 ```python
 register_serializable_type(GreeksData, GreeksData.to_dict, GreeksData.from_dict)
@@ -204,7 +219,7 @@ def on_data(self, data):
 
 ### Writing and reading data using the cache
 
-Here is an example of writing and reading data using the `Cache` from an actor (which includes strategies):
+Here is an example of writing and reading data using the `Cache` from an actor or strategy:
 
 ```python
 def greeks_key(instrument_id: InstrumentId):
@@ -219,7 +234,8 @@ def greeks_from_cache(self, instrument_id: InstrumentId):
 
 ### Writing and reading data using a catalog
 
-For streaming custom data to feather files or writing it to parquet files in a catalog (`register_arrow` needs to be used):
+For streaming custom data to feather files or writing it to parquet files in a catalog 
+(`register_arrow` needs to be used):
 
 ```python
 register_arrow(GreeksData, GreeksData.schema(), GreeksData.to_catalog, GreeksData.from_catalog)
@@ -257,3 +273,31 @@ GreeksTestData(
     ts_init=2,
 )
 ```
+
+### Custom data type stub
+
+To enhance development convenience and improve code suggestions in your IDE, you can create a `.pyi`
+stub file with the proper constructor signature for your custom data types as well as type hints for attributes. 
+This is particularly useful when the constructor is dynamically generated at runtime, as it allows the IDE to recognize 
+and provide suggestions for the class's methods and attributes.
+
+For instance, if you have a custom data class defined in `greeks.py`, you can create a corresponding `greeks.pyi` file 
+with the following constructor signature:
+
+```python
+from nautilus_trader.core.data import Data
+from nautilus_trader.model.identifiers import InstrumentId
+
+
+class GreeksData(Data):
+    instrument_id: InstrumentId
+    delta: float
+
+    def __init__(
+        self,
+        ts_event: int = 0,
+        ts_init: int = 0,
+        instrument_id: InstrumentId = InstrumentId.from_str("ES.GLBX"),
+        delta: float = 0.0,
+  ) -> GreeksData: ...
+```
diff --git a/docs/getting_started/backtest_high_level.ipynb b/docs/getting_started/backtest_high_level.ipynb
@@ -34,9 +34,9 @@
    "metadata": {},
    "source": [
     "## Prerequisites\n",
-    "- [NautilusTrader](https://pypi.org/project/nautilus_trader/) latest release installed (`pip install -U nautilus_trader`)\n",
+    "- Python 3.10+ installed\n",
     "- [JupyterLab](https://jupyter.org/) or similar installed (`pip install -U jupyterlab`)\n",
-    "- Python 3.10+ installed"
+    "- [NautilusTrader](https://pypi.org/project/nautilus_trader/) latest release installed (`pip install -U nautilus_trader`)"
    ]
   },
   {
@@ -56,19 +56,20 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import datetime\n",
     "import shutil\n",
     "from decimal import Decimal\n",
     "from pathlib import Path\n",
     "\n",
-    "import fsspec\n",
     "import pandas as pd\n",
     "\n",
-    "from nautilus_trader.backtest.node import BacktestNode, BacktestVenueConfig, BacktestDataConfig, BacktestRunConfig, BacktestEngineConfig\n",
-    "from nautilus_trader.core.datetime import dt_to_unix_nanos\n",
+    "from nautilus_trader.backtest.node import BacktestDataConfig\n",
+    "from nautilus_trader.backtest.node import BacktestEngineConfig\n",
+    "from nautilus_trader.backtest.node import BacktestNode\n",
+    "from nautilus_trader.backtest.node import BacktestRunConfig\n",
+    "from nautilus_trader.backtest.node import BacktestVenueConfig\n",
     "from nautilus_trader.config import ImportableStrategyConfig\n",
+    "from nautilus_trader.core.datetime import dt_to_unix_nanos\n",
     "from nautilus_trader.model.data import QuoteTick\n",
-    "from nautilus_trader.model.objects import Price, Quantity\n",
     "from nautilus_trader.persistence.catalog import ParquetDataCatalog\n",
     "from nautilus_trader.persistence.wranglers import QuoteTickDataWrangler\n",
     "from nautilus_trader.test_kit.providers import CSVTickDataLoader\n",
@@ -84,7 +85,7 @@
     "\n",
     "For this example we will use FX data from `histdata.com`. Simply go to https://www.histdata.com/download-free-forex-historical-data/?/ascii/tick-data-quotes/ and select an FX pair, then select one or more months of data to download.\n",
     "\n",
-    "Once you have downloaded the data, set the variable `DATA_DIR` below to the directory containing the data. By default, it will use the users `Downloads` directory."
+    "Once you have downloaded the data, set the variable `DATA_DIR` below to the directory containing the data. By default, it will use the users `Downloads/Data/` directory."
    ]
   },
   {
@@ -94,7 +95,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "DATA_DIR = \"~/Downloads/\""
+    "DATA_DIR = \"~/Downloads/Data/\""
    ]
   },
   {
@@ -283,13 +284,13 @@
     "    ImportableStrategyConfig(\n",
     "        strategy_path=\"nautilus_trader.examples.strategies.ema_cross:EMACross\",\n",
     "        config_path=\"nautilus_trader.examples.strategies.ema_cross:EMACrossConfig\",\n",
-    "        config=dict(\n",
-    "            instrument_id=instrument.id,\n",
-    "            bar_type=\"EUR/USD.SIM-15-MINUTE-BID-INTERNAL\",\n",
-    "            fast_ema_period=10,\n",
-    "            slow_ema_period=20,\n",
-    "            trade_size=Decimal(1_000_000),\n",
-    "        ),\n",
+    "        config={\n",
+    "            \"instrument_id\": instrument.id,\n",
+    "            \"bar_type\": \"EUR/USD.SIM-15-MINUTE-BID-INTERNAL\",\n",
+    "            \"fast_ema_period\": 10,\n",
+    "            \"slow_ema_period\": 20,\n",
+    "            \"trade_size\": Decimal(1_000_000),\n",
+    "        },\n",
     "    ),\n",
     "]"
    ]

diff --git a/docs/getting_started/backtest_low_level.ipynb b/docs/getting_started/backtest_low_level.ipynb
@@ -36,9 +36,9 @@
    "metadata": {},
    "source": [
     "## Prerequisites\n",
-    "- [NautilusTrader](https://pypi.org/project/nautilus_trader/) latest release installed (`pip install -U nautilus_trader`)\n",
+    "- Python 3.10+ installed\n",
     "- [JupyterLab](https://jupyter.org/) or similar installed (`pip install -U jupyterlab`)\n",
-    "- Python 3.10+ installed"
+    "- [NautilusTrader](https://pypi.org/project/nautilus_trader/) latest release installed (`pip install -U nautilus_trader`)"
    ]
   },
   {
@@ -58,11 +58,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import time\n",
     "from decimal import Decimal\n",
     "\n",
-    "import pandas as pd\n",
-    "\n",
     "from nautilus_trader.backtest.engine import BacktestEngine\n",
     "from nautilus_trader.backtest.engine import BacktestEngineConfig\n",
     "from nautilus_trader.examples.algorithms.twap import TWAPExecAlgorithm\n",