You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
goodboy opened this issue
May 17, 2021
· 0 comments
· May be fixed by #174
Assignees
Labels
data-layerreal-time and historical data processing and storageintegrationexternal stack and/or lib augmentations(sub-)systemsgeneral sw design and engtsdbtime series db stuff
Depth of book, otherwise known as L2 data. is something that's been on the list todo for quite a while.
We now not only have the data processing and streaming infra for it but also the UI components to display it efficiently in real-time on charts.
I originally had though about collecting and storing L2 using marketstore tick records as per #93 and #142. When you look at our tick schema it makes you think, ok so we need to store info at each level and I guess this goes all in one time series array?, well basically except there's already a project with a superior technique.
The forward gist is that techtonicdb is a much better fit with it's super compact DFT format: (timestamp, seq, is_trade, is_bid, price, size).
Component design, subsystem todos
we need an ability to spawn a local instance using the docker container
we probably want a trio-fied client to do this (which should be trivial using trio.SocketStream).
implement a client in our repo first, get it working with everything else, and then we can break it out later (todo make issue for that if we want)
we'll want to extend the existing L1 feed UI to show bars and/or lines at each book-price level (this could also get switched later to a curve like in cryptowatch if peeps prefer that)
notice the note at the end that will possibly be worth follow up:
In the future I plan on porting such visualization to react-stockcharts so I can interactively explore data. The algorithms are already here, what is left to be done is changing matplotlib calls to d3. If you would like to collaborate with me on this, please email me.
data-layerreal-time and historical data processing and storageintegrationexternal stack and/or lib augmentations(sub-)systemsgeneral sw design and engtsdbtime series db stuff
Depth of book, otherwise known as L2 data. is something that's been on the list todo for quite a while.
We now not only have the data processing and streaming infra for it but also the UI components to display it efficiently in real-time on charts.
I originally had though about collecting and storing L2 using
marketstore
tick records as per #93 and #142. When you look at our tick schema it makes you think, ok so we need to store info at each level and I guess this goes all in one time series array?, well basically except there's already a project with a superior technique.The forward gist is that
techtonicdb
is a much better fit with it's super compact DFT format:(timestamp, seq, is_trade, is_bid, price, size).
Component design, subsystem todos
techtonic
over TCP as per the messaging spectrio
-fied client to do this (which should be trivial usingtrio.SocketStream
).Further resources
techtonicdb
repopandas
andmatplotlib
techtonicdb
rust stackffi.py
script for reading raw data from the db's storage fileThe text was updated successfully, but these errors were encountered: