3.5.0
Overview
This release contains 64 commits since the last major release v3.4.0
. Many of which are significant improvements to Epic's node server functionality. The most significant of these changes are detailed below.
v3.5.0 includes a dramatic reduction in blockchain synchronization time, as well as effort involved. Previously, syncing a full node from zero took a very long time, and a lot of manual interaction. Users can now sync in a few hours, without intervention.
Approximate Benchmarking:
Prior full sync time: 72hrs, multiple manual restarts needed
New full-sync performance: 1hr 58min, no intervention necessary
This performance improvement is a combination of work done by:
- @johanneshahn in writing a much more performant replacement for Grin's
HeaderSync
code, which preserves slowsync backwards compatibility - And work by @who-biz toward a significant overhaul of
BodySync
phase
None of the differentiating code is present in any upstream codebase.
Note: The above benchmark used the "least PoW verification" configuration, on a dedicated Linux server. I.e. skip_pow_validation = true
and disable_checkpoints = true
. Please see changelog below for disambiguation on what these options do (and what they do not). Total synchronization time will vary, based on hardware specs.
What's Changed
-
Fixes config file settings for
peer_max_count
by @who-biz (4966bb9)- The correct settings of
peer_max_inbound/outbound_count
are now detailed properly in generatedepic-server.toml
- The correct settings of
-
Resolves compiler warnings by @who-biz (acc157e, 2ab41cf, 3b58407, 88743b1, 4179d5b, 569961e, f1721fd)
-
Adds hard-coded
BlockchainCheckpoints
, andskip_pow_validation
config setting by @who-biz (3163ac8)- A checkpoint is present for every 200,000th block in long-term blockchain history, with a final checkpoint at height 2.2M
- Setting
skip_pow_validation = true
in server settings will skip pow validation for all blocks prior to latest checkpoint. - This setting is toggled
false
by default.
-
Adds ussage of GitHub actions/badges and CI for PRs, commits to master branch by @who-biz (5c65fa2)
-
Adds
disable_checkpoints
setting to config options , refactor checkpoint code by @who-biz (bcf0ecc)- Has no effect if
skip_pow_validation
is not set totrue
(double safety-switch) - Allows users who specify
true
for both settings to skip PoW validation for the entirety ofHeaderSync
phase
- Has no effect if
-
Resolves
ScrollBase
deprecation in cursive crate - commits taken from upstream Grin by @who-biz (17bac4a) -
Fixes infinite loop for reorg case in
FastHeaders
logic by @who-biz (b77a741)- In cases of reorg, peers send us the last header that matches our history. This case was not handled in code, and resulted in an inability for
HeaderSync
to progress - This case is now handled gracefully
- In cases of reorg, peers send us the last header that matches our history. This case was not handled in code, and resulted in an inability for
-
Updates server dependencies (tokio, hyper, and many more) by @johanneshahn (d135732, d899850, 65783a3, 236ff25)
-
Fixes
cucumber
testing suite by @johanneshahn (88120db) -
General code cleanup following dependency upgrades by @who-biz
-
Adds a dynamic threshold to force full PoW verification during
BodySync
of last1000
blocks by @who-biz (cc05f3d, bfb0924) -
Adds threshold for ignoring new block/header broadcasts while syncing by @who-biz (4f17617)
-
Enables server to switch from
BodySync
back toHeaderSync
phase by @who-biz (59f5036, 9aada0b)- Previously, once
HeaderSync
phase had completed, code did not support switching back to this phase - Due to work-intensive PoW, and poor
OrphanBlockPool
handling (Grin),BodySync
phase was extremely slow - If many new blocks were found during this period, we could not switch back to
HeaderSync
to fetch them - Logic has now been added to clear
OrphanBlockPool
and reset all states so thatHeaderSync
may resume from a clean state
- Previously, once
-
Greatly improves handling of
OrphanBlockPool
, reducing sync time & cpu usage greatly by @who-biz #105OrphanBlockPool
is essentially used as an LRUCache during blockchain synchronization, when we receive blocks out-of-order.- Previously,
check_orphans
function had looping logic that would potentially execute 1000+ times, before function exit - Due to
OrphanBlockPool
being aRwLock<HashMap>
type - this was blocking reads/writes while queue continued to grow - Some of this looping logic is useful, so we have spliced
check_orphans
intocheck_orphans_loop
andcheck_orphans
check_orphans
now checks for only a single orphan at a time, minimizing prohibitive locking- When
OrphanBlockPool
grows large, we call the looping function to quickly depopulate it, with an earlybreak
to prevent problematic locks - Finally,
MAX_ORPHANS_COUNT
was decreased to 60 - again, to minimize prohibitive locking
-
Improves efficiency of peer comms, with less spammy block requests (0991826, 6603488)
Note: More information on the BodySync
improvements is available here: #105
More information on HeaderFastSync code can be found in commits located here: #89
Full Changelog: v3.4.0...v3.5.0