Node randomly starts to use all CPU available

[charlie-wasp]

What happened?

Hi!

I noticed that during time my IronFish nodes start to consume all available CPU.

For example here is one of the nodes' status:

:~# ironfish status
Version              1.1.0 @ 6dc93a4
Node                 STARTED
Node Name            ironfish
Block Graffiti       FISH
Network              Mainnet
Memory               Heap: 2.78 GiB -> 2.97 GiB / 3.98 GiB (69.7%), RSS: 4.58 GiB (14.8%), Free: 5.97 GiB (80.7%)
CPU                  Cores: 8, Current: 770.8%
P2P Network          CONNECTED - In: 1.66 MB/s, Out: 74.30 KB/s, peers 299
Mining               STARTED - 44 miners, 0 mined
Mem Pool             Count: 4 tx, Bytes: 16.99 KiB / 57.22 MiB (0.03%), Evictions: 0
Syncer               IDLE - 2.65 blocks added/sec
Blockchain           000000000000001ba30646e1726a21187d271babe38fb2acccfcf95625c6823d (18454), Since HEAD: 9s 154ms (SYNCED)
Accounts             0000000000000009293cf1f2e3c4cc2739771a02226689264f2c538206f9c2ea (18453)
Telemetry            STOPPED
Workers              STARTED - 45 -> 6 / 6 - 3.16 jobs Δ, 1.5 jobs/s

As you can see, it uses > 700% of 8 cores CPU. It seems to happen every 24-48 hours

Can you help me with that? Maybe, it is due to some misconfiguration?

Version

v1.1.0

Debug output

Database in use, skipping output that requires database.
Stop the node and run the debug command again to show full output.

Iron Fish version               1.1.0 @ 6dc93a4
Iron Fish library               1.1.0 @ 6dc93a4
Operating system                Linux x64
CPU model(s)                    Intel(R) Xeon(R) E-2274G CPU @ 4.00GHz
CPU threads                     8
RAM total                       30.98 GiB
Heap total                      3.99 GiB
Node version                    v18.16.0
ironfish in PATH                true
Garbage Collector Exposed       false
Telemetry enabled               false
Node name                       ironfish
Block graffiti                  FISH

Relevant log output

No response

Graffiti

No response

[holahula]

@charlie-wasp are you doing anything with your node? CPU usage is generally correlated with worker usage which you can check via ironfish workers:status. I personally have seen CPU usage that high when all the worker threads are actively decrypting notes / verifying transactions

[charlie-wasp]

@holahula mainly I use nodes for block templates streaming, all other requests are relatively insignificant

thanks for the tip, I'll check the workers 👍