GUI freezes after 3.5min of continuously reading real-time data

[highohm]

Hi all,

I have a Java application with JFreeChart library reading a real-time data stream over network from a controller over UDP-protocol. The data stream feeds in up to 128 signals in parallel with fixed 10ms cycle-time. The data stream is streamed to a file and in parallel is fed into TimeSeries with function addOrUpdate(). For each signal one TimeSeries. Over buttons you can select/deselect each signal to be displayed on a common chart with XYplot. Each TimeSeries is limited to 60s/60.000 values/items. When signals are displayed the chart is updated 300ms cyclically.

As long as I use the old library JFreeChart-1.0.7 everything works fine. No freeze but of course the app gets slowlier the more signals are displayed and the more items are in the time domain.

When I use newer JFC library, e.g. 1.0.14 or 1.5.3, currently 1.5.3, the GUI freezes after 3.5minutes and the CPU-load rises from 5% to 15%. This happens when the data stream feeds in at least about 25 signals in parallel with function addOrUpdate() into the TimeSeries. It does not matter if these signals are displayed on the chart or not. The control buttons still work, but have a long response delay, about 20s. So if I stop calling addOrUpdate(), the GUI 'wakes up' again slowly and after some time it is fully responsive again. It feels like if there is a calculation queue somewhere inside which gets saturated.

I use Windows 10 with openjdk-13.0.2.
While GUI freeze the data stream into file still is working and the memory consumption remains almost constant.

Does anyone have an idea, what the reason for the freeze is and how I can fix it?

Thank you
Christian

[trashgod]

All supported host platform GUIs are single threaded, so saturation is inevitable. SwingWorker limits updates to 30Hz, while a JavaFX Task can do 60Hz.

[highohm]

Thanks for your quick answer.

But now I have found the root cause of the problem:
My info about the item length of the TimeSeries was wrong. The actual limits are 200.000 items. This matches the freeze time of about 3,5min. So when the limits are reached the TimeSeries begin to remove the oldest items (call of removeAgedItems(false); within TimeSeries::addOrUpdate()). In that function the MinY and MaxY are recalculated in the subcall updateMinMaxYByIteration(). This update takes significant CPU-time dependent on the number and length of the TimeSeries since it iterates through all items on every call. Despite the CPU usage was not more than 15% while freeze the updateMinMaxYByIteration() somehow blocked the GUI.
After I commented out the call updateMinMaxYByIteration(), the GUI did not freeze any more.

This also explains why the GUI does not freeze with the old library JFC-1.0.7, because it does not have these MinY and MaxY variables and the update function updateMinMaxYByIteration().

I think about a more run-time efficient implementation of the updateMinMaxYByIteration(). But I have no good idea. An idea is to add a
a 2nd linked list in parallel to the TimeSeries::data that is sorted by the Y range-values. But this would take significant more memory usage with regard to my TimeSeries setup.

So far commenting out updateMinMaxYByIteration() works for me.

Best regards Christian

[trashgod]

Glad you found the bottleneck. I'm not sure how your application is synchronized, but both SwingWorker and Task can coalesce events to save on notifications. Similarly, you may be able to use the TimeSeries method, setNotify(), to optimize notifications.

[highohm]

Thanks. Regarding the synchronization:
The network datastream is received in a dedicated thread which transfers the data over the following simplified construct to the TimeSeries:

public void SetData(...) {
	EventQueue.invokeLater(new Runnable()
	{
		public void run() {
			...
			series.addOrUpdate();
			
			filestream out...
		}
	}
}

I also commented out the call to fireSeriesChanged() within series.addOrUpdate(). But this apparently did not take any effect.

The GUI chart update is controlled with another dedicated thread, that is called cyclically with a fixed cycle-time (300ms).
On every call it executes series.setNotify(true). This also calls fireSeriesChanged() as in addOrUpdate(). But apparently it only takes effect in the cyclic thread.
cyclic thread:

public void run()
{
	while(true)
	{
		EventQueue.invokeLater(new Runnable()
		{						
			public void run() 
			{
				for(int i=0;i<series[iChart].length;i++)
					series[iChart][i].setNotify(true);
			}
		});
		Thread.sleep(this.iCyclicDelay);
	}
}

[trashgod]

I would urge you to consider migrating to SwingWorker, seen here. It provides a clean separation between data acquisition (doInBackground()) and view update (process()). In process(), you can turn off notifications, loop through the coalesced chunks, and then enable notifications.