Improving Signal Processing with Movement Artifacts and Peak Correction

[DanDKL]

Hello, Neurokit2 community,

I'm currently using the Neurokit2 library for Heart Rate Variability (HRV) analysis on long recordings (5 to 15 minutes), and I'm finding it to be an excellent resource. However, I've encountered a challenge I'm hoping someone here can help me with.

The ECG data I'm working with occasionally has significant movement artifacts that seem to be causing some issues in peak detection and correction. I've noticed that the R-peaks are often misplaced during these artifact periods, which subsequently impacts the HRV analysis.

I've included the code I'm currently using below:

import os
import mne
import pandas as pd
import neurokit2 as nk
import matplotlib.pyplot as plt

# Define the directory where your ECG files are
directory = "C:/dir"

# Loop through each file in the directory
for filename in os.listdir(directory):
    try:  # Error handling
        if filename.endswith(".edf"):  # Check that the file is an EDF
            # Load the data
            raw = mne.io.read_raw_edf(os.path.join(directory, filename))
            data = raw.to_data_frame()
            
            # Check that the "ECG" channel exists
            if "ECG" in data.columns:
                # Preprocess the data
                clean_signals, info = nk.bio_process(ecg=data["ECG"], sampling_rate=1000)

                # Assess signal quality
                quality = nk.ecg_quality(clean_signals["ECG_Clean"], sampling_rate=1000, method="zhao2018", approach="fuzzy")

                # Identify R-peaks
                rpeaks_info = nk.ecg_findpeaks(clean_signals['ECG_Clean'], sampling_rate=1000)
                plt.figure(figsize=(12,3))
                plt.plot(clean_signals["ECG_Clean"])
                plt.scatter(rpeaks_info['ECG_R_Peaks'], clean_signals["ECG_Clean"][rpeaks_info['ECG_R_Peaks']], color='red')
                plt.title('R-Peaks Identified')
                plt.show()
                
                # Correct peaks
                artifacts, rpeaks_corrected = nk.signal_fixpeaks(rpeaks_info['ECG_R_Peaks'], iterative=True, method="Kubios")
                plt.figure(figsize=(12,3))
                plt.plot(clean_signals["ECG_Clean"])
                plt.scatter(rpeaks_corrected, clean_signals["ECG_Clean"][rpeaks_corrected], color='red')
                plt.title('Corrected R-Peaks')
                plt.show()

                # Use corrected R-peaks to replace R-peaks in 'info'
                info['ECG_R_Peaks'] = rpeaks_corrected
                
                # Plot the processed ECG signal along with the detected R-peaks
                nk.ecg_plot(clean_signals, sampling_rate=1000)
                plt.show()

                # Compute HRV indices
                hrv = nk.hrv(info, sampling_rate=1000, show=False)
                
                # Add the filename as the ID
                hrv["ID"] = filename
                
                # Add the quality assessment to the DataFrame
                hrv["QualityIndex"] = quality
                
                # Append the HRV results to an existing CSV file (or create a new one if it doesn't exist)
                with open('hrv_results.csv', 'a') as f:
                    hrv.to_csv(f, header=f.tell()==0, index=False)

    except Exception as e:
        print(f"Error processing {filename}: {e}")

I've also attached some plots that illustrate the problem:

As you can see from these plots, the R-peaks are not accurately detected during periods with significant movement artifacts.
I've tried various approaches to handle this, including applying a bandpass filter, smoothing the signal, and adjusting the threshold for peak detection, but none of these methods seem to completely solve the issue.

Does anyone have suggestions or recommendations on how I might better handle these artifacts to improve R-peak detection and correction? Any input or guidance would be greatly appreciated.

Thanks for your help!

[DominiqueMakowski]

Could you re-do this plot but only in this chunk (so we can compare a few "good" peaks with the noisy segment?

What is are the current frequencies of the bandpass filtering?

[DanDKL]

Here are the zoomed plots (identified and corrected). The corrected actually correct the erroneous marking on the end right. But the motion artifact remains problematic.

Regarding the current frequencies of the bandpass filtering, I did a bunch of tests with various frequencies but in the end, I revert back to the original pipeline of the MDPI paper with nk.bio_process. I've just added the kubios correction, hoping to improve the results. I do not know if this is the best approach though.

[DominiqueMakowski]

Mmh yeah, tricky. I can hardly identify the peaks by eyes in the signal, which often means no matter what algorithm you use the info is just lost to noise.

I would try explicitly filtering the signal before passing it to _process() (using signal_filter()) and trying different bandpass values and see how it affects this plot.

Otherwise, I would somehow mark noisy segments and then remove them or something (depending on what you want to do with the ECG)

[DanDKL]

You're right, the signal during these artifact periods is quite noisy and difficult to interpret. My main objective isn't necessarily to identify the peaks within these noisy segments but rather to process the ECG data as accurately as possible while handling these artifacts appropriately. The recordings I'm working with are fairly long (around 15 minutes each), so I'm perfectly fine with excluding sections that are too noisy.
I agree with your suggestion of marking noisy segments and removing them, do you know of an efficient way to automatically identify and exclude these noisy segments within NeuroKit2 or another library? I'm interested in implementing a solution that can identify periods of high noise or artifacts and exclude them from the analysis.

Thanks again for your help.

[DominiqueMakowski]

I am not aware of any implemented methods to detect that, though I agree it'd be useful.

I suspect some rolling measure (over chunks of ~1 or 2 seconds) of 1) variance alongside the raw ECG and/or heart rate signal could be useful, 2) as well as of complexity (to give an index of "randomness"), or a combination of these.

If you try / find something in the literature or in existing implementations, please don't hesitate to open a new issue on "Automated detection of noisy periods in ECG" and we can discuss it there and eventually add something to neurokit :)

[DanDKL]

Thanks for sharing these thoughts! Your point about using a rolling measure of variance and complexity certainly got me thinking. Interestingly, Kubios HRV does have an automatic artifact correction feature. I'm not sure if it could serve as a foundation, but I'm currently testing it and it seems to work pretty well (here). However, I'd prefer not to rely on paid, closed-source software, and would love to transition entirely to a solution like NK2 for this kind of processing.