Sensor fusion using recurrent neural network: obtaining a smoothed output

Question

I am trying to use a recurrent neural network to perform sensor fusion for an inertial measurement unit. IMUs are commonly used in conjunction with a Kalman filter (KF), which performs both fusion of accelerometer and gyroscope data along with 'smoothing', which results in a filtered, final output of orientation (and position if desired). Out of curiosity, I am trying to replicate this functionality using a neural network containing LSTM units: i.e., get the network to compute the hidden representation that Kalman filters estimate using training data. In order to get the network to replicate the working of a Kalman filter fusion algorithm, my first step was to train it on inputs of raw sensor data and the output coming from a KF.

The network takes 6x10 + 6 = 66 inputs (3 velocities from gyro, 3 accelerations from accelerometer from the present instant along with those from the past 10 instants of time) and aims to provide 4 outputs (quaternion representation of orientation). The network structure is

Input -> LSTM units (128) -> Dense layer (128) -> Dense layer (64) -> Output(4)

My initial training data consisted of noisy sensor data from the gyro and accelerometer in the input and smoothed KF orientation data as output, from which I was expecting the network to attempt to learn the nonlinear mapping between the sensors and the orientation. Yet, although the output I am training on is clean, there is a lot of noise in the network's predictions, even on the training data. For example:

Kalman filter output (used as output in training data):

Corresponding predictions from neural network:

The noise I am seeing in the predictions is not unnatural, because the input sensor data is very noisy as well, and the amount of noise in the output corresponds to that of the inputs: but is it not right to expect the neural network to 'learn' the smoothing as well, along with the fusion? Is there any way to reduce the noise and get the network to perform closer to the way the KF does?

Answer 1

To remove the noise I'd suggest you utilize Fourier Transformation to get the frequency spectrum of the signal. Then you can apply a threshold to remove high-frequency signal components which are mainly noises.

As I understood you are using python. The code below creates an artificial noisy signal and removes it using FFT (Fast Fourier Transformation) algorithm.

import numpy as np
import matplotlib.pyplot as plt

x = np.arange(0, 10, 0.01)

y = np.sin(x)
y_noisy = y + np.random.uniform(low=0, high=0.5, size=len(y))

fft_values = np.fft.fft(y_noisy)

mean_value = np.mean(np.abs(fft_values))

threshold = 10 * mean_value  # Fine-tune this

fft_values[np.abs(fft_values) < threshold] = 0

filtered_samples = np.fft.ifft(fft_values)

plt.plot(x, filtered_samples)
plt.show()

Good Luck!

Answer 2

I Would suggest you to take a look at encoder decoder network. AFAIK its being used in images to remove noises. Take a look at This paper once