Dear fellow data scientists
I work in finance and have decent experience with neuronal networks. Right now I am facing a challenge where I need to classify a set of signals.
The data set looks roughly as follows (signal can be thought of as a vector which might also contain floats instead of being binary):
Signal - Target
001110 0
001110 1
001100 1
011100 0
... ...
The problem is that the data is really noisy. That means that most of the signals do not really have a lot of predictive power to forecast the target. That makes using a neuronal network (or other classical classification algorithm) rather impractical since we will be fitting a lot of noise. At the same time, simply looking at correlations between signals and targets does not help, since the signals themselves are noisy. So e.g. [0,0,0,1,1,1] and [0,0,0,1,1,0] are not the same, but similar and both signals might be a very good predictor for, let's say, the target 1.
My goal is to have an algorithm that tries to find the 10 good signal "types" (in a fuzzy way) in the haystack of 1000s of signals. I am looking for suggestions as to what that algorithm could be.
One idea would be to first cluster the signals and then use the clusters as input, but maybe there are better suited algorithms exactly for such tasks.
Hope my question is clear. :)
