I'm working on a project involved with identifying different types of sounds (such as screams, singing, and bangs) from each other. We've got our data a reasonable number of different transformations (e.g.: spectrograms, chromagrams, MFCCs, etc.), but since most of our features are 2-dimensional matrices (some are actually 1-dimensional vectors), we'd like to reduce this information in some way, so that the machine learning we're hoping to do takes a "feasible" amount of time. However, I don't quite know enough about math and statistics to make an educated decision on this.
Our data consists of small sound files from ~1-10 seconds long. There are recordings of screams, singing, bangs (and other man-made noises), and birds (and other natural noises). We are hoping to be able to differentiate and identify each source type from the others. See https://github.com/BenSandeen/surveillance_sound_classifier/blob/master/Project.ipynb for the different plots we have made to guide our selection of features to use. Focus mainly on the 3x3 plots, as that's where the comparisons are being made. These plots are primarily time vs. frequency, with amplitude represented by color.
I was thinking that maybe we could "collapse" each matrix down to a vector by somehow choosing some representative frequency/amplitude-related feature at each time slice (we're using Short Time Fourier Transforms to analyze the sounds) and then get a vector of some length, containing a bunch of scalars. Although this could make accounting for different lengths of sounds difficult. Would it be reasonable to just set the shorter sounds' vectors to be filled in with zeros if they have no useful data? That would effectively make these sounds a projection onto some lower-dimensional space. Then, maybe we could just use dot products to compare the vectors; if they're parallel, they'll have a large for product, but of they're nearly perpendicular, they'll have a dot product near zero.
Alternatively, I was thinking that something like a trace of our matrices, or finding their characteristic polynomial, might be useful direction to pursue. I've read a bit about PCA, but I'm not quite understanding it enough to know if this may be what I'm looking for.
Can anyone think of any other ways of handling and reducing this data? For what it's worth, we're currently planning on using Sci-Kit Learn (sorry that I can't use more than 2 links yet) to perform our machine learning.