1
$\begingroup$

A lot of studies feed MFCCs as well as MFCC delta and double deltas directly to a CNN for audio classification. My question is, are the MFCC Deltas concatenated with the MFCC matrix? Most papers simply state they used MFCC + MFCC Delta + MFCC Double Delta and the plus sign is left to interpretation!

$\endgroup$
2
$\begingroup$

Yes, the delta and delta-delta variants are concatenated. However the details may vary a bit based on model type:

If the model takes a 1d (features,) input (such as a multi-layer-perceptron, logistic regression, random forest etc), then the delta coefficients are concatenated. So features is [mfcc1,mfcc2...,dmfcc1,dmfcc2... ].

For a model that takes 2d (time,features) input like a RNN, then it is concatenated on the features axis.

For a model that takes 3d (time,features,channels) inputs like a CNN, then the delta coefficients are usually its own plane in the channels dimensions. This ensures that the delta MFCC coefficient is in the same time x feature position as the corresponding MFCC coefficient, which is easiest for the convolutional kernel to exploit.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.