I'm trying something like extracting audio features for each video frame. I know there are 30 video frames and 16000 audio frames per second in the video file. I'm using pyAudioAnalysis python lib to achieve the goal but no success. Here is my code.
from __future__ import print_function from pyAudioAnalysis import audioBasicIO from pyAudioAnalysis import ShortTermFeatures,MidTermFeatures import matplotlib.pyplot as plt import os,shlex, subprocess import pandas as pd import numpy as np # -acodec pcm_s16le -vn -ar 16000 command_line = "ffmpeg -i test.mp4 -ac 1 -ar 16000 -vn test_mono.wav" #command_line = "ffmpeg -i test.mp4 -ab 160k -ac 2 -ar 44100 -vn test_stereo.wav" args = shlex.split(command_line) print(args) processResult = subprocess.call(args) # Success! print(processResult) [SamplingRate, signals] = audioBasicIO.read_audio_file("test_mono.wav") print(SamplingRate)
['ffmpeg', '-i', 'test.mp4', '-ac', '1', '-ar', '16000', '-vn', 'test_mono.wav'] 1
#mid_feature_extraction(signal, sampling_rate, mid_window, mid_step,short_window, short_step): MidFeatures,ShortFeatures,MidFeatureLabels=MidTermFeatures.mid_feature_extraction(signals, SamplingRate, 0.043*SamplingRate, 0.043*SamplingRate,0.00016*SamplingRate, 0.00016*SamplingRate) print('Mid Features Extr Success') MidFeatures_Dataframe = pd.DataFrame(data=MidFeatures.transpose(), columns=MidFeatureLabels) print(type(MidFeatures)) #print(MidFeatures) print(type(ShortFeatures)) #print(ShortFeatures) print(type(MidFeatureLabels)) #print(MidFeatureLabels) MidFeatures_Dataframe.to_csv('Audio_MidFeatures.csv') print('Mid Features File Success')
Mid Features Extr Success
Mid Features File Success
As per my calculations I should get 338 rows of audio features, and after a long time of struggle I'm getting 326 with the above parameters but still don't know how. If anyone can help me how, window and steps are working here. I know the basic concepts of window and step as work in CNN but not getting in this context.