0
$\begingroup$

I am trying to find the instances in a source audio file taken from a badminton match where a shot was hit by either of the players. For the same purpose, I have marked the timestamps with positive (hit sounds) and negative (no hit sound: commentary/crowd sound etc) labels like so:

shot_timestamps = [0,6.5,8, 11, 18.5, 23, 27, 29, 32, 37, 43.5, 47.5, 52, 55.5, 63, 66, 68, 72, 75, 79, 94.5, 96, 99, 105, 122, 115, 118.5, 122, 126, 130.5, 134, 140, 144, 147, 154, 158, 164, 174.5, 183, 186, 190, 199, 238, 250, 253, 261, 267, 269, 270, 274] 
shot_labels = ['no', 'yes', 'yes', 'yes', 'yes', 'yes', 'no', 'no', 'no', 'no', 'yes', 'yes', 'yes', 'yes', 'yes', 'no', 'no','no','no', 'no', 'yes', 'yes', 'no', 'no', 'yes', 'yes', 'yes', 'yes', 'yes', 'yes', 'yes', 'no', 'no', 'no', 'no', 'yes', 'no', 'yes', 'no', 'no', 'no', 'yes', 'no', 'yes', 'yes', 'no', 'no', 'yes', 'yes', 'no']

I have been taking 1 second windows around these timestamps like so:

rate, source = wavfile.read(source) 
def get_audio_snippets(shot_timestamps): 

    shot_snippets = []  # Collection of all audio snippets in the timestamps above 

    for timestamp in shot_timestamps: 
        start = math.ceil(timestamp*rate)
        end = math.ceil((timestamp + 1)*rate)
        if start >= source.shape[0]: 
            start = source.shape[0] - 1

        if end >= source.shape[0]: 
            end = source.shape[0] - 1  

        shot_snippets.append(source[start:end]) 
        
    return shot_snippets

and converting that to spectrogram images for the model. The model doesn't seem to be learning anything with an accuracy of around 50%. What can I do to improve the model?

The audio file: Google Drive

The timestamps labels: Google Drive (NEW)

Code: Github

Note: The timestamps file above was made recently and hasn't been used in the code above as I don't exactly know what window sizes to take for labelling purposes. The annotation file above has all the timestamps of hitting the shots.

Improve this question
$\endgroup$
8
  • $\begingroup$ When you annotated the audio, did you do it based on audio only, or while watching video? Cause some of the shots are not so easy to hear for me (though I am not very competent with badminton) $\endgroup$
    – Jon Nordby
    Dec 1, 2022 at 19:31
  • $\begingroup$ I checked the annotations. The durations (end-start columns) are often 0.3 second or longer. That seems rather long for what is essentially impulsive sounds. How is the "end" of the shot defined? Are the onsets precisely labeled with the start label? $\endgroup$
    – Jon Nordby
    Dec 1, 2022 at 19:33
  • $\begingroup$ The annotation was done purely based on audio, not referencing the video. Yeah, some of the shots were barely audible (drop shots close to the net in most cases). In such cases, we did not label them. As for what defines the "end" of the shot, for some shots, we extended the end duration a little bit to capture the echo of the shot of the sound as well. The onsets are labelled precisely I think. However, in many instances when the commentator's voice is dominating the shot sounds, we did not label those instances. $\endgroup$
    – ChaoS Adm
    Dec 2, 2022 at 5:00
  • $\begingroup$ Also, I am confused about the window sizes to take for the spectrograms. Since the durations of these new labels are different from each other (anywhere between 0.1 and 0.3 seconds), do we fix a window size that ranges from the start of the audio file to the end? In such a case, what do we do if the same label spans across two different windows (assuming the window sizes are fixed)? $\endgroup$
    – ChaoS Adm
    Dec 2, 2022 at 5:32
  • $\begingroup$ I think you should label each shot, and then use the label to indicate the cases you indicate. For example use "clear", "faint", "inaudible", "masked" (by annotator or other sound). Then one can do error analysis on these things, if needed. And if one wants to focus just on the clear cases (like now), then that is done just by using the "clear" labels. $\endgroup$
    – Jon Nordby
    Dec 2, 2022 at 10:51

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.