How to get Euclidean Distance between feature maps

Question

I'm trying to find "keyframes" within a video using this paper however I'm a little new to machine learning and I'm stuck on the distance vector step. The goal is to calculate the distance vector between consecutive frames to detect a big context change indicating a keyframe. here's what I've done:

Using Googlenet pre-trained model I extracted a (1024,7,7) feature map, so a vector of 7x7 matrices for every other frame in the video.
Now I want to calculate the euclidean distance between two consecutive frames but I'm not sure how that would be.

My intuition is that the distance vector would also be (1024,7,7).
The last step would be to apply a "convolution of 4-window of last distance values with vector [0.1, 0.1, 0.1, 0.99]" on the distance vector?? I don't understand this step either.

Any help or guidance would be appreciated!

Answer 1

The Euclidean between two images $p$ and $q$ can be calculated as follows:

$d(p, q) = \sqrt{(q_1 - p_1)^2 + (q_2 - p_2)^2 + ... + (q_{49} - p_{49})^2}$

which is the distance between the 49 (7x7) features of the two images.

This should then give you a vector of shape (1024, 1) where each value is the Euclidean distance of the feature maps of the previous image, with the first one being all NA since it's the first image. Then a convolution is applied with a window of 4 with the vector/kernel [0.1, 0.1, 0.1, 0.99], which basically multiplies 4 values of the (1024, 1) vector by the kernel values and adds them up. I.e. given the first 4 values of the (1024, 1) vector of [0.3, 0.5, 0.2, 0.4] the result of this multiplication would be:

$0.3 * 0.1 + 0.5 * 0.1 + 0.2 * 0.1 + 0.99 * 0.4 = 0.03 + 0.05 + 0.02 + 3.96 = 4.06$