The Oxford dictionary explains "interleave" as
Place something between the layers of (something)
or in the context of telecommunication, as
Mix (digital signals) by alternating between them.
In the FaceNet paper, Schroff et. al describe their first architecture as
multiple interleaved layers of convolutions, non-linear activations, local response normalizations, and max pooling layers
which means they create a CNN by "altering between" these layer types (e.g. Conv -> Pool -> Conv -> Pool -> ..
). Or with the wording of the first definition, you "place one layer" of each type "between" the other existing layers (i.e. start with a couple of Conv
layers, then place Pool
layer between two Conv
layers, and so on).
The final architecture would look somewhat like this:
Conv -> ReLu -> LRN -> Pool -> Conv -> ReLu -> LRN -> Pool -> Conv -> ...
So, this "interleaved" CNN is a network where the different layer types are applied in series, layer-by-layer.
As a comparison, in "inception" type CNN, you apply the different layer types in parallel (the graphic below is 1 layer):
+---> Conv 3x3 ---+
| |
prev layer ---+---> Conv 5x5 ---+---> next layer
| |
+---> Pool 3x3 ---+