This will become clearer if you try to visualize the convolutional filter functioning.
Let's say you have a filter that is extracting horizontal lines from the image. When the filter extracts the dominance of horizontal lines in the image and assign a value in the generated matrix. Now we have the weights of the horizontal lines after the operation of the filter. Among all the horizontal weights, the most significant will be the one that has maximum weights so, we apply pooling (Max-pooling in this case) and discard all the other weights to select only that one having the maximum impact. By doing so, the filter selects only the features having maximum wights and discard the remaining. As the significance of the required object is from the line having maximum weight, the transformation is invariant. This also makes to tensor smaller in size and the process becomes faster too.
Perhaps, you can also refer the CS231 for a better understanding of the explaination.