1
$\begingroup$

suppose I have an autoencoder as a two-stack LSTM that takes in sequences of $n$ features of some length $m$.

Let's say that the dimension of my encoding vector is $k$, so the architecture is of the form: $n \times m \to 1\times k \to n \times m$.

I'm looking into how to construct some explainability metrics on the encoding part of the autoencoder. More specifially, I'd like to know which features impact each of the $k$ encoding entries I have.

Naively, one could vary one feature at a time, check the impact on the encoding and see where it is greatest. This is both computationally expensive and neglects combinations of features.

Do you know of any research or methods that can assist?

$\endgroup$
1
  • $\begingroup$ Here are my thoughts, not sure any of this would work. It seems to me what you are trying to get is something like an attention mecanisms, so maybe you can have a quick look at transformer mecanisms, but as they are quite complicated, it would be more interesting to focus on something like squeeze and excitation layers that create some kind of weights for each map in CNN (not exactly your case, but may be close enough to apply the idea). I also heard about using PCA on the k space to extract human understandable ideas from it, dunno if it can help you there. $\endgroup$
    – Ubikuity
    Jul 1, 2021 at 13:51

1 Answer 1

1
$\begingroup$

Feature visualization/inversion might be a good tool here. Basically, instead of performing optimization on the weights, you perform an optimization on the inputs that maximizes each respective entry of your encoding vector. If you want to get fancy, you might try constructing an entire activation atlas.

The interpretml library also has a lot of good tools for model explainability, but they are more geared towards supervised models. Might still be worth checking out.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.