1
$\begingroup$

Below is one example Attention-based Encoder-decoder network for multivariate time series forecasting task. I want to visualize the attention weights.

input_ = Input(shape=(TIME_STEPS,N))
x = attention_block(input_)
x = LSTM(512, return_sequences=True)(x)
x = LSTM(512)(x)
x = RepeatVector(n_future)(x)
x = LSTM(128, activation='relu', return_sequences=True)(x)
x = TimeDistributed(Dense(128, activation='relu'))(x)
x = Dense(1)(x)
model = Model(input_,x)
model.compile(loss="mean_squared_error",optimizer="adam",metrics=["acc"])
print(model.summary())

Here is the implementation of my attention block:

def attention_block(inputs):
    x=Permute((2,1))(inputs)
    x=Dense(TIME_STEPS,activation="softmax")(x)
    x=Permute((2,1),name="attention_prob")(x)
    x=multiply([inputs,x])
return x

I will highly appreciate if a fresh implementation of the attention model is provided.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

A reference: https://github.com/zhaocq-nlp/Attention-Visualization/blob/master/exec/plot_heatmap.py. It plots attentions based on matplotlib

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review $\endgroup$
    – Ethan
    Jun 24, 2022 at 18:31
  • $\begingroup$ I need the attention values, i.e. the probability values, and if you have any example of time series forecasting, that helps. Thank you. $\endgroup$
    – Debashis Sahoo
    Jul 28, 2022 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.