Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

Filter by
Sorted by
Tagged with
0
votes
2answers
10 views

How to set vocabulary size, padding length and embedding dimension in LSTM network?

Usually in a LSTM network, we have certain parameters that need to be set before the model can begin training. I am specifically talking about vocabulary size, padding length and embedding dimension. ...
0
votes
0answers
7 views

How does ConvLSTM layer calculates the output

I'm building ConvLSTM for forecasting weather, but I'm not able to understand the math calculation behind the layer. Like in simple Conv layer we simply do the dot product of input with the kernel. ...
0
votes
0answers
14 views

What have my models learnt?

I am doing a time series classification task. I used LSTM, Bi-LSTM. Bi-LSTM works a little bit better than single layer LSTM. And concatenating two Bi-LSTM outputs with another input gives me a better ...
1
vote
1answer
7 views

Dimension error when tuning LSTM layer

I am working on a sentiment analysis problem which is a binary classification. These are some of the parameters that might be useful: 1.) Length of train list = 203 2.) Length of test list = 51 3.) ...
0
votes
0answers
22 views

Where to start with ChatBots?

I want to start my journey into ChatBots and how I can create them. I have read some articles regarding the type of chatbots. Basically there are 2 types of chatbots, one is a rule based and the other ...
0
votes
0answers
9 views

give a 0 loss to predictions under some conditions

I am using CNN LSTM to predict a continuous quantity A one step ahead using features A and other features B, C and D with a lag of 10. I have 15000 points in my datasets. I use the first 10000 to ...
0
votes
0answers
7 views

Best loss function for baseN encoding LSTM model

everyone, I am trying to train an LSTM model for sequence prediction. I have X and Y which are two numpy arrays. X is a list of integers (integer encoded strings) while to encode Y I used a BaseN ...
1
vote
1answer
94 views

LSTM with input of actual time step

I'm working on an implementation of LSTM neural network to forecast energy consumption. I have a dataset with load, series of weather parameters and indicator of it's bank holiday or not. I first ...
2
votes
1answer
294 views

should I shift a dataset to use it for Time series regression with RNN/LSTM?

I'm seeing this tutorial to know how to use LSTM to predict time series data and I noticed that he shifted the target/labels up so that the features are all in time t but the target is t+1 so my ...
0
votes
0answers
42 views

TypeError: 'float' object is not subscriptable

I am following this code: https://machinelearningmastery.com/evaluate-performance-deep-learning-models-keras/ and more specifically: Manual k-Fold Cross Validation part. I get this error: ...
1
vote
1answer
635 views

Error on custom RNN/LSTM with multiple inputs

I want to implement a custom RNN/LSTM model similar to this. The model should take two separate vectors as input and process them. I was following keras tutorial to implement a custom keras layer and ...
0
votes
1answer
33 views

Can RNN learn for each `t` in time from a whole new dataset (many entries)

Basically, my data set is not as simple multi-variate time-serie as it's often (to some extent) the case. For each month, I have N entries (not less than 3000). Can RNN of any variant (Please bear my ...
1
vote
2answers
385 views

Keras data structure for LSTM-networks

After reading a while, I am confused now about my LSTM data structure. Assuming that I have a supervised learning problem with 1000 samples and 40 features as input. Now I want to create 10 timesteps ...
0
votes
1answer
12 views

What model should I use to predict a time series like this?

This series is calculated from the difference of two day's stock index. I rescaled it using sklearn's StarndardScaler. It seems LSTM does not work well on this series.
0
votes
0answers
15 views

LSTM for non consecutive lags

Let's suppose I have a time series with hourly data. Firstly, I was using the previous 168 values aka lags/timesteps to forecast current value, i.e, I was trying to learn the following F function $X_t=...
1
vote
1answer
48 views

Working Behavior of BERT vs Transformers vs Self-Attention+LSTM vs Attention+LSTM on the scientific STEM data classification task?

So I just used BERT pre-trained with Focal Loss to classify Physics, Chemistry, Biology and Mathematics and got a good f-1 macro of 0.91. It is good given it only had to look for the tokens like ...
0
votes
0answers
5 views

How to pass multiple vectors to a RNN/LSTM network and get output as a vector. Can someone explain or give reference to code/text

I need to feed multiple vectors to a RNN/LSTM and get a vector as output utilizing dependencies between the vectors . How do i pass the vectors . Is there any code/reference ?
0
votes
1answer
11 views

Input- and Output Data Shape Difficulty

I'm a Keras beginner. My main problem right now is how to build a model that suits my data. For the Model itself I'd like to build it so the inputs/outputs are: Input Data: (List that contains) three ...
2
votes
1answer
393 views

How to represent the number of neurons in an LSTM for architecture schematic?

I'm trying to visualise a neural network schematic and found a great tool for building schematics here http://alexlenail.me/NN-SVG/index.html. I've edited the SVG file to change one of the dense ...
0
votes
1answer
26 views

LSTM internals: How does an LSTM handle dimension reduction?

I trained a Bidirectional LSTM of width 384 which took in an input of dimension (None, 4995, 12), using Tensorflow/Keras. I've been trying to understand how my input dimension is mapped to the $x_t$ ...
1
vote
2answers
158 views

MY lstm has a really low accuracy, is there anyway to improve it?

I am trying to make a model to classify whether these patients can be diagnosed with dementia by their 35 days of biometric data. A brief summary of a dataset is below. as an input X_train data, it ...
0
votes
0answers
12 views

Hidden state dimensions in Pytorch LSTM

Please read the question completely before you mark it as duplicate I was trying to understand the syntax of using an LSTM in PyTorch. I came across the following in PyTorch docs. ...
1
vote
2answers
139 views

Intuition behind the RNN/LSTM hidden state?

What's the intuition behind the hidden states of RNN/LSTM? Are they similar to the hidden states of HMM (Hidden Markov Model)?
0
votes
1answer
152 views

Advantages of CNN vs. LSTM for sequence data like text or log-files

When do you tend to use CNN rather than LSTM (or the other way round) in classification or generation tasks of sequential data like text or log-data? What are the reasons for the decision and what ...
0
votes
0answers
23 views

Adding a static layer to LSTM text generation

I am using Tensorflow guide to build a LSTM text generation model (uses the keras functional API) (https://www.tensorflow.org/text/tutorials/text_generation) I would like to add an additional input ...
0
votes
1answer
15 views

Running model.fit multiple times for an LSTM?

I have time-series histogram data from many separate machine runs (see this post for detail). I am working to train an LSTM in order to predict the final histogram in a machine run based on the past ...
0
votes
3answers
3k views

LSTM Multivariate time series forecasting with multiple inputs for each time step

I want to predict an output variable for the next day, for each of the users in my dataset. I was thinking of using LSTMs for achieving this. The dataset The dataset I am using has multiple inputs ...
2
votes
1answer
3k views

Predicting next number in a sequence - data analysis

I am a machine learning newbie and I am working on a project where I'm given a sequence of integers all of which are in the range 0 to 70. My goal is to predict the next integer in the sequence given ...
0
votes
0answers
45 views

How to get fine-grained sentiment score from text data under unsupervised learning?

In my experience I have only used LSTM models to do sentiment classification tasks on text data under supervised learning. For example, the imdb dataset from keras which is a binary classification ...
2
votes
2answers
499 views

How similar is Adam optimization and Gradient clipping?

According to the Adam optimization update rule: $$m \leftarrow \beta_1 m + (1 - \beta_1)\nabla J(\theta)$$ $$v \leftarrow \beta_2 v + (1 - \beta_2)(\nabla J(\theta) \odot \nabla J(\theta))$$ $$\theta \...
0
votes
1answer
369 views

LSTM - How to prepare train from a dataset which contains multiple observations for different events

I m using LSTM in a project related to MobiFall dataset which contains falls and daily activitives - such as walking, sitting etc - sensed by accelerometer, gyroscope and orientation sensors in x,y,z ...
-1
votes
0answers
9 views

Training and Validation loss are same but not decreasing for LSTM model

I have a timeseries data and I am doing univariate forecasting using stacked LSTM without any activation function, Like following. ...
1
vote
1answer
124 views

How to train NER LSTM on single sentence level

My documents are only a single sentence long, containing one annotation. Sentences with the same named entity of course are similar, but not context-wise. NER ...
0
votes
1answer
882 views

Understanding dimensions of Keras LSTM target

I'm learning about Keras and LSTMs and came across this tutorial, but I don't understand the dimensions of the target variable. Quoting the article below: The training y data in this case is the ...
0
votes
0answers
17 views

LSTM advantages

Can someone briefly explain what does this (bold) mean: LSTM is well-suited to classify, process and predict time series given time lags of unknown size and duration between important events. Thank ...
0
votes
1answer
61 views

How to use LSTM for time series data?

I've an ECG data spread over time. The duration for each data is around 3 minutes (approx 180 seconds). Each second around 200 recordings were taken. So total length for each sample is approx 36000. ...
0
votes
1answer
223 views

Keras LSTM model not performant

I have trained a LSTM model to detect fake domain names. My dataset is like this: domain,fake google, 0 bezqcuoqzcjloc,1 ... with 50% normal and 50% fake ...
3
votes
1answer
61 views

Predicting t+1 from a set of sequences

Say I have have an experiment where I release a single rat into a maze and wait for it to reach the end. Say I also track this rat's position in the maze at various times. Let's do this $n$ times. Now,...
6
votes
1answer
254 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
3
votes
2answers
89 views

Running an LSTM with Music Data

I'm working on a project for a class where I'm trying to create an algorithm that learns music and creates its own music. I'm having trouble on how to set up the data for it to be inputted into the ...
0
votes
1answer
85 views

Encoder-Decoder performance time

I have two encoder-decoder models. *First model: *Second model: When I check the performance of the models I get approximately the same performance time (First model ~ 42 sec, Second model ~ 40 ...
2
votes
1answer
47 views

LSTM for time series forcasting

I manipulate the time series using the different structures of the neural networks in order to make a prediction, and I wonder if there is a way to choose the parameters of the networks intelligently? ...
2
votes
1answer
1k views

Metrics for presenting RNN/LSTM result

I am working on two different architectures based on the LSTM model to predict the user's next action based on the previous actions. I am wondering, what is the best way to present the result? Is it ...
1
vote
0answers
61 views

Keras model prediction always has unwanted offset

I am trying to predict next 10 days by looking into the last 60 days. So tried to implement an LSTM layer. Before jumping into the question, I want to clarify a few points. Firstly, this is a Multiple ...
0
votes
1answer
49 views

Time series forecast for everyday for till a distant future

I have time-series data for every single day from the last 5 years with seasonal variation and a general increase in trend. This is what my data looks like: And I am trying to predict for every ...
2
votes
1answer
129 views

Timeseries LSTM: does test data need to come after training data?

I have one single, very long time series. I want to train an LSTM to distinguish between two behaviours (A or B) at every timestep (sequence-to-sequence). Because the time series is very long, I plan ...
1
vote
1answer
120 views

How to predict past test set on time series data using LSTM

I'm trying to do a regression on some inventory amounts with the following model using Keras: ...
0
votes
0answers
5 views

Reference code for LSTM Variational Autoencoder for dimensionality reduction

I have time series data, with many features. I would like to reduce the dimentionality by using LSTM VAE. Does anybody know an example code or a reference to guide me to impolement it? Both Pytorch ...
0
votes
0answers
20 views

keras_self_attention vs attention layer

can i get a help pleas what is the diffreent btween if i bulid attention class like this to my Bi LSTM MODEL and keras_self_attention ? from tensorflow.keras.layers import Layer from tensorflow.keras ...
0
votes
0answers
6 views

1
2 3 4 5
21