Questions tagged [lstm]
LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.
810
questions
0
votes
1answer
9 views
Why is my LSTM is working best with batch size of 2 and no hidden layers?
I am building an LSTM for price prediction using Keras. I am using Bayesian optimization to find the right hyperparameters. With ...
0
votes
0answers
11 views
Inserting input representation at each step of LSTM
In want to train a neural net to generate lyrics based on a provided melody. For that, I have to implement a recurrent neural net (LSTM or GRU) to carry out the task. During each step of the training ...
0
votes
0answers
6 views
How exactly the hidden state works in an RNN ? How to decide on how many past instances to consider?
I am unable to grasp the working of RNN because in different tutorials, it is explained differently.
Please correct me as I have considered that:
In a ...
0
votes
0answers
7 views
How to make a multivariate forecasting if one of features becomes known for the future with some confidence level, e.g. weather forecast data
Let's assume that we make forecasting of another metric partially based on forecasts of the weather forecast, e.g. of temperature, pressure, then we can potentially obtain those forecasts from one of ...
0
votes
0answers
24 views
Why does the LSTM overfit all the time
I have a time series prediction problem from building an LSTM.
My code:
...
0
votes
1answer
18 views
What is the reasonable max number of features for LSTM?
I've been trying to find information on how many features I can use in LSTM.
I found that LSTM can handle up to 500-1000 timesteps, but what about features (sequences)?
Can I use 1000 features per ...
0
votes
0answers
10 views
How I can train BiLSTM model with CNN for semantic similarity?
I'm build a Deep Leaning model with BiLSTM and CNN for two text's semantic similarity. My data set is format as :
[s1,s2,is_similarity] with is_similarity is from 0.00 to 5.00. I want to create a set ...
0
votes
0answers
10 views
Is the number of bidirectional LSTMs in encoder-decoder model equal to the maximum length of input text/characters?
I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/.
It ...
1
vote
1answer
14 views
How to feed key-value features (aggregated data) to LSTM?
I have the following time-series aggregated input for an LSTM-based model:
...
1
vote
3answers
26 views
Does shuffling data for time series forecasting help?
So I am trying time series forecasting using LSTM's. The aim is to predict $Y$ given $X$ using regression.
I had already converted the input data into a sliding window format such that if my input ...
1
vote
0answers
11 views
Is it a good idea to disable or strongly regularize in time series deep learning models?
I'm training a recurrent network on a stock price time series. As you can imagine, the price increases with time. I think the importance of the bias decreases as the stock increases, especially since ...
1
vote
0answers
21 views
How to add attention mechanism to my sequence-to-sequence architecture in Keras?
Based on this blog entry, I have written a sequence to sequence deep learning model in Keras:
...
0
votes
0answers
4 views
How to perform Time series ML analysis that learn on chunks of data rather than 1 single series?
From my reading, many Time Series Machine Learning technique treat the whole time series as single data set and try to learn from that.
I am how to learn from a collection of series rather than one.
...
0
votes
2answers
52 views
LSTM Multivariate time series forecasting with multiple inputs for each time step
I want to predict an output variable for the next day, for each of the users in my dataset. I was thinking of using LSTMs for achieving this.
The dataset
The dataset I am using has multiple inputs ...
0
votes
0answers
8 views
how many spectogram frames per input character does text-to-speech (TTS) system Tacotron-2 generate?
I've been reading on Tacotron-2, a text-to-speech system, that generates speech just-like humans (indistinguisahble from humans) using the github https://github.com/Rayhane-mamah/Tacotron-2.
I'm very ...
0
votes
0answers
7 views
Train one model across multiple multivariate time series of diffrent duration, using categorical metadata
I'm trying to create model for prediction multiple correlated time series features.
Issue is that input dataset consists of a number of "projects" with different duration and different categorical ...
-1
votes
0answers
15 views
it make sense to re-train a ml model every step?
my boss asks me to build a model (LSTM) like this:
I have a series called Data/ len(Data)=5000. I split it into Data_train=Data[:-300] and Data_test=Data[-300:]. ...
1
vote
1answer
29 views
Reason for Huge Jump in Loss For One Epoch Only?
I was wondering what possible reasons there could be for a huge jump in loss for only one epoch during training. I am getting a result like...
...
0
votes
0answers
7 views
Reshaping Input for LSTM Without Losing Samples
I am building an LSTM model to forecast 8 steps in the future using multivariate time series. The input for the LSTM model should be 3d (Num. of Samples, Num. of Time Steps, Num. of Features). If I ...
-1
votes
0answers
14 views
1
vote
1answer
17 views
Is my model to big? I am trying to predict orders for a company, and I don't know if there are typical values for macroparameters
I am building a model to predict orders, from its time series (univariate), for a company. I am working with 30 layers of 400 LSTM neurons each with the activation function hyperbolic tangent of Yann ...
0
votes
1answer
16 views
Output vector size of a LSTM
Is the size of the output vector of all machine learning algorithms the same? Can't an ML algorithm predict only one value as output?
I have trained an LSTM network with X, Y, heading, speed(from ...
0
votes
0answers
15 views
Which r-squared value should I report?
I am building a neural network that consists of an LSTM, dense and dropout layers using Keras to forecast 8 continuous values in the future. I unfortunately have a very small data-set of 56 ...
0
votes
0answers
12 views
CNN + LSTM dimension error
I am building a model to predict if a video images are describing a sleepy person or awake person.
I have trained a CNN custom model to classify blink eyes or not.
Now it's time to join these Conv2D ...
0
votes
1answer
37 views
Error when checking target: expected Output to have 2 dimensions, but got array with shape (631, 80, 2641)
I'm making a Text Chunking program using Bi-LSTM from the model I of the paper "Neural Models for Sequence Chunking". The inputs are sequences of words and the outputs are "B-NP", "I-NP", and "O".
...
0
votes
0answers
8 views
AI architecture for time and spacial sequences
I am working on a project where I analyse MEG data. I have 102 channels as a vector and a 2D matrix of the channels (11x14) to show spatial relations - I want to include that in the AI architecture. ...
1
vote
0answers
15 views
Scaling monotonically increasing features between 0 and 1
To keep the test set blind to the neural network algorithm it is generally better to build a scaler based on the training set and then scale the test set on this scaler.
I am building an LSTM for ...
0
votes
0answers
5 views
Time Series Prediction in Tensorflow where the first data point is different from the following
I've implemented a time series prediction using an LSTM network in Tensorflow. Right now, the input at each timestep is the same. However, the intial time has some data available that later times do ...
0
votes
1answer
8 views
Preparing multiple training time-series for Keras LSTM regression model training
I have training data organised in a numpy array in which:
* column is feature - last one is the target,
* every row is one observation.
The thing is that this 2D ...
1
vote
0answers
15 views
How to fix class imbalance in dialogue (text) time series data?
I have a dataset that looks like this:
...
0
votes
2answers
112 views
Opinions on an LSTM hyper-parameter tuning process I am using
I am training an LSTM to predict a price chart. I am using Bayesian optimization to speed things slightly since I have a large number of hyperparameters and only my CPU as a resource.
Making 100 ...
0
votes
0answers
9 views
LSTM Keras sorting out the X and y input dimensions
I am trying to build an LSTM and am confused about the best way to shape my data.
I have a dataframe that looks like this:
df.head(5)
...
1
vote
0answers
13 views
Normalizing historical data in time-series LSTMs
I am currently trying to solve a sequence prediction problem using LSTMs in a keras architecture. To illustrate the problem I give the following example which resemble the problem I must solve.
Lets ...
0
votes
1answer
24 views
Why is my model accuracy decreasing after the second epoch?
This is my training log for ten epoch for a sentiment analysis model:
...
1
vote
1answer
29 views
Comparing Language Model of two corpora
I know using Conditional Language Model I can learn the probability of a sentence given the corpus I used to train my model. I will then be able to generate meaningful text by sampling from the ...
0
votes
0answers
22 views
Entity recognition and linkage for reference data
I need to work on a project that deals with recognizing the attributes of references (citations)
as Author, Co-Authors, Title of the paper, Publication Venue etc and map them to the real-world ...
4
votes
1answer
89 views
Which target variable should I use?
I have a problem where I want an LSTM to predict the resistance of a body. This value can also be calculated if we know the drag coefficient and the speed of that body. In my case, at inference time, ...
0
votes
0answers
12 views
0
votes
0answers
7 views
How does stateful LSTM work with keras' batch_size > 1?
Let's say I have one input feature with 10 sequences of length 25 to predict the next value. So keras will receive an input vector of (10, 25, 1). If I use ...
1
vote
0answers
15 views
How to apply the Fourier or Wavelet transformed data to LSTM model
I am dealing with a PHM RUL problem, a time series problem of machine signal. I consider to apply Fourier transform or Wavelet Transform to my sensor feature and train the LSTM model. But I have some ...
0
votes
0answers
7 views
Suitable model architecture for categorical, sequential data
Suppose you have a dataset, containing the log data of a set of complex devices, e.g. turbofans.
For each device, the log consists of a time sequence of categorical events.
...
0
votes
0answers
24 views
Keras LSTM predicts every signal in the same category
I'm working on a project that involves signal classification. I'm trying different models of ANN using keras to see which one is better, for now focusing in simple networks but I'm struggling with the ...
0
votes
0answers
9 views
Deep Learning on Float-Value Time Series … How Much VRAM?
How much VRAM can you recommmend for a LSTM with about half a billion values of 32bit Floats per Feature?
Every sample takes 100(in the beginning) to 10.000(later, probably still not enough) ...
0
votes
1answer
16 views
Considering the output of a BLSTM in pytorch, what's the order of the elements?
I am currently using pytorch to implement a BLSTM-based neural network. I understand that the output of the BLSTM is two times the hidden size. However, I am currently unable to find out whether this ...
1
vote
1answer
9 views
Should output data scaling correspond to the activation function's output?
I am building an LSTM with keras which have an activation parameter in the layer. I have read that scaling on the output data ...
0
votes
0answers
34 views
LSTM model for multi-step forecasting with multivariate time series
Im am trying to do a multi-step forecasting with multivariate time series, I have 9 variables (Y,X1,..X8) with 2270 samples for each variable, and I am trying to predict the future values of Y (70 ...
2
votes
0answers
39 views
LSTM Multivariate, structuring data
I'll jump right to the structure of the data, and then I'll ask the question(s):
For a mass X ranging from 200 to 500 units, i have 100 seconds worth of 3 output_values.
So, the first few rows of the ...
0
votes
0answers
13 views
Normalising time-series by geo location
I trained the LSTM model to forecast the number of people infected with COVID-19. Since each country and state (geo location) has different number of population as well as the infected over time, I ...
0
votes
0answers
14 views
Should I use Stateful or Stateless LSTM
I am trying to use LSTM in Keras and I am not sure whether I should used statefull or stateless LSTM. I have read many resources online but seem like they do not apply to my case.
I have a long ...
1
vote
0answers
14 views
LSTM / GRU prediction with hidden state?
I am trying to predict a value based on time series by series of 24 periods (the 25th period)
While training I have a validation set with I babysit the training (RMSE) and each epoch, eval the ...
