Questions tagged [lstm]
LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.
1,170
questions
0
votes
0
answers
10
views
Model Architecture for Time-Series Forecasting with Categorical and Multivariate Data
Context:
I was looking at using an LSTM model to forecast the amount of gold gained for each of 10 heroes in a game of Dota 2, a MOBA game, as a base model in some type of model architecture. The game ...
1
vote
0
answers
13
views
0
votes
0
answers
5
views
Connecting Flatten layer to Dense layer
I'm struggling with my neural network.
In short, I need to recreate a model from anywhere on the internet, I've found a model that combines BiLSTM, LSTM and GRU. However, based on the error I got when ...
0
votes
0
answers
23
views
Can my LSTM model learn feature engineering on its own?
I have a timeseries dataset and I am training an LSTM model on it to perform multiclass classification.
My dataset has 7 columns => x1,x2,x3....x7
And has 4 labels => f1,f2,f3,f4
Since I have ...
0
votes
0
answers
5
views
Activity Classification through LOG file
I have a big dataset containing logs/steps that the user performed on my webpage (for example: Clicking on a "Homepage" button, typing some text in the field, etc.) These steps are labelled ...
0
votes
0
answers
9
views
LSTM Model for Multivariate Multi-Series
I'm looking to create an LSTM model to predict a certain label trained on multiple short-time series data. How would I go about doing this? Each time series has 10-30 time steps and 20 different ...
0
votes
0
answers
9
views
Can I use TimeDistributed layer for multiclass classification?
I have timeseries machine sensor data and I would like to predict when the machine fails using the sensor data. There are 4 Failure states and 1 Normal state, total of 5 classes.
I am trying to solve ...
1
vote
1
answer
29
views
Aside from trial and error, how do I select the number of layers and unit counts for LSTMS, GRUs, and Transformer units for text and time series?
When deciding on the number of units and layers for text processing or time-series prediction I rely heavily on trial and error. First, I look for a reference or paper on the topic such as the white ...
1
vote
1
answer
34
views
Why is my LSTM model not predicting well when predicting labels for a new dataset?
I have a 15 timeseries datasets with 25-30 columns and is labeled by following a complex formula applied on the 25-30 columns.
When training, I split the datasets as training datasets and unseen ...
0
votes
1
answer
59
views
Questions about hidden states of bidirectional LSTMs
I read this in an article about bidirectional LSTM:
In bidirectional LSTM, each word corresponds to two hidden states, one
for each direction. Thus, we concatenate these two hidden states to
...
0
votes
1
answer
57
views
Which ML algorithm is suitable for a dataset that has seasonality and trend?
I have a small dataset from 2006 to 2023, I would like to predict monthly sales for the next year. This is my data:
I already tried Prophet and NeuralProphet, but unfortunately they don't work well ...
0
votes
0
answers
28
views
How do I shape my output data for a time series classification problem using LSTM
I am wanting to use an LSTM for anomaly detection on a multivariate time series data. Let's say there are n rows each corresponding to a timestamp incrementing by an hour and d input features and d ...
2
votes
1
answer
74
views
What is the difference between hidden states in RNN and Transformers model?
I'm very terrible at NLP and I have searched for these questions but didn't find any answer, my question is, in RNNs, there are hidden states to remember information for processing the next state, and ...
0
votes
1
answer
60
views
Why not Back propagate through time in LSTM , similar to RNN
I'm trying to implement RNN and LSTM , many-to-many architecture. I reasoned myself why BPTT is necessary in RNNs and it makes sense.
But what doesn't make sense to me is, most of resources I went ...
0
votes
1
answer
37
views
Anomaly Detection in Log Data using LSTM
Problem Overview:
I am currently working on a project involving anomaly detection in log data. The anomalies are defined by deviations from historical patterns. The log data has a simple structure: [...
1
vote
1
answer
121
views
Understanding the concepts of word embedding in GPT-2
I have a program that calculate the word embedding using GPT-2 specifically the GPT2Model class:
...
0
votes
2
answers
22
views
how to fix my increasing validation loss and decreasing training loss?
here is the code that got me this, please i need an advise on what to do to correct this.
...
0
votes
0
answers
38
views
Can I use lstm/autoencoder to cluster multivariate time series?
I have a multivariate time series of driving scenarios which has X,Y positions, speed, orientation etc. of the vehicles. Each scenario A, B, C, D etc. are of different lengths with different delta ts ...
1
vote
1
answer
20
views
TensorFlow LSTM model with lower epoch loss, but higher average RMSE. How/why?
I am very perplexed by the lower loss but higher RMSE:
Here's a newer model with better loss scores on the same dataset and many predictors:
...
0
votes
0
answers
28
views
PyTorch input shape for text classification using LSTM
I have three sentiment classes: POSITIVE, NEGATIVE, and NEUTRAL, along with a dataset consisting of 3000 sentences and their corresponding sentiment labels (POSITIVE, NEGATIVE, or NEUTRAL). Each ...
0
votes
1
answer
29
views
Forecasting of multiple related time series
I would like to know if there are any methods to forecast multiple time series that are related ? I heard of LSTM but of what I can see it's more to forecast one time series and not several at the ...
0
votes
1
answer
27
views
Does a RNN also need a 3-dimensional input vector for a "Point-To-Point" forecast?
I know that for many applications a RNN (e.g. LSTM) needs a 3-dimensional input structure with [Batchsize, Sequence_Length, Features]. My question is if you also need a 3-dimensional input vector when ...
0
votes
0
answers
57
views
LSTM Layer producing same outputs for different sequences
Currently I try to train on a multi-label language task with imbalanced class distribution. I have the following model, where I removed some of the feed forward layers to decrease factors in the chain ...
0
votes
0
answers
13
views
Lack of Variability in Predictions from Multivariate LSTM Model
I've been working on a multivariate LSTM model for time series forecasting, but I'm encountering an issue where the predicted output doesn't exhibit enough variability. The predictions tend to be too ...
1
vote
0
answers
37
views
Is it possible to determine the probability of each time sample to belong a certain class using gaussian distribution with Recurrent Neural Networks?
I'm trying to train a deep learning model that predicts the probability of each time sample in a two-component time series . In this case, I want the target tensor (Y) to be a probability value for ...
1
vote
1
answer
28
views
Out-of-Range Target Variable in Sequence-based Machine Learning Model
I'm encountering a scaling issue in a machine learning project. I'm predicting a target variable from an input sequence (and doing this for many). However, I've encountered a challenge where the ...
0
votes
0
answers
14
views
Is it possible to use leftovers data (warehouse stocks data) to create sales forecasting?
For example, i have sales data by categories
| Date | GE | VIC |
| -- | -- | --|
|03.01.2022 |2|7|
|10.01.2022 |30 |12|
|17.01.2022 |15 |5|
|24.01.2022 |57 |8|
|.....|...|...|
|28.08.2023 |16 |2|
And ...
0
votes
1
answer
32
views
LSTM For Predicting Vector Sequences
I am attempting to construct a Keras model that intakes a sequence of vectors and outputs the most likely next vector in the sequence. I have followed a few tutorials, but nothing is quite seeming to ...
0
votes
0
answers
16
views
Tensorflow RNN - implementing recursive layer
I am dealing with a regression problem, for which I wanted to try to use a recurrent neural network. The general setting is that I have to predict a continuous quantity starting from the evolution, in ...
0
votes
0
answers
13
views
Training multi-variate LSTM model with sample observations with differenet mean values
I am developing an LSTM model to predict the force-deformation response for wind turbine blades. I have generated the training data from a high-fidelity model for wind speeds ranging from 3m/s to 25m/...
0
votes
0
answers
23
views
Cant pass input_shape to LSTM layer in Keras
I have a numpy array X_train of shape (number of samples, timestep , number of features) =...
0
votes
1
answer
87
views
Converting a Standard LSTM RNN over to a Transformer Model
I am looking for some advice on converting my existing CNN/LSTM RNN over to a Transformer type model. This regression model takes a sliding window size of 240 rows with 33 features. It aims to ...
0
votes
0
answers
29
views
What is the reason behind high frequency output from LSTM model?
Following is the time history response of my input features, which has relatively low frequency component
My LSTM network architecture is as follows:
...
0
votes
0
answers
13
views
Stable test in online time series forecasting problem
I have a Time Series Forecasting problem. You can think of it as predicting the daily closing prices of Apple stocks. My data is divided into 4-day segments, and the forecasting is based on predicting ...
0
votes
0
answers
25
views
Confusion about LSTM network with multiple LSTM units
An LSTM layer can have multiple LSTM units ( LSTM memory cells each with input/update, output, and forget gates). In this question, we have 2 LSTM layers each with two cells, and according to the ...
0
votes
0
answers
20
views
Can a multivariate MIMO LSTM forecaster learn the relationships between the multiple feature outputs?
Question: Can a multivariate MIMO LSTM learn the relationships between the multiple feature outputs?
This question arose when I decided to modify a multivariate (Multiple Input - Single Output, MISO) ...
0
votes
0
answers
20
views
Predicting quanting sold using Time series data
I am struggling with a time series dataset comprising 12 features, including quantity sold and weather data, totaling approximately 1800 values. My goal has been to forecast future values, quantity ...
0
votes
0
answers
17
views
Validation loss hump in LSTM
I'm using PyTorch to fit an LSTM to a binary time series dataset which has about 300 time series of about 20 items. I am using 15% of the time series as a validation set. I then have an MLP on top of ...
0
votes
0
answers
14
views
Meaning of mean squared error in multistep prediction
In multistep prediction with LSTM(keras), say we had this kind of result:
target = [[1,2,3] ,[4,5,6] ]
predictions = [[1.1,2.2,3.3] , [4.4,5.5,6.6]]
When we choose mean_squared_error as the loss ...
0
votes
0
answers
29
views
LSTM - How can I predict the status an hour before in advance?
I’m very beginner, I’m trying to design a prediction model for forecasting the status one hour ahead.I have 150 sample data, each consisting of of 24 hours of time-series data with multiple features (...
2
votes
1
answer
187
views
Role of stateful parameter vs shuffle parameter in LSTM keras
I'm trying to make prediction on a multivariate time series using LSTM. I know stateful=True in keras LSTM means state(hidden) of each sequence, in a batch, at index i - is passed to the next batch, ...
0
votes
0
answers
91
views
Custom Loss Function Returns Graph Execution Error: Can not squeeze dim[0], expected a dimension of 1, got 32
I have built a loss function which adds time and frequency weighted averages and variances to the MSE:
...
1
vote
1
answer
37
views
LSTM Training Interpretation
I have an LSTM and have the following chart showing training validation performance by epoch:
Could someone explain? How can my validation performance be better than my training performance in ...
0
votes
0
answers
212
views
Value Error: One of the dimensions in the output is <= 0 due to downsampling in conv1d_9
i am trying to implement classification model on my dataset, which has 3 columns and 651 rows
Displacement Time Labels
0.000245879 0.01 Undamage
0.001954869 0.02 Damage
0.006545664 0.03 Undamage
0....
0
votes
0
answers
19
views
RNN model for predicting sequences based on sequences of different lengths with Keras
I have data that are sequences of repeated values of different lengths. The value is categorical and can take values from 1 to 184. I used padded with 0 and masking:
...
0
votes
0
answers
25
views
Forecasting using LSTM Model
I have a dataset with 12 variables (x1, x2, ..., x12) and one target variable (y). Is it possible to perform a forecasting for the target variable (y) over a certain period of time ahead without ...
0
votes
1
answer
36
views
Identification of inexactly-recurring material in time series stream
I am working on a personal project involving the analysis of a stream of audio data and the identification of (non-verbatim) repeated subsequences.
My research on time series has so far lead me to ...
0
votes
0
answers
8
views
Creating insights from Battery monitoring parameters (State-of-charge, battery cell voltages, temperature, etc.) to use with AI or model based
So, in a new role currently and I decided to pursue the health monitoring and impending failures of batteries (Lithium Iron Phosphate, Lithium Ion and a few lead-acid as well) but having never done it ...
1
vote
0
answers
24
views
Validation Accuracy incorrect when multiple outputs are in the dense layer!
I have a set-up a parallel LSTM architecture with two LSTM layers and one Dense layer producing several outputs which are converted into a probability with a sigmoid function on the dense layer. For ...
0
votes
2
answers
256
views
How to add multiple embeddings (layers) to LSTM layer
The similar question was asked before here https://stackoverflow.com/questions/52627739/how-to-merge-numerical-and-embedding-sequential-models-to-treat-categories-in-rn/52629902#...