Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

Tensorflow data API: Building continuous streams of data from a Dataset of Datasets

I'm trying to build a language model with LSTMs (like ELMO). I've got a lot of documents and want to split them into words as input, but keep their order. So it should get all words of the first ...
1
vote
0answers
19 views

Data leakage in bidirectional LSTM timeseries data

Does it cause data leakage to train a bidirectional LSTM on data where a user can be a sample in the training data multiple times? Each row is a snapshot at a different point in time for a given ...
-1
votes
0answers
5 views

How to formulate my data for an LSTM, classification problem?

I have several input data that I have ordered chronologically (temperature, wind, etc.) and which cause me to output the output value (1or0). I have a csv file which contains chronologically ordered ...
1
vote
0answers
14 views

LSTM features classification output

I am very new at this, so I might be wrong about my choice of model, but my problem is the following. I am trying to generate music, hence the reason I am using an LSTM. I have the following sequence ...
2
votes
1answer
27 views

LSTM: many to one and many to many in time-series prediction

I am trying to predict the trajectory of an object over time using LSTM. I have three different configurations of training and predicting values in my mind and I would like to know what the best ...
0
votes
0answers
12 views

How to deal with discrete variables in Multivariable Time Series forecasting?

I am tackling this time series forecasting problem to basically predict number of sales in the future training dataset looks like this: ...
0
votes
0answers
11 views

Time series problem with LSTM is not predicting correctly

. My Initial dataset looks like this: Text sample of Dataset: ...
0
votes
0answers
7 views

How to feed multiple asymmetric inputs to LSTM layer?

I'm trying to create an encoder-decoder architecture with an LSTM encoder. The intention is to use both the input image as well as the class label as inputs to the encoder, and to have them share the ...
0
votes
0answers
11 views

What are the advantages of combining BiLSTM and CRF?

BiLSTM-CRF is a common model for sequence tagging (POS tagging, NER, ect.). What are the advantages of combining BiLSTM and CRF? What is the role of each one of the parts in this combination?
1
vote
0answers
24 views

State of the Art/Research 2020 of Time Series Forecasting/Prediction

Im looking for the state of the art/research of time series data for forcasting/prediction. As far as im aware it is Extrem Gradient Boosting (XGBoost) or LSTM (neuronal networks) or are there other ...
0
votes
2answers
19 views

Keras fit_generator() for long signals

I want to make an LSTM network and I have quite a long signal that I want to use as my training data My X_train is a CSV-file which contains 12 signals with a length of 54 837 488 My y_train is an ...
0
votes
0answers
22 views

Input data of variable length to an LSTM

I have a matrix as an input to my LSTM. I want to use a LSTM because the length of the matrix is variable and the width its a fixed size. I would like to know if it is the best option to set a size ...
0
votes
0answers
11 views

Does someone have a low level inplemenation of tensorflow's ConvLSTM2D

Right now I am using the ConvLSTM2D module of tensorflow and it works. But now I would like to feed an additional input between the convolutional stage and the lstm module. Therefore I would need a ...
0
votes
0answers
12 views

MAPE over 100% after normalization of dataset

I try to forecast power demand for next 24 hours. Years 2017 and 2018 are my training set, 2019 is test set. I use multistep vanilla LSTM . In first step I used original data with any preparation and ...
0
votes
0answers
10 views

time series forecasting of time to leave for multiple customers using one model

I am a beginner in the domain of forecasting and I was wondering if such a problem could be solved with time series analysis : given customer historical data of taxi pickups,along with the weather ...
0
votes
0answers
7 views

Is it possible to train a model on a more complicated data set and then retrain on a simpler dataset that has continuous frames for LSTM model?

I'm attempting to do lane detection using CNN-LSTM architectures on the TuSimple dataset. However the TuSimple data set isn't very difficult and the results are sometimes poor (visually). I want to ...
0
votes
0answers
11 views

Is there any multivariate time series prediction tutorial/code using RNN LSTM for R?

I have been searching and reading many articles, tutorials and academic journals including this forum. All I find is predicting a univariate time series. (except for François Chollet's tutorial, I ...
0
votes
0answers
11 views

Multiple time series sequence prediction for multiple multivariate time series

My question is somehow similar to this question, but not satisfied with the answer. I have 100 samples, each sample consists of ...
0
votes
0answers
16 views

Can we use RNN or LSTM for prediction and not forecast

I know RNN and LSTM learn from past data, and can forecast next data. In my situation, I have a learning data-set that hide other information I wish to discover or approximate.(This seems rather an ...
0
votes
0answers
8 views

Attention model with seq2seq over sequence

On the official tensorflow page there is one exmple of a decoder (https://www.tensorflow.org/tutorials/text/nmt_with_attention#next_steps): ...
1
vote
0answers
21 views

Remove subwords from BERT output

I'm trying to build a multilingual WSD system with BERT on top as the embedding layer. In order to have better performances, after BERT finishes its job (and performs Transfer Learning), I need to ...
2
votes
1answer
29 views

How do you think about neural networks and ways to design new models?

I'm currently learning about neural networks, and it seems to me that there usually is no good theoretical explanation given for why certain architectures work; there is most of the times, no formal ...
1
vote
1answer
23 views

How to code a simple forward propagation of recurrent neural networks?

I know the theory behind recurrent neural networks or RNN but I am confused about its implementation. This is an rnn equation I got from the web, I tried to code the forward propagation alone in ...
0
votes
1answer
20 views

keras CNN lstm add model depth

Would anyone have any advice on how to add model depth? This works below but I was hoping to experiment with adding in additional non-TimeDistributed layers. ...
0
votes
0answers
21 views

How to train LSTM for multiple time series with multiple variable and diferent size of time series?

I have a dataset of aircraft messages wich have an column that identify each aircraft example: idaircraft=1 , timestamp=340503404, altitude=xxxxxx,longitude = xxxxx, latitude = xxxxx, Touchdown = ...
0
votes
1answer
30 views

Multi-class classification with discrete output: Which loss function and activation to choose?

I'm working with a multi-class classification problem, using Keras Sequential models. In my dataset, the output class has one of the following values: ...
0
votes
1answer
26 views

Can we add other features along with Text in sentiment analysis

Can we add other features along with Text to a ML model . Like giving text and other features as one input combining them and predict the output value. As model can learn some more better if given ...
1
vote
0answers
15 views

A Python library to convert text into character-level embedding and LSTM

I'm trying to recreate an experiment described in a paper where each character in a URL is converted to a 128-dimension embedding. My dataset looks like below. I'm having trouble figuring out how to ...
1
vote
2answers
61 views

Do I need to convert strings before using LSTM?

I have a dataset which includes one column with a URL and another column with value 0 or 1 indicating if it is a phishing link. I want to process this dataset using LSTM. Do I need to first convert ...
0
votes
0answers
11 views

LSTM data preparation with multiple independent observation runs

I'm struggling to wrap my head around how to correctly deal with multiple independent time series observations for training an LSTM. If I simply concatenate my observation runs, then there will be an ...
0
votes
0answers
12 views

Handling missing timestamps in LSTM model

I have 10 minute SCADA data of wind turbines and many timestamps are missing in between because of regular shutdown and weather conditions. My objective is to predict gearbox failure. How can I handle ...
0
votes
0answers
11 views

Increase dimension of RNN LSTM cell in Keras

I want to increase amount of recurrent weights in rnn or lstm cell. The idea is that RNN neuron takes prevois output as input. I want to increase amount of previous values taken as input. If you ...
-1
votes
0answers
16 views

How to add non-time series features to multivariate LSTM models with several time-steps

I am new to LSTMs and I am a bit confused about multivariate LSTMs with multiple input time steps. I am using Keras with a Tensorflow backend. My dataset contains of parking occupancy rates in five ...
4
votes
2answers
89 views

RNN in pseudo-code

A few years ago, I understood the classical MLP neural network much better when I wrote an implementation from scratch (using only Python + Numpy, without using tensorflow). Now I'd like to do the ...
0
votes
0answers
7 views

LSTM Regression TS with >50% of zero outcomes for y

Modelling multivariate Time series with LSTM, the y of the TS are on >50% consist of zeros. the same is true for features I use loss = 'mean_squared_error', optimizer =Adam, and make grid search ...
0
votes
1answer
43 views

Embedding layer before LSTM layer

I am toying around with a clustering and churn prediction framework, cluschurn which they deployed in production at Snap, Inc. In their research paper, paper_link, they use 14 days of user data and ...
0
votes
0answers
23 views

How does LSTM solve the vanishing gradient problem?

I know that there are many answers. shortly gates solve(mitigate) vanishing gradient problem. But I saw two formidable answers. Thomas Effland's answer, and Nir Abel's answer. I think they explain ...
1
vote
1answer
18 views

Making outputs of sub-models the inputs of other models LSTM

I'm trying to create a neural network that is composed of different sub-models. Each sub-model have their own inputs, which some of them can be shared between the sub-models. For example, sub-model 1 ...
0
votes
0answers
15 views

How to label sequential data for LSTM usage

I want to process data to feed it to a LSTM later, each 100 rows correspond to single category how should I label the data? Should I concatenate the 100 rows into a single row? Data Sample:- ...
1
vote
2answers
72 views

Language translation with convolutional neural network

Many examples of language translation neural networks: "the cat sat on the mat" -> [model] -> "le chat etait assis sur le tapis" use RNN, and in particular LSTM. See for example Sentences language ...
0
votes
1answer
28 views

How to feed training set labels into Keras LSTM

I’m implementing an LSTM with Keras and I know that I have to reshape the training dataset in a 3D object. Basically I have a ...
2
votes
1answer
25 views

Explanation for Why Logistic Regression can be so Accurate in Sentiment Classification?

My question is about how a logistic regression model performs so accurately. In some exploratory experimentation, I compared a logistic regression model against a long short term memory recurrent ...
0
votes
0answers
21 views

Backpropagation LSTM: compute gradient over multiple timesteps

Has anyone implemented simplified code to compute gradient of error over multiple timesteps for a single example. If the timesteps are large, even solving on paper it is getting really complicated. ...
0
votes
1answer
42 views

LSTM Multi-class classification for large number of classes

I want to build a model that classifies 473 classes -product categories-, but I'm facing a problem with loss not decreasing. Data I have almost 3,000 data points for each class -473 classes- (data ...
0
votes
0answers
27 views

LSTM backpropagation issue in Keras code

I have trained the network with data in batches of some batch size >1. After training, I am using trained network and then manually update parameters for every example using backpropagation. The ...
1
vote
1answer
29 views

Training on skewed dataset

I have a problem of multi class classification and I'm using a simple 2-Layer Bi-directional LSTM with keras. The model in a simple form: ...
-1
votes
1answer
29 views

LSTM for text with different sentences size, but same input-output sizes

Hello fellow Data Scientists I'm trying to use a LSTM (using word embeddings) to generate a system that can tag each word of a sentence. For this, I give it a set of sentences of different sizes and ...
1
vote
2answers
34 views

Feed data into Keras LSTM layer

I'm trying to understand how to feed data into LSTM layer of Keras, but I'm in trouble and I don't understand how to do it. I've ...
3
votes
1answer
28 views

Is it possible to create a neural network with two inputs, with sequential layers?

Is there a natural way, in terms of structure of the layers of a NN, in order to pass 2 inputs vectors to the NN? Example: text authorship identification Input #1: sentence1 by unknown author ...
0
votes
0answers
18 views

A training approach to do transfer learning on a pre-trained recurrent neural network to classify for each set of predicted features?

Let's say we want to classify the genre of a number of books as classes. We have a neural network that can read and encode each sentence in the book using a layer of neural network (e.g. word ...

1
2 3 4 5
16