Skip to main content

Questions tagged [sequence]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
14 votes
2 answers
15k views

How to implement "one-to-many" and "many-to-many" sequence prediction in Keras?

I struggle to interpret the Keras coding difference for one-to-many (e. g. classification of single images) and many-to-many (e. g. classification of image sequences) sequence labeling. I frequently ...
Hendrik's user avatar
  • 8,667
12 votes
1 answer
15k views

what is the first input to the decoder in a transformer model?

The image is from url: Jay Alammar on transformers K_encdec and V_encdec are calculated in a matrix multiplication with the encoder outputs and sent to the encoder-decoder attention layer of each ...
mLstudent33's user avatar
10 votes
2 answers
971 views

Classification of vector sequences

My dataset is comprised of vector sequences. Each vector has 50 real-valued dimensions. The number of vectors in a sequence range from 3-5 to 10-15. In other words, the length of a sequence is not ...
Vladislavs Dovgalecs's user avatar
8 votes
3 answers
2k views

Algorithm for segmentation of sequence data

I have a large sequence of vectors of length N. I need some unsupervised learning algorithm to divide these vectors into M segments. For example: K-means is not suitable, because it puts similar ...
generall's user avatar
  • 273
6 votes
2 answers
18k views

Sequence data vs time series data

What is the difference between sequence data and time series data? My understanding is that sequence data is any data where the order matters and time series is a special type of sequence data ...
user2840286's user avatar
5 votes
1 answer
189 views

Classification of DNA Sequences

I have a database of 3190 instances of DNA consisting of 60 sequential DNA nucleotide positions classified according to 3 types: EI, IE, Other. I want to formulate a supervised classifier. My ...
akellyirl's user avatar
  • 723
5 votes
1 answer
2k views

Why do position embeddings work?

In the papers "Convolutional Sequence to Sequence Learning" and "Attention Is All You Need", positions embeddings are simply added to the input words embeddings to give the model a sense of the order ...
Robin's user avatar
  • 1,337
5 votes
2 answers
268 views

Predict a sequence given many sequences

I'm trying to find an algorithm that would fit this use case: My data: a bunch of fixed-size integer arrays, e.g. [0,2,3,4,5] [1,2,3,1,5] [4,1,2,4,5] ... Input: ...
Andrew Furman's user avatar
4 votes
2 answers
2k views

In sequence models, is it possible to have training batches with different timesteps each to reduce the required padding per input sequence?

I want to train an LSTM model with variable length inputs. Specifically I want to use as little padding as possible while still using minibatches. As far as I understand each batch requires a fixed ...
Tonnz's user avatar
  • 63
4 votes
3 answers
611 views

Are there cyclic decision trees?

Usual decision trees are directed acyclic graphs. Are there generalizations of decision trees that contain cycles analogously to recurrent neural networks? If such trees exist, can they be applied to ...
Vladislav Gladkikh's user avatar
4 votes
2 answers
1k views

Can bidirectional RNN use variable sequence length?

A bidirectional RNN consists of two RNNs, one for the forward and another for the backward sequential directions, which outcome is concatenated at each time step. Would this configuration restrict the ...
DiMorten - Jorge Chamorro's user avatar
4 votes
1 answer
194 views

Best HMM Package

What is the best HMM (Hidden Markov Model) library available in Python? I have already looked into seqlearn and hmmlearn, but both of them don't seem to be actively maintained. Thanks in advance!
Edamame's user avatar
  • 2,765
4 votes
2 answers
1k views

Is this a problem for a Seq2Seq model?

I'm struggling to find a tutorial/example which covers using an seq2seq model for sequential inputs other then text/translation. I have a multivariate dataset with n number of input variables each ...
Ellio's user avatar
  • 93
4 votes
1 answer
1k views

Viterbi-like algorithm suggesting top-N probable state sequences implementation

Traditional Viterbi algorithm (say, for hidden Markov models) provides the most probable hidden state sequence given a sequence of observations. There probably is an algorithm for decoding top-N ...
Anton's user avatar
  • 243
3 votes
1 answer
90 views

Recommender Model for Human Action in Income Protection

Problem Domain I'm working on a project that involves building a model to provide recommendations on the next best step for Human supervisors to take on income protection claims. Income protection is ...
Doogyb's user avatar
  • 31
3 votes
1 answer
3k views

Principle behind seq2seq model's example in keras?

I am referring to seq2seq model's example code in keras (https://github.com/fchollet/keras/blob/master/examples/addition_rnn.py). The model is : ...
SHASHANK GUPTA's user avatar
3 votes
1 answer
119 views

User actions sequence classification

I have a training set where each row is a series of user actions on a website (logged in, sent an invoice, etc.) and times deltas in ms between these actions. Each row has a label — a corresponding ...
Klimentij Bulygin's user avatar
3 votes
2 answers
2k views

What clustering algorithm is appropriate for clustering paths?

I have a dataset with vectors in 2-dimensional space that form separate sequences (paths). Full data is presented below: , while a random sample of 5 paths looks like below (please note that ...
Kuba_'s user avatar
  • 264
3 votes
1 answer
349 views

LSTM training/prediction with no starting sequence

ML newbie here. As an exercise, I'm trying to build a character based language model based on a simple 1 layer LSTM. Based on what I've learned about LSTMs, a common usage is to take in a sequence of ...
Seth's user avatar
  • 181
3 votes
3 answers
130 views

Classification when the classification of the previous itens matter

I have a classification problem to solve, that seems to be common but I am struggling to find the name of this task and the best way to model this problem. Suppose I have a series of events that are ...
bratao's user avatar
  • 31
3 votes
2 answers
3k views

Running out of memory when training Keras LSTM model for binary classification on image sequences

I'm trying to come up with a Keras model based on LSTM layers that would do binary classification on image sequences. The input data has the following shape: ...
Alex's user avatar
  • 131
3 votes
0 answers
304 views

Why embedding or rnn/lstm can not handle variable length sequence?

Pytorch embedding or lstm (I don't know about other dnn libraries) can not handle variable-length sequence by default. I am seeing various hacks to handle variable length. But my question is, why this ...
sovon's user avatar
  • 521
3 votes
0 answers
1k views

TypeError: unsupported operand type(s) for %: 'int' and 'NoneType'(Stateful LSTM Keras)

So I have a trained LSTM model with which I am trying to predict future values. The model is stateful as seen below ...
hisairnessag3's user avatar
3 votes
0 answers
191 views

Next events prediction based on previous events

I have data set of sequences of user executed commands sorted in the order of its occurrence. The data looks like this. ...
msp12's user avatar
  • 31
2 votes
1 answer
1k views

How to evaluate sequence to sequence models?

I wonder how to evaluate variable long sequence-to-sequence predictions? Let us say I have the following $Y$ and $\hat{Y}$ $Y = [["1", "2", "2"], ["3", "2", "2"], ["1", "3", "2", "2"]]$ $\hat{Y} = [[...
Hendrik's user avatar
  • 8,667
2 votes
1 answer
170 views

Find average sequence from a set of sequences [closed]

I have a set of user sessions. Session consists of an ordered list of types of actions that user made (for example, bought a gun, played a mission, etc). I want to create/calculate session that have ...
bobby's user avatar
  • 131
2 votes
1 answer
535 views

Query on unstable loss curves for RNN

I’m currently building sequence models for forecasting, and have tried using RNNs, LSTMs, and GRUs. Something unusual I noticed was the highly unstable loss curves, where the loss sometimes goes back ...
Eugene Yan's user avatar
2 votes
1 answer
298 views

Multiple merging multiple convolutions

(First post here) I am rather new to neural networks, having used Tensorflow for a couple months now, and am looking for some advice I have on an idea to improve the accuracy of my model. I am looking ...
superhelicase's user avatar
2 votes
2 answers
7k views

What algorithms are good to predict next numbers?

Let's consider we have several hundreds of numbers like ( 1, 2, 5, 8, 7, 15, 19, 8, 4, 6, ...) those are closed numbers of a stock on consecutive days for example. I like to know what algorithms are ...
user3486308's user avatar
  • 1,290
2 votes
1 answer
2k views

Very long sequence in neural networks

Beginner's question regarding sequences in neural networks: suppose I have classification problem that looks like: X = very long sequence of varying length. Y = class (assume for simplicity y=0/1). ...
Roger T's user avatar
  • 31
2 votes
1 answer
389 views

LSTM with return_sequences - "Training a model on multiple timesteps simultaneously"

So I'm following Tensorflow's LSTM/time series tutorial and there's something I don't understand. I do understand what happens with return_sequences on/off, however,...
lte__'s user avatar
  • 1,350
2 votes
1 answer
442 views

How to train a model on top of a transformer to output a sequence?

I am using huggingface to build a model that is capable of identifying mistakes in a given sentence. Say I have a given sentence and a corresponding label as follows -> ...
MetaInformation's user avatar
2 votes
1 answer
31 views

Use LSTM to predict the proportion of steps with nonzero feature values

I am trying to do a simple regression for sequences. Each input $X_i$ is a $n=2000$ by 1 matrix, formatted as $n_i$ 0-s followed by $(n-n_i)$ 1-s. The output $y_i$ should be $n_i/n$, i.e. the ...
physcis_beginner's user avatar
2 votes
2 answers
91 views

Sequence prediction with unlimited predictions

I have a special kind of prediction problem. I have observed $M$ sequences $X_m = [x_1, x_2, ..., x_N]$ where the distance $d$ between $x_n$ and $x_{n+1}$ is drawn from the same normal distribution, ...
hwaxxer's user avatar
  • 123
2 votes
1 answer
2k views

bert-as-service maximum sequence length

I installed bert-as-service (bert-as-service github repo) and tried encoding some sentences in Japanese on the multi_cased_L-12_H-768_A-12 model. It seems to work ...
mLstudent33's user avatar
2 votes
2 answers
1k views

Give Variable Length input to LSTM

My input data consist of list of list. Both list have dynamic length for every example like below. ...
Batuhan B's user avatar
  • 226
2 votes
1 answer
503 views

Neural network for variable length data classification

How can I create a network which can predict labels of variable lengths data: Training data: ...
rnso's user avatar
  • 1,578
2 votes
1 answer
391 views

Next event prediction in the sequence

I have a sequence of event and would like to predict the next one. The training data looks like this: Ev1,Ev2,Ev5,Ev6,Ev7 Ev1,Ev6,Ev99 Ev4,Ev3,Ev6 So, the idea is to get Ev7 given Ev1,Ev2,Ev5,Ev6 ...
Peter Krejzl's user avatar
2 votes
0 answers
130 views

1D Sequence Classification

Cross-post from https://stackoverflow.com/questions/71752744/1d-sequence-classification I am working with a long sequence (~60 000 timesteps) classification task with continuous input domain. The ...
Kevin's user avatar
  • 163
2 votes
1 answer
28 views

How to use time-sequence data with "meta data" of single value per sample?

I'm, trying to estimate a fish weight by a time-sequence signal of the change in resistance when the fish goes through a gate with electrodes installed. When the fish pass through the gate there's a ...
Shay's user avatar
  • 23
2 votes
1 answer
256 views

Predicting sequence element based on the previous M and the following N elements

I have an array of sequences of equal length, each sequence contains 300 numbers (M=300). Each element in a sequence is a number from 1 to 9: ...
stv's user avatar
  • 85
2 votes
1 answer
45 views

Comapring hidden markov models

Given a set of sequence transitions, there are different orders of hidden markov models that can be fitted to a dataset. Is there any test to determine which is the best model for a given sequence ...
APaul31's user avatar
  • 21
2 votes
1 answer
210 views

What's an appropriate datastore for variable length sequence data for PyTorch consumption?

I have a large number of sequences - potentially hundreds of thousands - each consisting of between 100 and 10,000 items, which each consist of about 5 floats. I need a datastore that can rapidly ...
Sam's user avatar
  • 121
2 votes
0 answers
709 views

Proper loss function for sequence prediction model with multi-step output

Consider a typical time series (sequence) prediction problem that use previous $k$ step historical features to predict the next step target. We use RNN model as an ...
JunjieChen's user avatar
2 votes
1 answer
567 views

Sequence models word2vec

I am working a data-set with more than 100,000 records. This is how the data looks like: ...
stats_171990's user avatar
2 votes
0 answers
70 views

Mapping “event” series, with segments of variable length, to time series for loss calculation

I have a time series of data which I map to a time series of non-exclusive binary features (in my case, short-time audio spectrograms to features of the audio). I will refer to a collection of these ...
Anaphory's user avatar
  • 140
2 votes
0 answers
85 views

Classifying sequences of mouse clicks with different number of clicks per sequence

I have a series of sequences, each sequence contains multiple mouse click with the following features: [x coordinate on screen, y coordinate on screen, duration of click], and each sequence is labeled ...
Andrej Naumovski's user avatar
2 votes
0 answers
502 views

k- means clustering on Markov chain trasition probability

I have data set of 50 students. I want to cluster them on their sequential data ( While doing a job they followed multiple sequences A, B, c total 7 stages). I am planning to apply k-means clustering ...
Bloodstone Programmer's user avatar
2 votes
0 answers
84 views

Classifying Sequences Where Some Sequences in Both Classes

I am building a modified RNN (specifically a GRU) to classify sequences. These sequences are of variable length, and contain "states". Each point in the sequence is categorical and they look like [A,B,...
Carter's user avatar
  • 21
2 votes
2 answers
211 views

Modeling the influence of events order on probability

The case is to model if the sequence of events influences the probability of binary target variable. We have for example five different events which occur in time (event: A,B,C,D,E). They can occur in ...
Luc's user avatar
  • 31