Seq to Seq modelling - ML Algorithms to use

Question

Am new to ML. While I learnt the classical ML concepts like Linera regression, Logistic regression, Boosting and tree based techniques, now am slowly trying to learn Deep Learning techniques like CNN, RNN, LSTM ,GRU tech

My question is

a) What are the techniques that we can use to perform Seq to Seq modelling? ex: I give a input sequence and get an output sequence.

For ex: If I would ask what are the techniques that I can use for classification tasks, you guys would suggest algos like a) Logistic, b) Trees (Random Forest, Decision Tree), c) SVM, d) NN

Similarly, I would like to know what are the algorithms and techniques that I should learn to perform Seq to Seq modelling?

Is it only the below?

a) RNN (LSTM, GRU)

I am a noob but I see online that people talk about transformers etc. Is transformers an algorithm?

Can provide me the list of algorithms and techniques on how can I do seq to seq modelling?

Basically, I am looking to fill values under the column algos that can be attempted table (shown below) for problems that involve Sequence input and Sequence output

can help please?

Data type	n	op variable type	Objective	Algos that can be attempted
Sequence	10K	Sequence	Predict a Sequence	LSTM
Sequence	10K	Sequence	Predict a Sequence	GRU
Sequence	10K	Sequence	Predict a Sequence	??
Sequence	10K	Sequence	Predict a Sequence	??
Sequence	10K	Sequence	Predict a Sequence	??
Sequence	10K	Sequence	Predict a Sequence	??

Answer 1

First, you need to understand what Sequential Modeling is? There are two categories in which Sequential Modeling falls in

1. Suppose the data itself is a Sequence of the stream like audio, time-series data, textual data.
2. Other is if you have the model who is working in sequential manner, what it means? Suppose you are giving a model training data but the model has not the capability to take the input at once, like in RNN you give the input in a sequence to the model first $x_1$ at Cell No.1 then $x_2$ at cell no2 and go on. So, your model itself hasn't the capability which could take the input at once, like in Neural Network, CNN, we give the input training examples at once in a batch.

You are asking about the problem that how many models are there who has the capability to deal with sequential data?

You have studied LSTM and GRU. These two are the core models before going to advance architectures. Besides these models, we have statistical models which deal with Sequential data, like n-gram model (deals with language modeling) Hidden Markov Model (for parts of speech tagging), etc. So, now understand what is Transformers basically? If you have studied LSTM and GRU well, then there are two big problems with them that are the model works in a sequential manner it means we can't take advantage of GPUs and the other problem is transfer learning is not possible using GRU and LSTM because of their recurrence.

To, sort out these problems "Attention all you need" in this paper they proposed a model name "Transformer" and the technique which is used to enable the solutions for the above problem was "Attention Mechanism".

The Architecture itself isn't Sequential but it can train and gives better results than LSTM and GRU on Sequential Data.

Besides this many state-of-the-art models are build on top of the Transformer Model like "BERT", "T-5", "Roberta", etc. These all work for sequential data training.