# Can a ML model predict output vector based on input vector?

This is a conceptual question. I am aware that many ML models predict the value of a variable on a row-by-row basis. Are there models that do so for vectors? For example, if my data is this:

Input row 1: [A B C], [A B D], [C D G]
Input row 2: [A B G], [A G K], [M D X]
Input row 3: [S K C], [A X D], [C M G]

Output row1: [A B N]
Output row2: [K B N]
Output row3: [J B L]


Can a model learn from vectors and predict the output vector? Or is there an intermediate step to reduce the vectors into single values?

For example, if I add a new row:

Input row4: [X,Y,Z] [M,N,O] [L,N,K]


Can the model predict what the output vector will look like?

• I think you’re looking for multivariate regression. The linear case will be the easiest to understand at first, and then it is not much of a leap to get the gist of how you could do this with a nonlinear regression and/or discrete outcome (e.g., bivariate probit), for example.
– Dave
Oct 28 '21 at 0:45

When you say that an ML model predicts a value based on a variable on a row-by-row basis, you are saying that there is a model $$f$$ such that: $$y = f(\underline{x})$$ for every row $$\underline{x}$$, which is a vector.

So we went from a vector as your input to a scalar as your output. A simple example would be linear regression, when you have a vector input $$[x_1, x_2, \cdots x_n]$$ and a single output $$y$$. For example: $$\mathrm{Income} = \beta_0 +\beta_1\times \mathrm{Age} +\beta_2\times \mathrm{IQ}$$

In this case, your $$\underline{x}$$ is $$[1, \mathrm{Age},\mathrm{IQ}]$$ where the 1 is added to include the bias term in regression.

The output can also be a vector. This is best illustrated if we consider a classification problem where you have multiple outputs. For simplicity, let's keep our $$\underline{x}$$ vector the same, but instead of measuring Income, let's say we are interested in determining the seniority level of a given person of a certain Age and IQ. Our $$y$$ values can now be represented as a vector $$\underline{y} = [\mathrm{Junior}, \mathrm{Mid\,manager}, \mathrm{Senior \,Manager}]$$. Our model $$f(\underline{x})$$ can determine the probability of a person being of a certain seniority level given their Age and IQ. For an example, we might get: $$\underline{y} = [0.8, 0.1, 0.1]$$, indicating that this particular person has an 80% chance of being a Junior, and a 10 percent chance being anything else.

Finally, to respond to your example where each input is comprised of multiple vectors. This is how image classification works! Each $$\underline{x}$$ point is a $$w\times h\times c$$ object where $$w$$ is width, $$h$$ is height and $$c$$ is the channels (RGB). If we consider a classification problem for predicting handwritten digits 0-9 (see 3b1b's video), then our output will be a length 10 vector, where each value is the probability that our given $$\underline{x}$$ is the number at which the index appears.

Different models may treat the input object differently: some may flatten it, thereby creating a vector that is of length $$w\times h$$, but other models may perform operations on the object itself to compress it or increase its dimensionality.