This is a conceptual question. I am aware that many ML models predict the value of a variable on a row-by-row basis. Are there models that do so for vectors? For example, if my data is this:

Input row 1: [A B C], [A B D], [C D G]
Input row 2: [A B G], [A G K], [M D X]
Input row 3: [S K C], [A X D], [C M G]

Output row1: [A B N]
Output row2: [K B N]
Output row3: [J B L]

Can a model learn from vectors and predict the output vector? Or is there an intermediate step to reduce the vectors into single values?

For example, if I add a new row:

Input row4: [X,Y,Z] [M,N,O] [L,N,K]

Can the model predict what the output vector will look like?

  • $\begingroup$ I think you’re looking for multivariate regression. The linear case will be the easiest to understand at first, and then it is not much of a leap to get the gist of how you could do this with a nonlinear regression and/or discrete outcome (e.g., bivariate probit), for example. $\endgroup$
    – Dave
    Oct 28 '21 at 0:45

When you say that an ML model predicts a value based on a variable on a row-by-row basis, you are saying that there is a model $f$ such that: $y = f(\underline{x})$ for every row $\underline{x}$, which is a vector.

So we went from a vector as your input to a scalar as your output. A simple example would be linear regression, when you have a vector input $[x_1, x_2, \cdots x_n]$ and a single output $y$. For example: $\mathrm{Income} = \beta_0 +\beta_1\times \mathrm{Age} +\beta_2\times \mathrm{IQ}$

In this case, your $\underline{x}$ is $[1, \mathrm{Age},\mathrm{IQ}]$ where the 1 is added to include the bias term in regression.

The output can also be a vector. This is best illustrated if we consider a classification problem where you have multiple outputs. For simplicity, let's keep our $\underline{x}$ vector the same, but instead of measuring Income, let's say we are interested in determining the seniority level of a given person of a certain Age and IQ. Our $y$ values can now be represented as a vector $\underline{y} = [\mathrm{Junior}, \mathrm{Mid\,manager}, \mathrm{Senior \,Manager}]$. Our model $f(\underline{x})$ can determine the probability of a person being of a certain seniority level given their Age and IQ. For an example, we might get: $\underline{y} = [0.8, 0.1, 0.1]$, indicating that this particular person has an 80% chance of being a Junior, and a 10 percent chance being anything else.

Finally, to respond to your example where each input is comprised of multiple vectors. This is how image classification works! Each $\underline{x}$ point is a $w\times h\times c$ object where $w$ is width, $h$ is height and $c$ is the channels (RGB). If we consider a classification problem for predicting handwritten digits 0-9 (see 3b1b's video), then our output will be a length 10 vector, where each value is the probability that our given $\underline{x}$ is the number at which the index appears.

Different models may treat the input object differently: some may flatten it, thereby creating a vector that is of length $w\times h$, but other models may perform operations on the object itself to compress it or increase its dimensionality.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy