-1
$\begingroup$

This is a conceptual question. I am aware that many ML models predict the value of a variable on a row-by-row basis. Are there models that do so for vectors? For example, if my data is this:

Input row 1: [A B C], [A B D], [C D G]
Input row 2: [A B G], [A G K], [M D X]
Input row 3: [S K C], [A X D], [C M G]

Output row1: [A B N]
Output row2: [K B N]
Output row3: [J B L]

Can a model learn from vectors and predict the output vector? Or is there an intermediate step to reduce the vectors into single values?

For example, if I add a new row:

Input row4: [X,Y,Z] [M,N,O] [L,N,K]

Can the model predict what the output vector will look like?

$\endgroup$
1
  • $\begingroup$ I think you’re looking for multivariate regression. The linear case will be the easiest to understand at first, and then it is not much of a leap to get the gist of how you could do this with a nonlinear regression and/or discrete outcome (e.g., bivariate probit), for example. $\endgroup$
    – Dave
    Oct 28 '21 at 0:45
2
$\begingroup$

When you say that an ML model predicts a value based on a variable on a row-by-row basis, you are saying that there is a model $f$ such that: $y = f(\underline{x})$ for every row $\underline{x}$, which is a vector.

So we went from a vector as your input to a scalar as your output. A simple example would be linear regression, when you have a vector input $[x_1, x_2, \cdots x_n]$ and a single output $y$. For example: $\mathrm{Income} = \beta_0 +\beta_1\times \mathrm{Age} +\beta_2\times \mathrm{IQ}$

In this case, your $\underline{x}$ is $[1, \mathrm{Age},\mathrm{IQ}]$ where the 1 is added to include the bias term in regression.

The output can also be a vector. This is best illustrated if we consider a classification problem where you have multiple outputs. For simplicity, let's keep our $\underline{x}$ vector the same, but instead of measuring Income, let's say we are interested in determining the seniority level of a given person of a certain Age and IQ. Our $y$ values can now be represented as a vector $\underline{y} = [\mathrm{Junior}, \mathrm{Mid\,manager}, \mathrm{Senior \,Manager}]$. Our model $f(\underline{x})$ can determine the probability of a person being of a certain seniority level given their Age and IQ. For an example, we might get: $\underline{y} = [0.8, 0.1, 0.1]$, indicating that this particular person has an 80% chance of being a Junior, and a 10 percent chance being anything else.

Finally, to respond to your example where each input is comprised of multiple vectors. This is how image classification works! Each $\underline{x}$ point is a $w\times h\times c$ object where $w$ is width, $h$ is height and $c$ is the channels (RGB). If we consider a classification problem for predicting handwritten digits 0-9 (see 3b1b's video), then our output will be a length 10 vector, where each value is the probability that our given $\underline{x}$ is the number at which the index appears.

Different models may treat the input object differently: some may flatten it, thereby creating a vector that is of length $w\times h$, but other models may perform operations on the object itself to compress it or increase its dimensionality.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy