# Magnifying or reducing the size of input groups into a neural network

Say you've got two inputs (X1 and X2) that you want to use to predict Y.

You're not sure how important X1 and X2 are for predicting Y, but you assume about even.

One-hot encoding is a good strategy for X1 and it yields a vector of size 10,000.

X2 is an unsigned int between 0 and 1, so you can just pass it as-is.

So your network will look something like

10,001 -> {some hls} -> Out-layer


In theory it could "learn" to assign a lot of importance to X2 compared to X1, but in practice I assume this is hard when the difference in dimension varies so much. At least based on some datasets I tested this was certainly true.

The one "simple" solution I can think of to this issue is instead having a network shapes more like:

10,000 -> {some hls} -> 1,000 --
|
|-> {some hls} -> Out-layer
|
1 -> {some hls} ------> 1,000 --


So basically have some encoders/backbones/whatever-you-want-to-call-them, that increase/reduce the size of certain groups of inputs and train them at the same time as the normal network.

My question here is:

a) Is the problem I identified here a "real" one or would it not come up in practice ? b) Does it have a name and are there already established solutions to it ? c) Is the solution I proposed here a "good one" ? d) Do you have any examples of networks where this method is actually being used ? Preferably implemented in pytroch but it doesn't matter really, the implementation per-say seems easy enough. f) Are there potential pitfalls in terms of performance and or implementation to this solution that might not be obvious ?

X1 -{OHE}-> 10,000 -> {..} -> 1,000 you could go straight to X1 -{embedding}-> 50, where the embedding dimension should probably a lot lower than 1,000.