# What can I infer from a linear correlation of regression coefficients?

I am working on a dataset for classification where each observation is a series of values of a certain measurement $$Y$$ for a fixed range of values of measurement $$X$$ (i.e. a discrete mapping from $$A \subseteq X$$ to $$Y$$). (I'm not allowed to reveal what $$X$$ and $$Y$$ are due to confidentiality issues.) I am doing some single-variable regression on this relationship to obtain a smooth $$X$$ $$\to$$ $$Y$$ function for each observation. The basis functions I'm using for this are

$$\left\{x\mapsto 1,\ x \mapsto \frac{1}{\sqrt{x}},\ x\mapsto e^{-x \cdot 10^{-3}}\right\}$$

This is done by applying each of these functions to the range of $$X$$ values; then this transformed feature set is passed to a standard linear regressor. (The $$10^{-3}$$ factor in the exponential is to avoid numerical issues; if I just use $$e^{-x}$$ the coefficient for it becomes 0 due to the $$e^{-x}$$ features vanishing to 0 for large values of $$x$$.) This essentially amounts to finding the best-fitting linear combination of the basis functions.

It seems like this choice of basis functions yields curves (dashed) that fit pretty well to the original data (solid):

(Again, I'm not labelling the axes due to confidentiality issues.)

When I went to inspect how well the regression coefficients separated instances of each class, I found this:

The coefficient of 1 (that is, the independent term) and that of $$\frac{1}{\sqrt{x}}$$ seem to be linearly correlated. This lead me to think that perhaps one of the two basis functions was redundant. However, when I removed either, the regression performed worse. For example, removing the independent term:

So it is not necessarily true that one of these two basis functions are redundant. My question is: if the values of a subset of the regression coefficients seem to be linearly correlated, what can I infer from it, if anything?

• no no the fact that coefficients between different basis functions are correlatd DOES NOT mean that one basis function is redudant. In fact it means that both are needed in a definite ratio one to the other Feb 25 '21 at 9:50