No the cleanest solution but it works
currentyear = 2021
df["Vals"] = df.apply(lambda x: x["column1"] if x["y1"] == currentyear else x["column2"] if x["y2"] == currentyear else x["column3"], axis = 1)
Hope it helps
In my opinion, I would treat each signal on its own. The approach also depends on the signals and on your definition of anomalies/outliers (for example unexpected long peaks?). But I can point some methods that you can try if they work on your signals:
If your signal is normally distributed (or very close to normal distribution) you can remove points (or ...
The binary features obtained from one-hot encoding a categorical feature must be obtained from the training set only. This implies that any new value in the test set cannot be used.
I recommend the following method: before encoding the variable in the training set, discard all the rare values (for examples the ones which have a frequency lower than 3) and ...
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
ct = ColumnTransformer(transformers = [('encoder', OneHotEncoder(),[1,2])], remainder ='passthrough')
X = np.array(ct.fit_transform(X))
Note that you are passing [1:2] as the list of indices to apply the transformation on the ColumnTransformer but it should be ...