0
$\begingroup$

Days ago,One AI financial service provider offered us a lesson and mentioned that you are supposed to perform specific feature engineering according to the specific algorithm you are using.For example,when using logistics regression,fitting more features（uncorrelated）like binning the continuous variable into discrete ones are often suggested.Because logistics regression is a simple algorithm and we try to raise dimension in the way that samples will be separated better.

I searched a lot(maybe not yet),most of materials are "why/what feature engineering important","scaling/standardization/binning continuous variable","dealing with null value" or some theoretical comments with no discrete manipulation.

why and how the specific feature engineering should work on specific algorithm.Or any advice on this saying,is it right or wrong?what do you think.any comments are appreciated.

（I am not good at English,sorry about that if I am not clear enough）

I am not looking forward a detailed answer,some deep thinking about this part is good.

| improve this question | |
$\endgroup$
  • $\begingroup$ Hi Lucy, what is your specific question? Are you seeking a course on feature engineering, a quote with vision on the subject matter or are you simply trying to validate the assertion that the provider made? $\endgroup$ – S van Balen Feb 28 '18 at 10:50
  • $\begingroup$ @SvanBalen why and how the specific feature engineering should work on specific algorithm? $\endgroup$ – Lucy Feb 28 '18 at 13:54
  • $\begingroup$ Perhaps I don't understand you right, but there is no panacea for feature engineering. It is more of an art than a script. $\endgroup$ – S van Balen Feb 28 '18 at 14:25
  • 1
    $\begingroup$ So just to be more clear: Some methods are more vulnerable for uncorrelated features (Treelearners, KNN), some less (Naive Bayes), some methods can not find 2nd order correlations (Naive Bayes) some can very well (Treelearners), some cannot handle missing values (neuralnets), etc. But I could not distill all of that in one answer $\endgroup$ – S van Balen Feb 28 '18 at 14:42
1
$\begingroup$

In general, features are engineered so as to retain optimum relevant information present in the dataset with succinct representation and then features are adapted so that an algorithm can accept it as input.

Feature engineering generally involves methods like binning, PCA, etc. Adapting those features to pass to algorithm is another step which is a very small part of the feature engineering step. Adapting example: for an image, we may have to reshape the image like below so that x[0] points to the image

x = image.img_to_array(img)
normalise(x) // feature engineering
x = x.reshape((1,) + x.shape) // Adapting for algorithm

With this understanding, if features are generated once for a dataset, it may be used for any relevant algorithm with corresponding adaption.

| improve this answer | |
$\endgroup$
0
$\begingroup$

Some algorithms like boosting trees (XGBoost, for example) easy deals with almost anything "strange" in data: NaN values, outliers, different scalings. And such algorithms can find complex feature interactions in data by itself. But if you use some different or simple algorithms such as linear regression, you have to do some features preprocessing to get good results from it: fill or drop NaNs and outliers; make similar scales for all features; try to make polynomial features and so on. Otherwise you can get strange or very inaccurate results.

| improve this answer | |
$\endgroup$
  • $\begingroup$ This sounds somewhat oversimplified. Yes more advanced parametric methods are designed to handle some problems, but no method is the holy grail. In my experience XGBoost can definitely helped by feature engineering. It can, for instance, be vulnerable for noisy features, like most tree based learners. $\endgroup$ – S van Balen Feb 28 '18 at 14:29
  • $\begingroup$ Yeah, feature engeneering is an art and science, it can't be comprehended or explained so easy ) $\endgroup$ – CrazyElf Feb 28 '18 at 15:37
  • $\begingroup$ @CrazyElf each algorithm matchs their feature engineering might have their explaination from theoretical or applied way.that's what I am confused and most of material on line are oversimplified $\endgroup$ – Lucy Mar 1 '18 at 0:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.