23

Very interesting question (+1). While I am not aware of any software tools that currently offer comprehensive functionality for feature engineering, there is definitely a wide range of options in that regard. Currently, as far as I know, feature engineering is still largely a laborious and manual process (i.e., see this blog post). Speaking about the feature ...


10

There is no definite source on how to do feature engineering. It is often dependent on the problem you are trying to solve. Some say it is more of an art than it is science. But I would go through some of the high scoring kaggle kernels / winning solutions if available. Just head over to kaggle and browse through the competitions. There is a lot of very ...


7

1) Yes, it makes sense. Trying to create features manually will help the learners (i.e. models) to graspe more information from the raw data because the raw data is not always in a form that is amenable to learning, but you can always construct features from it that are. The feature you are adding are based on one feature. This is common. However, your ...


7

A note: for those who've ended here looking for a hashing technique, geohash is likely your best choice. Representing latitude and longitude in a single linear scale is not possible due to the fact that their domain is inherently a 3D space. Reducing that as per your needs would require a spatial flattening technique that's unheard of to me. Reasoning As ...


6

There's three main approaches to solving this: Building two models separately and then training an ensemble algorithm that receives the output of the two models as an input Concating all the data into a single vector/tensor as a preprocessing step and then train a simple single input NN The multiple input NN architecture you proposed The ensemble approach ...


6

This is an old question. I am surprised that I don't see anyone mentioned Mean Encoding (a.k.a Target Encoding). It is very popular in supervised learning problems. Besides, I have seen people use frequency or the cdf of the frequency (to avoid noise generated by heavy-tailed pdf), and they achieved pretty good results with lightGBM. However, i cannot really ...


6

One-hot-encoded ZIP codes shouldn't present a problem with modern tools, where features can be much wider (millions, billions even), but if you really want you could aggregate area codes into regions, such as states. Of course, you should not use strings, but bit vectors. Two other dimensionality reduction options are MCA (PCA for categorical variables) and ...


6

Use the extra features for unsupervised learning. You might enjoy Vladimir Vapnik's take on this in the context of SVMs, which he calls privileged learning: Learning with Intelligent Teacher: Similarity Control and Knowledge Transfer


6

If you can keep adding new data (based on a main concept such as area i.e. the ZIP code) and the performance of your model improves, then it is of course allowed... assuming you only care about the final result. There are metrics that will try to guide you with this, such as the Akaike Information Criterion (AIC) or the comparable Bayesian Information ...


5

Featuretools is a recently released python library for automated feature engineering. It's based on an algorithm called Deep Feature Synthesis originally developed in 2015 MIT and tested on public data science competitions on Kaggle. Here is how it fits into the common data science process. The aim of the library is to not only help experts build better ...


5

Why don't we convolve our images against the last convolution layer and see how many of these complex feature filters get activated? The answer is that all the layers are fully dependent on the exact features of the previous layer. The last layer is simply not capable of taking a raw image as input and outputting meaningful values. Most likely it is not ...


5

This sort of effect can be achieved with pandas.DataFrame.resample() combined with Resampler.aggregate() like: Code: df.resample("1Min").agg({'A': sum, 'B': np.mean}) Test code: df = pd.read_fwf(StringIO(u""" A B 2017-01-01T00:01:01 0 100 2017-01-01T00:01:10 1 200 2017-01-01T00:01:16 2 300 ...


4

Missing Data Imputation: Complete case analysis Mean / Median / Mode imputation Random Sample Imputation Replacement by Arbitrary Value Missing Value Indicator Multivariate imputation Categorical Encoding: One hot encoding Count and Frequency encoding Target encoding / Mean encoding Ordinal encoding Weight of Evidence Rare label encoding BaseN, ...


4

Yes I think so. Just by looking at Feature Learning and Feature extraction you can see it's a different problem. Feature extraction is just transforming your raw data into a sequence of feature vectors (e.g. a dataframe) that you can work on. In feature learning, you don't know what feature you can extract from your data. In fact, you will probably apply ...


4

Given that in your training data this feature has different values and some predictive power, I think not keeping this feature would be a mistake (without looking into overfitting due to having too many features). You cannot just discard the feature from your training set if it does influence the target because then these would be from a different population ...


4

Usually, the richer the features the better. One thing to keep in mind, however, regressions, in general, do not work well with data that is highly correlated (multicollinearity). When you expand your features this way, it is something you might want to keep in mind. There is a lot of information on this very topic (and potential ways to mitigate), just ...


3

The first question is: what to do you want to see in user profile? Top-10 tracks, top-10 artist by user? How many tracks/artists a user listens to in a day on average (may be, in last month)? May be you want to get some general information related to the whole user base: Which artist/track is the most popular among users from different countries (top-N of ...


3

You can use embedding which is mentioned in the comments. e.g. A general blog post, Keras documentation for embedding layer which can be used to learn the embedding. This is widely used by deep learning models when you need to reduce the number of features and it works for one categorical feature as well.


3

Yes, you can mix any different sort of inputs when the scales of the features are similar, which is achieved by normalising the feature vectors. I assume you mean too many features when you say 'too much input' If you mean the size (number of training examples) of input data, size of input data is not directly related to overfitting. Overfitting depends on ...


3

The main reasons for seeking an efficient feature selection are the machine learning algorithm get faster training, reduces the complexity of a model, facilitates interpretation and improves the accuracy of a model. Look for Filter Methods , Wrapper Methods and Embedded Methods to learn more about your issue. Filter methods are generally used as a ...


3

The best practice is to not attempt to flatten Earth into a onee dimensional line... Because as you may know, Earth more resembles a sphere than a line. It is much better to treat it as such properly. There do exist approaches to flatten a k-dimensional space into a one dimensional order though. These are known as space filling curves and are from the 19th ...


2

Scikit-learn has recently released new transformers that tackle many aspects of feature engineering. For example: You can do multiple missing data imputation techniques with the SimpleImputer (http://scikit-learn.org/stable/modules/generated/sklearn.impute.SimpleImputer.html), including mean, median and arbitrary value imputation in both numerical and ...


2

I think there might be a problem in the way you are stating the problem. You say that you test data doesn't have two fields, but that can not be correct. You have to take all your data and split it into 2 groups, the training set and the test set. In a proportion of 80%-20% or 70%-30%. Then you train your algorithm with the data in the training set, and ...


2

You can download a free as in beer software Qlikview that allows you to do interactive data discovery via graphical interface similar to Excel but also featuring a powerful scripting language for data load and transformation. Huge flat files is no problem at all. It is an in-memory technology so you'd need a computer with a lot of RAM. The advantage though ...


2

Feature Engineering is at the heart of Machine Learning and is rather laborious and time consuming. There have been various attempts at automating feature engineering in hopes of taking the human out of the loop. One specific implementation that does this for classification problems is auto-sklearn. It uses an optimization procedure called SMAC under the ...


2

Multiplication is not a linear operation. Your linear SVM constructs a (hyper-)plane $$ w_0 = w_1 x_1 + w_2 x_2 $$ for some weights $w_0, w_1, w_2.$ By introducing the AND-feature, you add another dimension: $$ w_0 = w_1 x_1 + w_2 x_2 + w_3 x_1 x_2. $$ It might well be that your two-dimensional data set is not linearly separable, but the three-dimensional ...


2

I assume that you are solving a supervised classification problem, that is, you train your model on a labeled sample. I can think of two approaches to this problem. I. Classify tags, use neighbor tags for the features For each tag you can calculate features like: (x, y) distances to the closest image (x, y) distances to the closest image above this tag ...


2

Another approach to evaluate features is called Permutation Importance. In short, this approach is random sampling values of each feature and each time measuring the negative impact this has on the model performance. The feature for which random sampling of its values has highest negative impact on the model performance is considered as most important to the ...


2

In the cartpole example, a state-action feature could be $$\begin{bmatrix} \text{Cart Position}\\ \text{Cart Velocity}\\ \text{Pole Angle}\\ \text{Pole Tip Velocity}\\ \text{Action} \end{bmatrix}$$ where Action is either left, right, or do nothing. The reward is not part of the feature vector because reward does not describe the state of the agent; it is ...


2

A feature vector is a vector that is containing basis functions. These basis functions are combining states and actions. We can use a feature vector to approximate our action-value function $q(\boldsymbol{s},\boldsymbol{a})$. If for example $\boldsymbol{\Phi}(s,a)=[\phi_0, \phi_1(\boldsymbol{s},\boldsymbol{a}), \ldots,\phi_m(\boldsymbol{s},\boldsymbol{a})]^T....


Only top voted, non community-wiki answers of a minimum length are eligible