2
$\begingroup$

Scenario - I have data that does not have labels but I can create a function to label the data based on behavior and deploy the model so I don't have to keep labeling the data. Is this considered machine learning ?

Objective: Classify accounts with Volume spikes based on high, medium or low labels to deploy on big data (trillions of lines of data) .

Data: The data I have includes the following attributes: Account, Time, Date, Volume amount.

Method:

  1. Create a new feature column called spike and create a pandas function to ID a spike greater than 5. Is this feature engineering ?

  2. Next I create my label column and classify it as low medium or high spike.

  3. Next I train a machine learning classifier and deploy it to label future accounts with similar patterns in big data.

Thoughts on this process ? Is this approach correct for machine learning ?

$\endgroup$
1
$\begingroup$

The approach you're describing might be good but the main question is how the automatic labeling works. You say you can "create a function to label the data based on behavior": what is the behavior? Do you manually decide the label or is entirely automatic? If it's completely automatic and general enough so that it can work for any instance, then you don't need any ML since you can directly apply your function. On the other hand if it's specific to a subset of data (or requires some amount of manual decision) then it makes sense, and the challenge will be whether the features are informative enough to predict the label for fresh instances from a different subset. ML assumes that your test instances come from the same distribution as the training set.

Create a new feature column called "label" that classifies accounts based on a user defined function that IDs spikes in volume. Is this feature engineering ?

It's not feature engineering if it's the class that you will try to predict later with new instances. Feature engineering would be selecting particular features, for example instead of volume amount you could have minimum, maximum and mean volume, or discretizing the time, etc.

$\endgroup$
2
  • $\begingroup$ if thats not machine learning then some 1 explain what it is ? im taking a subset - creating a feature (featuring egineering) and then deploying it on big data to classfy it $\endgroup$ – grim_reaper Jun 26 '19 at 23:40
  • $\begingroup$ You seem to confuse features and labels. The ML part is when you use an algorithm to estimate the function f such that f(x) = y, with x your features and y your class/label. So technically feature engineering can only be related to the features, not the label. But sure, in a very broad sense this is part of a ML process. $\endgroup$ – Erwan Jun 27 '19 at 7:16
1
$\begingroup$

From what I understood, you are trying to synthetically create the label column using a function. If the function is good enough to label the data correctly in that case you can use the same function to label new data as well.

In a supervised machine learning algorithm, you want to predict the target variable by predicting the function(model) from the given features. Here you are defining the function your selves to IMHO you do not need a ML model to do that.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.