Converting Text Entries into values for machine learning - KNN

Question

Consider the following dataset.

| Area | Job Type | Complete |
-------------------------------
| AAA  | Install  |     N    |
| AAB  | Repair A |     Y    |
| OOC  | Repair C |     Y    |
| LCX  | Cease    |     N    |
-------------------------------

I am using JavaScript (Started getting into Tensorflow.js amongs other ML algorithms and theories.) and I am struggling to find a suitable ML method to process this. There could be over 100 areas and over 15 Job Types, but Complete can only be Y/N.

I was thinking about assigning a number to each case as follows

AAA -> 1    |   Install  -> 1   |    Y -> 1
AAB -> 2    |   Repair A -> 2   |    N -> 0
OOC -> 3    |   Repair C -> 3   |
LCX -> 4    |   Cease    -> 4   |
... -> x    |

Is this viable? would it work?

I want to give it another case and return the % chance of that case happening. I have tried to use Naive Bayes classifier and has some success.

-------------------------------
| LCX  | Cease    |     10%    |
-------------------------------

This is just a small sample there are other x's that I want to include that are a mix of floating points and other String Values. the whole main data-set contains over 40 million entries and over 40 possible columns that could be a factor that effects 'completed' so plenty of training data to go around!

What would the best method be what approach would you recommend?

Answer 1

It is not recommended to use the method you suggested of converting text to integer values (unless you are using decision trees or XGBoost as the predictive model). If you are usin KNN (and many other models) you should use a technique called one-hot encoding. The reason for this is that (using your example above) Install is 1 and Repair A is 2, but Install isn't necessarily quantitively twice as small as Repair A but you are assigning them 1 and 2 which are quantitively related (2 is twice as big as 1). So your model will interpret it that way.

That is why you need to use One Hot encoding.

I'm not privy to ML in JS but in python you could do: With pandas:

import pandas as pd

df = pd.DataFrame({
      'A':['a','b','a'],
      'B':['b','a','c']
    })
df
Out[]: 
   A  B
0  a  b
1  b  a
2  a  c

# Get one hot encoding of columns B
one_hot = pd.get_dummies(df['B'])
# Drop column B as it is now encoded
df = df.drop('B',axis = 1)
# Join the encoded df
df = df.join(one_hot)
df  
Out[]: 
   A  a  b  c
0  a  0  1  0
1  b  1  0  0
2  a  0  0  1

Maybe someone else can give you specific code in JS.