Problem Statement:
Given the details about a product, we need to map it to its category.
Currently we are using Product Name as a feature and Product Category as the Label
There are around 50,000 categories available currently and it will grow in future.
I created a small dataset which consisted 20 categories and 100 records for each label. so the total record count is 2000. Using RandomForest I got 92% accuracy.
Problem:
So I went on to create a model with 1800 categories[labels] and records for each categories varies from 500-1500. When I ran the same model with new dataset I got only 19% accuracy and more than 50% of the predicted value pointed to the same label.
Dataset Sample:
Product_Combined Category
2Pcs Led Light Lamp Strip Dimmer Switch Brightness Adjustable Control 12-24V 8A Arts, Crafts & Sewing | Painting, Drawing & Art Supplies | Drawing | Light Boxes
10 Pcs 1/4" Male To 1/4" Female Screw Adapter For Tripod Camera Flash Bracket Stand Arts, Crafts & Sewing | Painting, Drawing & Art Supplies | Drawing | Light Boxes
L-Fine A4 Tracing LED Light Pad Box(13.86x9.45 Inches) with Adjustable Light Intensity for Artists,Drawing, Sketching, Animation Arts, Crafts & Sewing | Painting, Drawing & Art Supplies | Drawing | Light Boxes
BZONE Solar Powered Operated Copper Wire LED Fairy Light Decorative String Lights for Indoor Outdoor Home Garden Lawn Patio Party Christmas Valentine''s Day (16.4ft, Pink Color) Arts, Crafts & Sewing | Painting, Drawing & Art Supplies | Drawing | Light Boxes
LitEnergy 32.5 Inch Diagonal A2 Tracing Table with LED Light and Paper Arts, Crafts & Sewing | Painting, Drawing & Art Supplies | Drawing | Light Boxes
Code:
import string
import codecs
import pandas as pd
import numpy as np
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from stemming.porter2 import stem
from sklearn.metrics import confusion_matrix
from nltk.stem import PorterStemmer
from nltk.corpus import stopwords
from sklearn.model_selection import cross_val_score
from sklearn.externals import joblib
stop = stopwords.words('english')
data_file = "Book3.txt"
#Reading the input/ dataset
data = pd.read_csv( data_file, header = 0,
delimiter= "\t", quoting = 3, encoding = "ISO-8859-1")
data = data.dropna()
#Removing stopwords, punctuation and stemming
data['Product_Combined'] = data['Product_Combined'].apply(
lambda x: ' '.join([word for word in x.split() if word not in (stop)]))
data['Product_Combined'] = data['Product_Combined'].str.replace(
'[^\w\s]',' ').replace('\s+',' ')
data['Product_Combined'] = data['Product_Combined'].apply(
lambda x: ' '.join([stem(word) for word in x.split()]))
train_data, test_data, train_label, test_label = train_test_split(
data.Product_Combined, data.Breadcrumb, test_size=0.3, random_state=100)
RF = RandomForestClassifier(n_estimators=100)
vectorizer = CountVectorizer( max_features = 50000, ngram_range = ( 1,3 ) )
data_features = vectorizer.fit_transform( train_data )
RF.fit(data_features, train_label)
test_data_feature = vectorizer.transform(test_data)
Output_predict = RF.predict(test_data_feature)
print ("BreadCrumb_Accuracy: " + str(np.mean(Output_predict == test_label)))
with codecs.open("out_bread_crumb.txt", "w", "utf8") as out:
out.write("Input\tPredicted\tActual\n")
for inp, pred, act in zip(test_data, Output_predict, test_label):
try:
out.write("{}\t{}\t{}\n".format(inp, pred, act))
except:
continue
Output:
Input Predicted Actual
Centuri Duster Dispos Compress Gas Duster 10 oz 2 Pk Automotive | Exterior Accessories | Towing Products & Winches | Winches Electronics | Computers & Accessories | Computer Accessories & Peripherals | Cleaning & Repair | Compressed Air Dusters
BB Mall Phone Ring Stand Metal Stainless Steel Univers 360 Rotat Ring Kickstand iPhon 6 6s 6 s plus Samsung Note 5 Note 4 S5 iPad All SmartPhon Tablet Black Automotive | Exterior Accessories | Towing Products & Winches | Winches Cell Phones & Accessories | Accessories | Mounts & Stands | Stands
Standard Motor Product 6444 Ignition Wire Set Automotive | Exterior Accessories | Towing Products & Winches | Winches Automotive | Replacement Parts | Ignition Parts | Spark Plugs & Wires | Wires | Wire Sets
Walker 52271 Extension Pipe Automotive | Exterior Accessories | Towing Products & Winches | Winches Automotive | Replacement Parts | Exhaust & Emissions | Exhaust Pipes & Tips
ACDelco KS10640 Profession Time Compon Seal Automotive | Exterior Accessories | Towing Products & Winches | Winches Automotive | Replacement Parts | Bearings & Seals | Seals | Camshafts
As you can see more than 50% of the actual test data was labeled as Automotive | Exterior Accessories | Towing Products & Winches | Winches