Cosine similarity problem

Question

I'm new to ml and am trying out some tiny projects. I have this snippet.

import pandas as pd
import json
import logging
import numpy as np
import matplotlib.pyplot as plt
from utils.utils import fetch_data, preprocess_data_cls, scale_features
from sklearn.metrics.pairwise import cosine_similarity

with open('configs/config_nn.json', 'r', encoding='utf-8') as f:
    config = json.load(f)

DATA_PATH = config["DATA_PATH_TRN"]
RANDOM_STATE = config["RANDOM_STATE"]
ARGUMENT = 'suspicious_nn'

fraud_nn_df = fetch_data('fraud_nn')
no_fraud_nn_df = fetch_data('no_fraud_nn')
suspicious_nn_df = fetch_data('suspicious_nn')

conc_df = pd.concat([fraud_nn_df.drop(columns=['label']), no_fraud_nn_df.drop(columns=['label'])], ignore_index=True)
conc_labels = pd.concat([fraud_nn_df['label'], no_fraud_nn_df['label']], ignore_index=True)

conc_prp, _ = preprocess_data_cls(conc_df, 'TRNC')
conc_prp_scaled = scale_features(conc_prp)

suspicious = suspicious_nn_df.drop(columns=['label'])
suspicious_prp, _ = preprocess_data_cls(suspicious, 'SPS')

suspicious_prp_scaled = scale_features(suspicious_prp)
suspicious_cosine_sim = cosine_similarity(suspicious_prp_scaled, conc_prp_scaled)
suspicious_nn_labels = []

for i in range(suspicious_nn_df.shape[0]):
    most_similar_index = np.argmax(suspicious_cosine_sim[i])
    suspicious_nn_labels.append(conc_labels.iloc[most_similar_index])

suspicious_nn_df['label'] = suspicious_nn_labels

print(suspicious_nn_df.head())

There are three dfs: one for fraud transactions (label 1), one for no fraud transactions (label 0), and one for questionable transactions (label -1). I'd like to classify the suspicious data based on whether it's closer to one or zero. Is the above technique appropriate?

Thank you in advance.

Answer 1

Your approach is overall correct, however here are a few observations:

• your classification technique consists in taking the label of the closest neighbor, in the sense of cosine similarity. You coded this by hand, which is fine, but is optimized in scikit-learn KNeighborsClassifer, with n_neighbors=1 in your case;
• you can use any classifier that exists (typically you can try out many of the ones implemented in scikit-learn, like RandomForest, SVC, ...). The good practice is to try many of them and pick the best one;
• your labelled data should be split into three parts: a train set, a validation set and a test set. Train and validation are here to help you select the best classifier (including its hyperparameters), and test set is here to give you a performance. In terms of performance, you may want to have a look at scikit-learn metrics;
• beware of your preprocessing and scaling steps: they should be the same for your labelled data and your "suspicious" data. For instance, if you have a min-max scaling like $X_i^{new} = \frac{X_i-min(X)}{max(X)-min(X)}$, your $max$ and $min$ should always be the ones of your training data (to be able, for instance, to classify a single new point of data). You can use scikit-learn MinMaxScaler or StandardScaler