# Find the Outlier

I have data that contains points (geo coordinates of a random planet, integer pairs) that represent places where land is definitely there. Here is an example with points representing the Earth.

My goal is to go through samples (also integer pair points, but we don't know their values) and determine where the land is and where it isn't.

I tried to use Novelty/Outlier Detection and used sklearn here are the algorithms that I used:

prediction1 = OneClassSVM(nu=2e-1, kernel='rbf', gamma=2.5e-3).fit(data).predict(samples)
prediction2 = OneClassSVM(nu=0.01, kernel='rbf', gamma=2.5e-3).fit(data).predict(samples)
predictions = np.maximum(prediction1, 0, prediction1) + np.maximum(prediction2, 0, prediction2)

predictions = np.where(predictions >= 1, 1, 0)


And a prediction (for creation of the image I just test each point)

Here you can see the separation between layers

My problem is that my approach is not accurate enough, so I'd like to hear about other algorithms that could help solve this problem and hopefully will be better suited.

Here are some more predictions and data points so you can experiment with this

• Welcome to DataScienceSE. I suspect that the only way to obtain better than these methods would be to have a larger data sample. Jan 6, 2022 at 23:55
• @Erwan there must be a way to do something, maybe write your own algorithm, which is not in sklearn
– user130643
Jan 7, 2022 at 8:07

I don't know if my method gives better accuracy than yours but I think you can find some insights from my approach that you can use to further improve your results.

Unlike your approach of using an ensemble of models on the entire dataset, I've tried using the fact that we will have clusters of land(continents for example) for a given dataset and hence I've tried to fit one OneClassSVM for each such cluster:

1. Data Preparation:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.svm import OneClassSVM

plt.rcParams.update({"figure.facecolor": "w"})

1. Clustering using KMeans(n_clusters=5):
kmeans = KMeans(n_clusters=5)
kmeans.fit(earth_df)
clusters = kmeans.predict(earth_df)
centroids = kmeans.cluster_centers_
earth_df['cluster_no'] = clusters



Output:

1. Visualization via Color-Encoding:
colors = ['green', 'red', 'black', 'yellow', 'maroon']
clusters = range(5)

_, ax = plt.subplots(figsize=(7,7))
ax.axis('off')
ax.set_title("Color-Coded Clusters")

for cluster_no, color in zip(clusters, colors):
cluster = earth_df[earth_df['cluster_no'] == cluster_no]
ax.scatter(cluster.X, cluster.Y, color=color, marker='.')

ax.legend(clusters)

ax.scatter(centroids[:,0], centroids[:,1], marker='x', color='cyan', s=150)
plt.show();


Output:

1. Fitting one SVM per cluster:
svms = []
for cluster_no in clusters:
svm = OneClassSVM(kernel='rbf', gamma=0.0025, nu=0.2,
tol=0.001, shrinking=True, max_iter=- 1)
cluster = earth_df[earth_df['cluster_no'] == cluster_no]
cluster = cluster.drop(columns='cluster_no')
svm.fit(cluster)
svms.append(svm)

1. Visualizing the results in form of Decision Function & Prediction, separately for each SVM:
data = earth_df.drop(columns='cluster_no')
_, axs = plt.subplots(5, 2, figsize=(14, 30))
for i in clusters:
df = svms[i].decision_function(data)
prediction = svms[i].predict(data)

ax_df, ax_pred = axs[i]
ax_df.axis('off')
ax_df.set_title(f"Decision Function for Cluster-{i} SVM")
ax_df.scatter(data.X, data.Y, c=df, cmap='coolwarm')

ax_pred.axis('off')
ax_pred.set_title(f"Prediction for Cluster-{i} SVM")
ax_pred.scatter(data.X, data.Y, c=prediction, cmap='coolwarm')

plt.show();


Output:

1. Visualizing for two dummy locations:
fig, ax = plt.subplots(figsize=(6, 6))
ax.axis('off')
ax.set_title("Points for earth")

custom_points = np.array([[-12, -36], [2, 7]])

ax.scatter(earth_df['X'], earth_df['Y'], color='black', marker='.')
ax.scatter(custom_points[:,0], custom_points[:,1], color='cyan')

plt.show();


Output:

1. Predicting on dummy location:
for i, svm in enumerate(svms):
print(f"For SVM-{i}:", svm.decision_function(custom_points))


Output:

For SVM-0: [-5.5578652  -5.55743803]
For SVM-1: [-5.12195068 -5.1219504 ]
For SVM-2: [-7.12232844 -6.28086617]
For SVM-3: [-5.38072626 -5.43922668]
For SVM-4: [-4.0920019   0.02235967]


I've made no optimization on hyperparameters and used the ones that you provided as it is. A point denoting land is expected to perform well on any one of the SVM and you could control the degree of distance before which it starts being predicted as an outlier(as opposed to using .predict() directly).

Ideally, we'd do this via a validation set but I have skipped that part. This is more memory intensive than a basic 3-model ensemble but gives your models an easier subtask.

• Hmm, thanks for the idea, that sounds interesting! I will try to use this in my solution.
– user130643
Jan 7, 2022 at 14:31