Questions tagged [k-nn]

K-Nearest Neighbor (K-NN) is a classification algorithm that determines the label of some data point based on the most common label of the closest k other points.

Filter by
Sorted by
Tagged with
10
votes
1answer
575 views

Learning with Positive labels only

I have ~7 million rows of customer data (~500 sparse attributes) A million out of them have opted in to a new service. How do I use this signal to predict which of the remaining customers are likely ...
7
votes
1answer
718 views

How to decide how many n_neighbors to consider while implementing LocalOutlierFactor?

I have a data set with rows: 134000 and columns: 200. I am trying to identify the outliers in data set using LocalOutlierFactor from scikit-learn. Although I ...
7
votes
1answer
69 views

How to decide the shape of input features, when each data file is of different length?

To help me understand the benefits and shortcomings of decision trees, KNN, Neural Networks, ...
6
votes
1answer
114 views

Mapping of categorical features into binary indicator features

I am currently reading an introductory machine learning book by Daumé (ch. 03, p. 30) and when discussing the mapping of categorical features with "n" possible values into "n" binary indicator ...
6
votes
1answer
85 views

How to add data points to a trained KNN

I have a trained KNN, I created with https://github.com/kevinzakka/blog-code/blob/master/knn/knn.py I want to add more data points to the KNN but I am on a raspberry pi so limited by RAM and ...
5
votes
1answer
761 views

Coordinate System's influence on $L$ distances (Manhattan and Euclidean)

I don't understand this picture, which says if we change the coordinate system, we would have the same result for $L_2$ distance, whereas, our result would differ for $L_1$ distance. What does it ...
5
votes
2answers
213 views

Is K-NN applicable for binary variables?

I need help because I'm just new to machine learning and I do not know if k-nearest neighbors algorithm can be used to identify the appropriate program(s) for Student 11 in the table below. The ...
4
votes
4answers
6k views

How to save a knn model?

I need to save the results of a fit of the SKlearn NearestNeighbors model: knn = NearestNeighbors(10) knn.fit(my_data) How do you save to disk the traied ...
4
votes
1answer
8k views

Sklearn: unsupervised knn vs k-means

Sklearn has an unsupervised version of knn and also it provides an implementation of k-means. If I am right, kmeans is done exactly by identifying "neighbors" (at ...
4
votes
2answers
7k views

Why is cross-validation score so low?

I am using Scikit-Learn for this classification problem. The dataset has 3 features and 600 data points with labels. First I used Nearest Neighbor classifier. ...
4
votes
2answers
11k views

How to calculate the silhouette coefficient?

Calculate the silhouette coefficient of point Pi from the above image. To apply the given formula, how to know which is a(i) and b(i)?
4
votes
1answer
102 views

Is there an algorithm that imputes missing values based on n nearest columns? (KNN hybrid)

I have a dataset of 70 columns that have missing values. Each column has a few columns (3-5) that it is significantly more correlated than the others but each column's correlated columns are very ...
4
votes
3answers
141 views

A metric between trees

I have certain tree structures. I am not an expert in machine learning. As I would with take KNN, I would calculate distances via metric function and a new data point and the points from the training ...
4
votes
1answer
71 views

KNN Regression: Distance function and/or vector representation for datetime features

Context: Trying to forecast some sort of consumption value (e.g. water) using datetime features and exogenous variables (like temperature). Take some datetime features like week days (...
4
votes
1answer
733 views

K Nearest Neighbour with different distance matrix to each datapoint

I'm wondering if there is library support in python (such as sklearn) for doing KNN on a data set that has a custom distance matrix (positive definite) for each data point (x is a query point, $x_i$ ...
3
votes
3answers
76 views

Optimal selection of k in K-NN

I am currently reviewing some concepts related to Machine Learning, and I started to wonder about the hyperparameter selection of K-NN classifier. Suppose you need to solve a classification task with ...
3
votes
2answers
4k views

sklearn.neighbors.NearestNeighbors - knn for unsupervised learning?

From basic theory I know that knn is a supervised algorithm while for example k-means is an unsupervised algorithm. However, at Sklearn there are is an implementation of KNN for unsupervised learning ...
3
votes
3answers
614 views

How to predict the value in KNN?

I am trying to build the KNN algorithm for IRIS dataset. First, I've computed the distance and stored it in 1d array. However, I am really struggling to build the prediction function. Therefore two ...
3
votes
1answer
4k views

Evaluation metrics for Decision Tree regressor and KNN regressor

I have started working on the Decision Tree Regressor and KNN Regressor. I have built the model and not sure what are the metrics needs to be considered for evaluation. As of now I have considered ...
3
votes
1answer
2k views

How does sklearn KNeighborsClassifier compute class probabilites?

The KNeighborsClassifier has a method for predicting class probabilities. However, I cannot find any documentation describing how these probabilities are computed. ...
3
votes
1answer
135 views

Is the distance in Nearest Neighbors a good measure of similarity?

Let's train a Nearest Neighbor model with just one sample in it: In [48]: nn = NearestNeighbors().fit([[0, 1, 0, 0]]) So this one sample has just one significant ...
3
votes
2answers
3k views

Regression for discrete values?

Im a noob in ml / statistical algorithm, but I do have worked with simple classifiers and regression I like some opinions if I am going the right way, given my limited knowledge My problem is ...
3
votes
2answers
63 views

Understanding Learning Curves

I would like to clarify my understanding of learning curves with two example plots below. I am experimenting with small data sets here between 500 and 1500 samples to clarify my understanding. My ...
3
votes
2answers
95 views

Which model is better for incremental learning?

I'm trying to implement face recognition. I'm planning to use some model (like DeepFace) to extract discriminative features and then use a classifier to recognize the faces. I'm confused as to which ...
3
votes
3answers
72 views

Why does my KNeighborsClassifier graph look like this?

I'm new to data science/ml and working on using the sklearn libraries to classify data. I'm currently using the KNeighborsClassifier with 5 fold cross validation whilst tweaking the k value but its ...
3
votes
2answers
77 views

Why isn't local averaging (including KNN) used often for regression?

My professor said that the "holy grail of regression" is the function E(Y|X=x) i.e. the conditional expectation of Y on X. In practice, you'd take a small window of X and take the average value of Y ...
3
votes
0answers
46 views

Could my AI classifier be extended to categorical data? [closed]

I created a new AI classifier. Physics Based Classifier
3
votes
2answers
50 views

find most dense neighborhood of points in high dimensional space

I'm working on a project where I have many high-dimensional points and I want to find the most dense neighborhood of them. Ideally, out of my ~500 points that are each a 4x300 matrix (300 ms time ...
3
votes
0answers
96 views

What does big O mean in KNN optimal weights?

Wiki gives this definition of KNN In pattern recognition, the k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. In both cases, the input consists ...
3
votes
1answer
433 views

k-Nearest Neighbours with time series data - how to obtain whole-time-period estimators

I have a large dataset for the activities performed by multiple staff in a factory over a long period of time - 01/01/2017 - present. The activities performed by different staff are recorded at each ...
2
votes
1answer
7k views

How Does Weighted KNN Work?

I am reading notes on using weights for KNN and I came across an example that I don't really understand. Suppose we have K = 7 and we obtain the following: Decision set = {A, A, A, A, B, B, B} If ...
2
votes
1answer
1k views

How does KNN work if there are duplicates?

I am currently debating with my friend about how KNN handles duplicates. Suppose K = 2, and we have a 1-dimensional set of data points to illustrate my dilemma I = {1, 2, 2, 2, 2, 2, 6} Thus is it ...
2
votes
1answer
273 views

K-Nearest Neighbours algorithm explanation needed

I need some explanation for K-Nearest Neighbors algorithm. Why is the training process needed in KNN algorithm? In regression models the training process means to find optimum parameters for a ...
2
votes
1answer
2k views

Knn and euclidean distance

I'm studying the knn classification algorithm. Why can the euclidean distance be considered a nice measure of affinity between examples ? In one dimension (1 attribute) this seems correct, but if I ...
2
votes
1answer
60 views

Creating a single number from a numpy array - Python [closed]

I am working on a gender classification project. I am extracting the pixels of an image using a Numpy array in Python, similar to the one below: ...
2
votes
1answer
358 views

scalable tools to build kNN graph over sparse data

I'm looking for scalable tools to build kNN graph over sparse data points. The dimension and number of data points can be both up to millions. What I have tried already: sklearn.neighbors....
2
votes
2answers
129 views

Find which type of log using machine learning?

I am the beginner of Machine learning. The process is: I have different log files (System log, MSSQL Server log, Linux log, MySQL Log, FTP log, IIS log).If any input is given, I will find out which ...
2
votes
3answers
81 views

Machine learning on classifying speech

So, I have 9k of 1 second wav files of a person speaking. These are labeled by whether the person speaking is wearing a face mask or not. I am supposed to come up with a machine learning model to ...
2
votes
1answer
91 views

KNN custom transformer shows same accuracy for every k i set

I built custom trasformer for KNN and i can't figure why my k-number, when i set it, always shows same accuracy... ...
2
votes
2answers
370 views

KNN with mixed data (feature set)

I have a dataset where the feature set consists of hour of the day (between 0 to 23), day of the week (Monday to Sunday), number of shops (a positive integer) and road category (0 to 8 on an ordinal ...
2
votes
2answers
60 views

Varying the number of neighbors in nearest-neighbors regression

For nearest-neighbors regression, it is plausible to increase the number of neighbors used to predict $f(x)$ when there are many data points near $x$. What is a good algorithm for varying the number ...
2
votes
1answer
393 views

values passed to user-defined distance function by KNeighborsClassifier is wrong

I have a data-set in which all features are binary and the class of each data-point is also binary. I am trying to use KNearestClassifier with a user-defined distance function as follows: ...
2
votes
2answers
178 views

Taking Neural Network's false positives as the recommendation system result?

I am creating a recommendation system and considering two parallel ways of formalizing the problem. One classical, using proximity (recommend the product to the customer if a majority vote of 2k+1 ...
2
votes
1answer
852 views

How can I apply PCA to KNN?

I want to know the do not want to how to use library I will denote a $n\times p$ data matrix by $X$, where $n<p$. That is, each row of $X$ is one sample data with $p$ feature variables. By using ...
2
votes
2answers
36 views

Missing value Imputation in dataset

I have two separate files for Testing and Training. In the training data, I am dropping rows that contain too many missing values . But , In the test data , I cannot afford to drop the rows so I have ...
2
votes
0answers
54 views

Classification and clustering of Time series data of temperature

I have a time series recorded data of temperature. This is what my data looks like: The change in data represents specific event or a class which I would like to detect when new incoming data. ...
2
votes
2answers
57 views

Using KNN to categorise inventory (physical stock items) - is it the best way?

I'm working on a machine learning problem involving inventory (i.e. physical retail stock), however through the cleaning (outlier removal) process some of the items (via their corresponding ...
2
votes
1answer
176 views

KNN Imputation utilize mean or mode?

In my current project, I am doing KNN imputation with K = 5 and I am using sklearn.impute.KNNImputer. I have a mix of continuous and nominal variables(encoded as 0/1 or ordinal ones that have been ...
2
votes
1answer
120 views

Question about reshaping array size for KNN Classifiers

I keep trying to run a new set of data through my KNN Classifier but would recieve the message: ValueError: query data dimension must match training data dimension ...
2
votes
1answer
70 views

Item-based recommender using K-NN

I'm trying to build an item-based recommender using k-nn. I have a list of items, all of which have some properties (features) in common. ...