Hot answers tagged

11

The topic you are interest in is called "PU learning" or "positive and unlabeled learning". You can start by having a look into survey literature.


8

Unsupervised k-NN Unlike k-means, the unsupervised k-nn does not associate a label to instances. All it can do is tell you what instances in your training data is k-nearest to the point you are polling for. For example: import numpy as np from sklearn.neighbors import NearestNeighbors samples = [[0, 0, 2], [1, 0, 0], [0, 0, 1]] neigh = NearestNeighbors(...


8

import pickle knn = NearestNeighbors(10) knn.fit(my_data) # Its important to use binary mode knnPickle = open('knnpickle_file', 'wb') # source, destination pickle.dump(knn, knnPickle) # load the model from disk loaded_model = pickle.load(open('knnpickle_file', 'rb')) result = loaded_model.predict(X_test) refer: https://www....


7

Not really sure about it, but KNN means K-Nearest Neighbors to me, so both are the same. The K just corresponds to the number of nearest neighbours you take into account when classifying. Maybe what you call Nearest Neighbor is a KNN with K = 1.


6

For example, consider the green line. What is its length? In $L_2$, the answer is $1$, in $L_1$, the answer is $1$ as well. Now, for the same line, let's rotate it $45^\circ$ counterclockwise. What is the length again? In $L_2$, its length remains to be $1$. However, in $L_1$, using Manhattan distance, it's length is now $\frac{1}{\sqrt{2}}+\frac{1}{\sqrt{...


5

Don't have enough reputation to comment to a resource, so answering this myself. About Annoy Annoy is a library being used here for finding approximate nearest neighbours, approximate being the key word here. Understanding K-NN and Approximate NN Now, let's see what is the difference with example of a problem. Say you have 10 entities (words / ...


5

weights = 'distance' is in contrast to the default which is weights = 'uniform'. When weights are uniform, a simple majority vote of the nearest neighbors is used to assign cluster membership. When weights are distance weighted, the voting is proportional to the distance value. Nearby points will have a greater influence than more distance points (even if ...


4

The class probabilities are the normalized weighted average of indicators for the k-nearest classes, weighted by the inverse distance. For example: Say we have 6 classes, and the 5 nearest examples to our test input have class labels 'F', 'B', 'D', 'A', and 'B', with distances 2, 3, 4, 5, and 6, respectively. Then the unnormalized class probabilities can ...


4

Your reasoning is correct - you should consider duplicate points as separate. You can see that this must be the case in several ways: Introduction of small random noise to the data should not affect the classifier on average. This would not be the case if you removed duplicates. Suppose that your input space only has two possible values - 1 and 2, and all ...


4

If I understand your question correctly, you're looking for something that another (beyond KD-trees) standard space partitioning algorithm does. This one is called BSP-trees (for Binary Space Partition) and it uses vectors instead of dimensions for its space subdivision/partitioning. You can see it contrasted to KD-trees in slide 29 (previewed below) and ...


4

Yes, you just have to find a suitable distance metric, instead of using the default Euclidean distance. Euclidean distance will work, but it loses a lot of its positive points when used on a non-euclidean space. For you specific case, the Jaccard distance basically measures how many 1's are equal on both instances, ignoring the dimensions where both are 0's....


4

Generally when ever we are trying to compare between models and to choose the best one, we go for other metrics like AIC, BIC, AUC(this is not applicable as it is used for classification algorithm) etc along with $R^2$. Now why are they important criteria because AIC tries to select the model that most adequately describes an unknown, high dimensional ...


4

We can view nearest neighbor as a voting process where we consult our $k$ nearest neighbor. We give the $i$-th data point a voting weight $w_i$. In your example, each data point in class $A$ has weight $\frac1{0.95}$ and each data point in class $B$ has weight $\frac1{0.05}$. There are $4$ votes from class $A$ and $3$ votes from class $B$. We give class $A$...


4

Scikit wrote in his documantation: sklearn.neighbors provides functionality for unsupervised and supervised neighbors-based learning methods. Unsupervised nearest neighbors is the foundation of many other learning methods, notably manifold learning and spectral clustering. Supervised neighbors-based learning comes in two flavors: classification for data ...


3

a(i) : the average distance between 'i' and all other data within the same cluster (source) b(i) : the lowest average distance of 'i' to all points in any other cluster, of which 'i' is not a member (source) So, from the question, a(i) will be 24 as point 'Pi' belongs to cluster A and b(i) will be 48 as it is the least average distance that 'Pi' has from ...


3

You can think of examples as vectors in $\mathbb{R}^p$, where $p$ is the number of features. Two examples will be very similar if the distance between them is close to $0$ (in the extreme case, if two examples are equal their euclidean distance is $0$). One way to measure the distance is using euclidean distance, but other distances can be used, as cosine ...


3

The unsupervised version simply implements different algorithms to find the nearest neighbor(s) for each sample. The kNN algorithm consists of two steps: Compute and store the k nearest neighbors for each sample in the training set ("training") For an unlabeled sample, retrieve the k nearest neighbors from dataset and predict label through majority vote /...


3

K Nearest Neighbors is a Classification Algorithm. Just as with every classification algorithm is important that the algorithm doesn't "remember" the answers and that the answer it gets can be generalized to all the population, not just learned from the database. 1. Why is the training process needed in KNN algorithm? In regression model the training ...


3

Importing the library from sklearn.externals import joblib Saving your model after fitting the parameters clf.fit(X_train,Y_train) joblib.dump(clf, 'scoreregression.pkl') Loading my model into the memory ( Web Service ) modelscorev2 = joblib.load('scoreregression.pkl' , mmap_mode ='r') Using the loaded object prediction = modelscorev2.predict_proba(y)...


3

This is counter-intuitive, because one would expect [0,2,0,0] to be more similar to [0,1,0,0] than [0,1,1,0]. No this is expected, since the two points are exactly at the same distance in the Euclidean space. To see it take a simplified 2D version of your points: A (1,0) B (2,0) C (1,1) Both B and C are exactly at distance 1 from A. But for my ...


3

No, you can definitely search for k-NN with more than 2-dimension data. Here is an example based on sklearn: X = [[0, 0, 0], [3, 3, 3], [1, 2, 3]] from sklearn.neighbors import NearestNeighbors neigh = NearestNeighbors(n_neighbors=2) neigh.fit(X) print( neigh.kneighbors([[2,2,2]]) ) PCA is used to reduce the input dimensionality but this is not mandatory ...


3

Overfitting is "The production of an analysis which corresponds too closely or exactly to a particular set of data, and may therefore fail to fit additional data or predict future observations reliably." (Oxford dictionary) When you fit a ML model, you use a dataset that you assume is a sample of the real statistical distribution you want to model. ...


3

Your idea isn't wrong, however in k-NN there always might be a case where you have the same number of votes for 2 or more classes (e.g. you have $k=6$ and you have 3 samples of one class vs 3 of another). With your solution you are just overcoming one very small case of ties and the $k$ that you choose might not be the optimal $k$ for classification, which ...


3

As I was playing with this problem, Djib just wrote an answer which is certainly better than whatever I could have come up with. To illustrate Djib's point, here is a small demonstration that as soon as there are more than 2 classes there's no value of $k$ which guarantees the absence of tie (except if $k=1$ of course). By definition we have $k=|c_1| + |c_2| ...


2

I know this question has been here for two years, however, I was having the same problem when using cross_val_score on my data and I ended up here. The results returned from the cross_val_score function were very different from what I get when I do cross validation manually using train_test_split, as you were doing with the Nearest Neighbor classifier. ...


2

In your random forest, this is due to the fact that your final model is overfitting. Sklearn's GridSearchCV has a default argument refit = True, that takes the model with the best performance based on cross-validation and retrains it in the whole dataset. Your accuracy score is very high due to the fact that it is only measured on your training data, and the ...


2

The confusion comes from the way Sklearn designed their code. Short answer The "unsupervised" version you mention is not a K-Nearest Neighbour algorithm (which is implemented here). In its description, it only reads: "Unsupervised learner for implementing neighbour searches." This learner is actually used by KNNClassifier in order to perform neighbour ...


2

I think what you might be looking for is, L2Knng: Fast Exact K-Nearest Neighbor Graph Construction with L2-Norm Pruning They have multiple runtime options specifically for different kinds of datasets (including sparse data). The link for the same is : http://glaros.dtc.umn.edu/gkhome/node/1162


2

Data exploration I would suggest exploring the data a little further, which might help decide what would be the best approach for this bird-song dataset. For example, have a look at the spectrogram of each bird (there are only 66 different genus types), to see how you might extract more data from the samples. Here is the spectrogram of a sample taken from ...


2

Welcome to Data Science! You're question needs a little more detail... there are many many ways to make an array into a single number. You should say a little more about what you mean by meaningful. Here are a few examples, using your example array, which may seem outrageously simple, but do indeed form the basis to many of the techniques used in modern ...


Only top voted, non community-wiki answers of a minimum length are eligible