I have a semi structured data set. I need to collect some data (unlabeled) randomly for labeling.
As initiative at first I separated labeled and unlabeled data. Then I convert those data from string to numeric as all data were string.
After that I cluster unlabeled data into 2(cluster 0 and cluster 1) by applying k-means clustering algorithm.
Now when I collect some data randomly from both cluster it takes indexes from both labeled and unlabeled data. But my expectation was, indexes will come just from unlabeled data as cluster was created by using unlabeled data.
I couldn't figured out the problem. I am giving my code below.
The indexes of labeled data stats from 0 to 343 (344 data) and unlabeled data starts from 344 to 863 (519 data)
# define X and y feature_cols = ['TopLeft', 'TopMiddle', 'TopRight', 'MiddleLeft','MiddleMiddle', 'MiddleRight', 'BottomLeft', 'BottomMiddle', 'BottomRight'] # X is a matrix, hence we use  to access the features we want in feature_cols X =train_unlabeled[feature_cols]` # y is a vector, hence we use dot to access 'label' y =train_unlabeled['Class'] X_scale = scale(X) reduced_data = PCA(n_components=None).fit_transform(X_scale) kmeans = KMeans(init='k-means++', n_clusters=2, n_init=10) model=kmeans.fit(reduced_data) reduced_data.shape
Now I was trying to collect 95% data from each cluster (cluster was created by
x = 0.95 i=0 C_i = np.where(labels == i).tolist() n_i = len(C_i) sample_i = np.random.choice(C_i, int(x * n_i)) print (i, sample_i) list1=(sample_i) x = 0.95 i=1 C_i = np.where(labels == i).tolist() n_i = len(C_i) sample_i = np.random.choice(C_i, int(x * n_i)) print (i, sample_i) list2=(sample_i)
When I print the index while collecting data randomly depending on cluster(0 and 1) I found these indexes
As the index of unlabeled data is started from 344, so how the index of labeled data comes in cluster 0 and ? I didn't understand it.