3
$\begingroup$

I want to analyze the effectiveness and efficiency of kernel methods for which I would require 3 different data-set in 2 dimensional space for each of the following cases:

  1. BAD_kmeans: The data set for which the kmeans clustering algorithm will not perform well.
  2. BAD_pca: The data set for which the Principal Component Analysis (PCA) dimension reduction method upon projection of the original points into 1-dimensional space (i.e., the first eigenvector) will not perform well.
  3. BAD_svm: The data set for which the linear Support Vector Machine (SVM) supervised classification method using two classes of points (positive and negative) will not perform well.

Which packages can I use in R to generate the random 2d data-set for each of the above cases ? A sample script in R would help in understanding

$\endgroup$
1
  • $\begingroup$ Can you clarify what these data sets are that you are trying to generate? what have you tried in R so far? $\endgroup$ – Sean Owen Oct 21 '14 at 14:01
2
$\begingroup$

None of the algorithms you mention are good with data that has uniform distribution.

size <- 20             #length of random number vectors
set.seed(1) 
x <- runif(size)          # generate samples from uniform distribution (0.0, 1.0)
y <-runif(size) 
df <-data.frame(x,y)

# other distributions: rpois, rmvnorm, rnbinom, rbinom, rbeta, rchisq, rexp, rgamma, rlogis, rstab, rt, rgeom, rhyper, rwilcox, rweibull.

See this page for tutorial on generating random samples from distributions.


For specific set of randomized data sets that are 'hard' for these methods (e.r. linearly inseparable n-classes XOR patterns), see this blog post (incl. R code): http://tjo-en.hatenablog.com/entry/2014/01/06/234155.

$\endgroup$
6
  • $\begingroup$ How can I generate a confusion matrix for the above mentioned input data to check various Performance measures such as variance, precision, recall, F1 measure ? $\endgroup$ – Nikhil Katre Oct 20 '14 at 15:31
  • $\begingroup$ I do know that I can use the function confusionMatrix(predicted, actual) of the caret package, the only problem I am facing is with the predicted values. Since the data generated is ambiguous, I am not able to understand what the predicted values should be ? $\endgroup$ – Nikhil Katre Oct 20 '14 at 15:42
  • $\begingroup$ In the worst case (uniform distribution), there is no "predicted value" because there are no clusters in the data. For data generated by other algorithms (e.g. linearly inseparable n-classes XOR), the algorithm does generate n classes, so they would be the basis for comparing predicted (i.e. perfect classification) vs actual (what ever your ML method yields) $\endgroup$ – MrMeritology Oct 20 '14 at 17:38
  • $\begingroup$ I did get the clusters using the above dataset through kmeans. Hence I wished to find the performance metrics so that after running kernel tricks, I can compare the performance metrics before and after applying kernel tricks $\endgroup$ – Nikhil Katre Oct 20 '14 at 17:43
  • $\begingroup$ One more thing I noticed is that, the SSE generated using kmeans lies between 0.2 and 0.35, which makes me wonder whether the dataset can actually be considered 'bad' for kmeans? $\endgroup$ – Nikhil Katre Oct 20 '14 at 17:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.