0
$\begingroup$

Working on some voluminous data sets, I have been interested in methods for dimension reduction and plotting. I have stumbled upon this novel technique : UMAP (https://arxiv.org/pdf/1802.03426.pdf), which allow to reduce the dimension of a data set to plot it in 2 dimension. It seems fast and efficient.

In the main paper authors even provide some beautifull plots. As for exemple :

enter image description here

I have been unable to reproduce this kind of graphics in R with ggplot2 (computer crashing because of too many points, overlaping points...). How would one go about building a similar plot ?

$\endgroup$
  • $\begingroup$ I gave you -1, because the used software is provided in the citations. $\endgroup$ – Valentas Sep 19 at 8:15
  • $\begingroup$ I missed that. Thanks for pointing it out. Any idea on how to perform that in R ? or where to ask ? $\endgroup$ – lcrmorin Sep 19 at 8:35
1
$\begingroup$

Ggplot2 is great for simple visualisations but doesn't handle huge datasets very well, as you found out. Running out of memory is also often an issue with huge sets, and calculating prime divisibility on 30 million integers definitely counts as huge.

If you have access to more powerful systems or cloud platforms that should work a lot better. Some of the AWS and Azure etc. solutions aren't that expensive so you could try one of them.

$\endgroup$
  • $\begingroup$ I have access to a lot of calculation power if need be. What would be the code to obtain the image above ? $\endgroup$ – lcrmorin Sep 19 at 6:02
1
$\begingroup$

The following appear to function quite smoothly and can be reparametrized easily if need be.

The key was to build a new theme to apply to the ggplot2 graph and using size options (size=1, shape=".") :

library(ggplot2)

theme_black = function(base_size = 12, base_family = "") {

  theme_grey(base_size = base_size, base_family = base_family) %+replace%

    theme(
      # Specify axis options
      axis.line = element_blank(),  
      axis.text.x = element_blank(),  
      axis.text.y = element_blank(),  
      axis.ticks = element_blank(),  
      axis.title.x = element_blank(),  
      axis.title.y = element_blank(),  
      axis.ticks.length = unit(0.3, "lines"),   
      # Specify legend options
      legend.background = element_rect(color = NA, fill = "black"),  
      legend.key = element_blank(),  
      legend.key.size = unit(1.2, "lines"),  
      legend.key.height = NULL,  
      legend.key.width = NULL,      
      legend.text = element_text(size = base_size*0.8, color = "white"),  
      legend.title = element_text(size = base_size*0.8, face = "bold", hjust = 0, color = "white"),  
      legend.position = "right",  
      legend.text.align = NULL,  
      legend.title.align = NULL,  
      legend.direction = "vertical",  
      legend.box = NULL, 
      # Specify panel options
      panel.background = element_rect(fill = "black", color  =  NA),  
      panel.border = element_blank(),  
      panel.grid.major = element_blank(),  
      panel.grid.minor = element_blank(),  
      panel.margin = unit(0.5, "lines"),   

      # Specify plot options
      plot.background = element_rect(color = "black", fill = "black"),  
      plot.title = element_text(size = base_size*1.2, color = "white"),  
      plot.margin = unit(rep(1, 4), "lines")

    )

}

So the following code with noisy colors... :

n = 100000

X1 = rnorm(n = n, 0, 1)
X2 = rnorm(n = n, 0, 1)
Y = sin(5*X1*X2+rnorm(n = n, 0, 1))


df = data.frame(X1,X2,Y)

gg <- ggplot(df, aes(x=X1, y=X2)) + 
  geom_point(aes(col=Y), size=1, shape=".") + 
  scale_color_gradient(low="blue", high="red") +
  xlim(c(-5, 5)) + 
  ylim(c(-5, 5)) + 
  labs(title="Scatterplot", 
       caption = "Source: SMAE") +
  theme_black()


plot(gg)

will give the picture below.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.