-1
$\begingroup$

I'd like to run a model on RStudio Server, but I'm getting this error.

Error: cannot allocate vector of size 57.8 Gb

This is what my data looks like and it has 10,000 rows.

   latitude longitude                 close_date close_price
1  1.501986  86.35068 2014-08-16 22:25:31.925431   1302246.3
2 36.367095 -98.66428 2014-08-05 06:34:00.165876    147504.5
3 36.599284 -97.92470 2014-08-12 23:48:00.887510    137400.6
4 67.994791  64.68859 2014-08-17 05:27:01.404296    -14112.0

This is my model.

library(caret)
training.samples <- data$close_price %>%
  createDataPartition(p = 0.8, list = FALSE)
train.data  <- data[training.samples, ]
test.data <- datatraining.samples, ]

model <- train(
  close_price~., data = train.data, method = "knn",
  trControl = trainControl("cv", number = 1),
  preProcess = c("center","scale"),
  tuneLength = 1
)

My EC2 instance has more than 57 GB available. This is the memory.

             total       used       free     shared    buffers     cached
Mem:      65951628     830424   65121204         64      23908     215484
-/+ buffers/cache:     591032   65360596
Swap:            0          0          0

And it has enough storage space, too. This is the hard drive space.

Filesystem     1K-blocks    Used Available Use% Mounted on
devtmpfs        32965196      64  32965132   1% /dev
tmpfs           32975812       0  32975812   0% /dev/shm
/dev/xvda1     103079180 6135168  96843764   6% /

And these are details on the machine.

R version 3.5.3 (2019-03-11)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Amazon Linux AMI 2018.03
| improve this question | | | | |
$\endgroup$
1
$\begingroup$

It doesn't matter that your instance has more than 57.8GB. R is asking for another 57.8GB on top of whatever it is already using. Not to mention any operating system overhead.

Shrink your dataset and see if it what you are doing works at small scale before trying to do it at big scale. Maybe it will work on 9,917 rows and that's good enough. Or it might fail on 500 rows in which case you need to rethink what you are doing.

| improve this answer | | | | |
$\endgroup$
0
$\begingroup$

You need to increase the amount of RAM available for R.

Use memory.limit(). You can increase the default using this command, memory.limit(size=2500), where the size is in MB.

Check out the R documentation for more details. https://www.rdocumentation.org/packages/utils/versions/3.4.1/topics/memory.size

| improve this answer | | | | |
$\endgroup$
  • $\begingroup$ My memory limit is set to Inf and that is only for Windows machines. $\endgroup$ – Cauder May 18 '19 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.