I used word2vec on a customer review dataset provided by my company and ran it on my local machine - Mac OS Sierra - 10.12.1(Beta) with 8gb RAM. It is working fine. But I wanted to fasten up the process and henceforth tried to run it on an AWS Instance with 16gb RAM. But when I tried to run it there, the program keeps crashing with memory issues or segmentation faults. I tried to provide 16gb of swap memory more. Even that memory gets full and the program crashes.

  • Am I missing to do something here?
  • Why can't the memory be handled by the server like my MAC?
  • Can I do something so that the program successfully runs on my server too?

Note: The code and the dataset is the same on both local machine and the server.

  • $\begingroup$ Could you share some details around the size of the dataset and the code which you are using? I am guessing that you would be using some open-source solution for word2vec. $\endgroup$ – Shagun Sodhani Oct 22 '16 at 4:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.