Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [cloud-computing]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
41 votes
6 answers
54k views

Are there free cloud services to train machine learning models?

I want to train a deep model with a large amount of training data, but my desktop does not have that power to train such a deep model with these abundant data. I'd like to know whether there are any ...
Green Falcon's user avatar
  • 14.1k
34 votes
5 answers
13k views

What are the use cases for Apache Spark vs Hadoop

With Hadoop 2.0 and YARN Hadoop is supposedly no longer tied only map-reduce solutions. With that advancement, what are the use cases for Apache Spark vs Hadoop considering both sit atop of HDFS? I've ...
idclark's user avatar
  • 521
8 votes
1 answer
41k views

How to download dynamic files created during work on Google Colab?

I have two different files and on the first, I tried to save data to file as: np.save(open(Q1_TRAINING_DATA_FILE, 'wb'), q1_data) On second file, i'm trying to ...
vikbehal's user avatar
  • 185
3 votes
2 answers
3k views

Different number of images in classes

I am working on a deep learning CNN project. The dataset contains more than 500 classes and the classes have different numbers of items (images). For example, some of the classes have 5 images and ...
N.IT's user avatar
  • 1,995
3 votes
2 answers
102 views

Alternatives to GCP / AWS / Azure

Can anyone recommend an alternative to the big 3 cloud computing alternatives? I know they're the best but I found them overly complicated because they cater to massive enterprises. The amount of set ...
lara_toff's user avatar
  • 221
2 votes
1 answer
1k views

Train a deep reinforcement learning model using two computers

I would like to know if there is a way to train a deep rl model using two different computers. The first one would execute the game and send requisitions to the second computer which would store and ...
Matheus Prandini's user avatar
2 votes
0 answers
55 views

How much data and computation power do I need to train a machine translation model using Transformer architecture?

I am working right now on creating a dataset to use in creating a machine translation model to translate between two dialects. I have two questions that I am trying to find an answer for: How much ...
user13288833's user avatar
1 vote
1 answer
69 views

Which cloud platform to maximize my impact as a data scientist? [closed]

I am looking to pick up the knowledge/software skills to move towards becoming an end to end deep learning engineer. By this I mean handling the following on my own: preprocess big data at low ...
Ollie's user avatar
  • 113
1 vote
1 answer
91 views

Cost Efficient Machine Learning Development Using Cloud GPUs

Let's say I have a Juypter Notebook I am working on where I am analyzing, visualizing, testing, etc. various Machine Learning Models with different hyperparameters on some arbitrary data set or I am ...
Phoenix's user avatar
  • 51
1 vote
2 answers
35 views

Deep learning on cloud

I am trying to implement some deep learning models with large amount of data around 10gigabyte. Although, my Laptop and Collab-free crashes when it tries to load them. Do you think it worths to buy ...
mlds1337's user avatar
1 vote
1 answer
156 views

Best way to get data into Colab Notebook

I am working concurrently with multiple very large datasets (10s-100s of GBs)). I signed up for Colab Pro+ thinking it is the best option. However, I face a significant bottleneck in getting data into ...
DankMasterDan's user avatar
1 vote
1 answer
512 views

Keras multi GPU in vast.ai

I am trying to run a keras model on vast.ai using multiple GPUs. For that I am using keras.utils.multi_gpu_model , however I keep having this error: ...
Joaquin Rives's user avatar
1 vote
0 answers
36 views

What VM should I use? [closed]

So I am starting a project that has some code in it that has to do with machine learning (for example recommender system). At the moment I am thinking that if I want to deploy this application, should ...
v3tra's user avatar
  • 11
1 vote
0 answers
19 views

Moving paper documents to cloud (AWS) - [closed]

I'm trying to solve a simple problem and seeking advise/suggestions - I have lot of paper documents in my drawers which I want to store on cloud (S3) as PDFs. My idea blueprint is - 1) scanning all ...
ranit.b's user avatar
  • 451
1 vote
0 answers
27 views

What is the easiest way to duplicate my model and run in 10 cloud machines? [closed]

I have built a model in a cloud machine(google cloud). It runs for a few hours to a day. I need to scan some parameters like learning rate and batch size. How do I duplicate my compute engine, run ...
user3226167's user avatar
1 vote
0 answers
19 views

How to Read Photos for Deep Learning on Google Cloud Platform Compute Engine VM Instance, Through Jupyter Lab

I've set up a project on GCP with a Compute Engine VM and Storage Bucket. Access Scopes set to Allow full access to all Cloud APIs Have set a default Region and Zone I believe I have completed SSH ...
Matan's user avatar
  • 81
1 vote
1 answer
2k views

how to run my python code on google cloud without fear of getting disconnected - an absolute beginner?

I have been trying to use python 3 for text mining on a 650 MB csv file, which my computer was not powerful enough to do. My second solution was to reach out to google cloud. I have set up my VMs and ...
Pie-ton's user avatar
  • 113
1 vote
0 answers
213 views

Is there a way to use Orange on a remote server or in the cloud? [closed]

My question there is a way to run the Orange, not on a local computer, but on a remote server, even in the cloud? My notebook has only 8GB memory and limited number of CPU core, which is insufficient ...
obalasz's user avatar
  • 11
1 vote
0 answers
31 views

Cloud computing with country-specific region for Switzerland

Are there any cloud computing services that allow for processing and storing data exclusively in Switzerland? Are there any that have machine-learning-specific functionality? Most do not seem to have ...
Bobby's user avatar
  • 114
0 votes
1 answer
37 views

Amazon Cloud Image istance most suited for R data mining

I'm new to the field of machine learning, I always used my laptop for regular statistical analysis with no performance problems. Though lately I started programming with caret and I find myself stuck ...
GGA's user avatar
  • 125
0 votes
1 answer
28 views

Which specific AWS service to use for running Benchmark Regressions on datasets far too large to run locally on my laptop [closed]

I am in the middle of a research project with a collaborator in which he has proposed a novel statistical learning processor for optimal variable selection, and I am running the 3 Benchmark Variable ...
Marlen's user avatar
  • 167
0 votes
1 answer
66 views

What is CI/CD Pipeline in DevOps? [closed]

I just graduated with a non-tech background. As a person from a non-IT Background is DevOps a good career choice? Is it a good time to enroll in devops training?
Nathan Martin's user avatar
0 votes
1 answer
202 views

How to run unmodified Python program on GPU servers with scheduled GPUs?

Say I have one server with 10 GPUs. I have a python program which detects available GPU and use all of them. I have a couple of users who will run python (Machine learning or data mining) programs and ...
Gqqnbig's user avatar
  • 103
0 votes
1 answer
51 views

How can I use my laptop to implement analysis in Python but use a different fast server for computation of analysis?

I am working in a company but I am new in the field. We have a central server which is faster than my laptop (of course). So my goal is that I still use my laptop to do python analysis for machine ...
junmouse's user avatar
0 votes
0 answers
15 views

Best cloud provider for Computer Vision in 2024. (Market relevant)

So I am currently unemployed, I am an ML engineer, Computer Vision focused. I have experience with many cloud providers but no certifications. Generally job postings will ask for experience/...
Manuel Ruiz's user avatar