Skip to main content

Questions tagged [gpu]

Graphics Processing Units (GPUs) within the context of Machine Learning often refer to the hardware requirements, design considerations, or level of parallelization for implementing and running various machine learning algorithms.

Filter by
Sorted by
Tagged with
0 votes
0 answers
61 views

What are the ways to decrease the inference time for the Orca-mini v3_7b model on an Azure ML real-time endpoint? [closed]

What can be done to reduce inference time and achieve low latency for the deployed Orca-mini model in Azure ML? These are the steps that have been tried (more detailed info below): Different GPU SKUs ...
Demon's user avatar
  • 61
0 votes
0 answers
14 views

Why can't I increase my GPU utilization?

I have a simple UNet model (~1M params) written in Keras 3.0.1, running with a torch backend. My CUDA version is ...
Savindi's user avatar
0 votes
1 answer
449 views

Transformers Trainer: "RuntimeError: module must have its parameters ... on device cuda:6 (device_ids[0]) but found one of them on device: cuda:0"

I ask this since I could not fix it with the help of: Stack Overflow RuntimeError: module must have its parameters and buffers on device cuda:1 (device_ids[0]) but found one of them on device: cuda:2 ...
questionto42's user avatar
1 vote
1 answer
72 views

"model.to('cuda:6')" becomes (nvidia-smi) GPU 4, same with any other "cuda:MY_GPU", only "cuda:0" becomes GPU 0. How do I get rid of this mapping?

Strange mapping: example In the following example, the first column is chosen in the code, second column is the one that does the work instead: 0:0 1234 MiB 1:2 1234 MiB 2:7 1234 MiB 3:5 2341 MiB 4:1 ...
questionto42's user avatar
0 votes
1 answer
89 views

Holding batch size constant, will a bigger dataset consume more GPU memory?

If you hold (mini) batch size constant (as well as everything else) but increase the number of examples (and therefore the number of training iterations), should you expect a (significant) increase in ...
ubadub's user avatar
  • 101
0 votes
1 answer
149 views

How to run our python scripts utilizing our device's GPU?

My laptop has NVIDIA GeForce GTX1650 GPU. I want to utilize this GPU to run my Python script. Any help in the form of code would be really helpful. I mean tried researching this so much but I couldn't ...
Escanor6's user avatar
1 vote
0 answers
102 views

Using gpu accelerated libSVM in python

I have been using libSVM in python notebook to classify my dataset and it takes approximately 5 hours for one run and for 5 fold cross validation, it will take almost a day+ time. I am planning to ...
khushi's user avatar
  • 111
0 votes
0 answers
182 views

LLM GPU Scalability for multiple inferences

New to LLMs and have a question on scalability. Supposing I take a pre-trained open-source LLM and only wish to perform inference (eg. a simple chatbot on a local machine). If it takes me 2 GPUs to ...
David's user avatar
  • 1
1 vote
1 answer
114 views

Seeking guidance on understanding graphics card parameters for deep learning training

I am currently in the process of purchasing a new Nvidia graphics card for training deep learning models, and I have a few questions regarding the parameters involved and their relationship to the ...
ja1ba6's user avatar
  • 11
2 votes
1 answer
7k views

Load an LLM in multiple GPUs

I am doing a POC on LLM text generation. I have one AWS p3.8x instance which has 4 GPUs each of 16 GB size. I am pretty new to use LLM and GPU. When I am trying load one LLM pertained model (WizardLM) ...
Saikat Bhattacharya's user avatar
0 votes
0 answers
36 views

Pytorch fails to detect workstation installed nvidia GPUs

We are trying to execute a deep learning model on a Linux workstation that from all counts has 2 NVIDIA GPUs installed. The model runs fine on our HPC cluster, but when we try to run it locally on the ...
Ioannis K. Moutsatsos's user avatar
0 votes
1 answer
492 views

Dataloader is slow with mps - PyTorch

For some reason, when using mps the dataloader is much slower (to a point in which its better to use cpu). Any idea why? Code for reproduction: ...
Ben Lahav's user avatar
2 votes
2 answers
766 views

Which desktop hardware is best for DL?

I will be building my home Deep Learning workstation. Right now, I'm digging for some time about the best HW to use for home conditions. The workstation will be used for my work as a developer, but I ...
Jaroslav Štreit's user avatar
1 vote
0 answers
139 views

Memory usage of a DNN model during inferencing

I am running a DNN model (YOLOv7) on GPU to detect objects in a video stream. The memory usage on my task manager shows that I am using up to 2GB of my RAM! The GPU Ram usage is almost 1 GB. I am ...
Hossein's user avatar
  • 11
1 vote
0 answers
34 views

Reproducibility issue between GPU graphic cards for DNN Tensorflow models

I am facing reproducibility issues on my DNN (Tensorflow, Keras) models when using different GPU cards. For example, when I use two a100 cards, I would be able to reproduce my results. If I use one ...
Sung Joon Won's user avatar
2 votes
0 answers
19 views

Can the Apple M1's iGPU access the entire RAM as "video memory" when training with typical deep learning frameworks?

Can the Apple M1's iGPU access the entire RAM as "video memory" when training with typical deep learning frameworks (e.g., tensorflow_macos)? If not, what memory do they use as video memory?
Franck Dernoncourt's user avatar
2 votes
0 answers
100 views

How can I make my neural network learn faster?

I would like to train an LSTM-based variational autoencoder on a large dataset (37 million sentences). However, I have calculated that my training speed as of now is too slow (on Google Colab). I am ...
postnubilaphoebus's user avatar
7 votes
1 answer
9k views

What size language model can you train on a GPU with x GB of memory?

I'm trying to figure out what size language model I will be able to train on a GPU with a certain amount of memory. Let's for simplicity say that 1 GB = 109 bytes; that means that, for example, on a ...
HelloGoodbye's user avatar
1 vote
0 answers
1k views

Is RTX 2060 (12 GB) or RTX 3060 (12 GB) suitable for complex deep learning tasks? [closed]

After many months of saving, I can now afford to buy RTX 3060 (12GB). However, I am afraid to waste my money after saving them for months. I am preparing myself for starting my Ph.D. degree. It will ...
John adams's user avatar
0 votes
1 answer
541 views

Running Model on both GPUs and CPUs

I have access to a hpc node, of 3 GPU and maximum of 38 CPU. I have a transformer model which I run of a single GPU at the moment, I want to utilize all the GPUs and CPUs. I have seen couple of ...
Fhunmie's user avatar
  • 17
1 vote
1 answer
227 views

How does compute required scale with number of model parameters?

GPT-3 has 175 billion parameters, required ~$3.114 * 10^{23}$ FLOPS, and took approximately one month to train on ~10k Tesla V100 GPUs. It seems commonly stated that the brain has the equivalent of ~...
PerpetualPerception's user avatar
0 votes
2 answers
4k views

Tensorflow gpu not available for jupyter notebook

I am trying to make tensorflow work with gpu support. First I have tensorflow-gpu installed: However, there is no gpu available....
ptushev's user avatar
  • 11
0 votes
0 answers
2k views

ValueError: Mixed precision training with AMP or APEX (`--fp16` or `--bf16`) and half precision evaluation (`--fp16) can only be used on CUDA devices

i’m fine tuning the wav2vec-xlsr model. i’ve created a virtual env for that and i’ve installed cuda 11.0 and tensorflow-gpu==2.5.0 but it gives the following error : ValueError: Mixed precision ...
ali hayen's user avatar
1 vote
1 answer
2k views

BERT base uncased required gpu ram

I'm working on an NLP task, using BERT, and I have a little doubt about GPU memory. I already made a model (using DistilBERT) since I had out-of-memory problems with tensorflow on a RTX3090 (24gb gpu'...
Gius's user avatar
  • 78
2 votes
1 answer
908 views

How do NVIDIA GPU restrictions affect AI computational frameworks?

I know this question is very vendor specific and as time passes it might change but I am wondering how NVIDIA available GPU cards nowadays (2022) are restricted in any way license wise or hardware ...
Thomas's user avatar
  • 129
0 votes
0 answers
73 views

How to build a neural network without using keras compile method

I have the following neural network: ...
Sharhad Bashar's user avatar
0 votes
1 answer
4k views

cuDNN isn't found FWD algo for convolution. How to TRAIN DARKNET ON GE FORCE GTX 1650

ISSUE: while training Darknet with GE FORCE GTX 1650 using following: CUDA 11.0 cuDNN 8.0.5 OPENCV 4.5 Model starts training with config file details as below for [net] section: ...
TDI-India's user avatar
1 vote
1 answer
91 views

Cost Efficient Machine Learning Development Using Cloud GPUs

Let's say I have a Juypter Notebook I am working on where I am analyzing, visualizing, testing, etc. various Machine Learning Models with different hyperparameters on some arbitrary data set or I am ...
Phoenix's user avatar
  • 51
1 vote
0 answers
31 views

Is there a reason not to wirk with AMP (automatic mixed precision)?

According to: Introducing native PyTorch automatic mixed precision for faster training on NVIDIA GPUs It's better to use AMP (with 16 Floating Point) due to: Shorter training time. Lower memory ...
user3668129's user avatar
0 votes
2 answers
458 views

mBART training "CUDA out of memory"

I want to train a network with mBART model in google colab , but I got the message of ...
AFB's user avatar
  • 41
1 vote
0 answers
531 views

Intel Core i7 CPU (10th gen) trains Tensorflow models faster than Intel Iris Plus Graphics GPU

Intro Recently I was able to get my GPU (Intel Iris Plus Graphics) to work with "accelerating" Tensorflow. It took many hours of research and following tutorials (most of which didn't help). ...
Jacob Hornbeck's user avatar
2 votes
3 answers
22k views

How do I install CUDA GPU for Visual Studio 2022 for windows 10?

I cannot find the visual studio 2019 version and every time I try to install CUDA 11.2.2 on my laptop, It warns me about not that I haven't installed Visual Studio. I've tried installing the C++ add-...
rutvik jere's user avatar
1 vote
1 answer
742 views

Does GPU decreases training time for on-policy RL?

I was wondering whether using a GPU will be effective if I am using an on-policy (eg PPO) RL as the model? I.e, how can we use a GPU to decrease training time for an on-policy RL model? I recently ...
Wenuka's user avatar
  • 95
0 votes
2 answers
229 views

OOM memory after kernel restart, was working before

Ran my CNN on a SageMaker notebook and it started training, but I had to restart the kernel due to AWS disconnecting. However when I tried to rerun my code, I received an OOM error, and it never ...
Finn Williams's user avatar
0 votes
0 answers
2k views

ValueError: GPU is not accessible. Was the library installed correctly?

I installed spacy 3 in a venv and tried to execute: spacy.require_gpu() Then I got this as output: ...
SRJ577's user avatar
  • 197
0 votes
2 answers
2k views

Model accuracy when training on GPU and then inferencing on CPU

When we are concerned about speed, GPU is way better than CPU. But if I train a model on a GPU and then deploy the same trained model (no quantization techniques used) on a CPU, will this affect the ...
Devashish Prasad's user avatar
5 votes
3 answers
10k views

Alternatives with better GPU than Google Colab Pro

I am currently running/training MAchine learning models that are very GPU expensive, Google Colab Pro is not giving me enough GPU/RAM Is there any alternatives with better GPU and more RAM than ...
The Dan's user avatar
  • 193
1 vote
0 answers
85 views

Suitable CPU with RTX3070 GPU for deep learning [closed]

I am trying to get a GPU for my deep learning research. Due to stock limitation, I can only get a RTX3070. In my lab, there's some PC/workstation whereby I can install the RTX3070. However, I wonder ...
quarkz's user avatar
  • 11
0 votes
1 answer
364 views

GPU shows 0 utilization even when tensors and model are mounted on the gpu?

I am trying to run some PyTorch scripts on a remote GPU server. While calling the script in the ubuntu terminal i start as:...
Barah Fazili's user avatar
3 votes
1 answer
2k views

Why the my Tensorflow code just use one GPU when I assign more than one

I am trying to run my code on a supercomputer with 8 gpus. Though, I assign 8 gpus but one of them is just occupied. I read some notes in websites and it seems that Tensorflow automatically use gpu if ...
javad hamidi's user avatar
0 votes
0 answers
354 views

ImportError: DLL load failed while importing _pywrap_tfe when running exe file created by Pyinstaller

I have created the exe file of my code which uses tensorflow on gpu. It seems that building process using Pyinstaller goes fine. However, when I run the exe file in a system without GPU, I get the ...
Carol's user avatar
  • 1
1 vote
1 answer
115 views

DIGITS Docker container not picking up GPU

I am running DIGITS Docker container but for some reason it fails to recognize host's GPU: it does not report any GPUs (where I expect 1 to be reported) so in the upper right corner of the DIGITS home ...
Bojan Komazec's user avatar
1 vote
1 answer
237 views

How many video streams can single GPU handle for object detection

I need to detect objects from multiple video streams at realtime (or close to it, like 10 FPS). How many GPUs do I need to detect objects using YOLOv3 or MobileNet for, say, 10 video streams? Is it ...
viator's user avatar
  • 111
0 votes
1 answer
203 views

How to run unmodified Python program on GPU servers with scheduled GPUs?

Say I have one server with 10 GPUs. I have a python program which detects available GPU and use all of them. I have a couple of users who will run python (Machine learning or data mining) programs and ...
Gqqnbig's user avatar
  • 103
3 votes
2 answers
102 views

Alternatives to GCP / AWS / Azure

Can anyone recommend an alternative to the big 3 cloud computing alternatives? I know they're the best but I found them overly complicated because they cater to massive enterprises. The amount of set ...
lara_toff's user avatar
  • 221
0 votes
1 answer
198 views

Why does my GPU immediately run out of memory when I try to run this code?

I am trying to write a neural network that will train on plays by Shakespeare and then write its own passages. I am using pytorch. For some reason, my GPU ...
Christoffer Corfield Aakre's user avatar
1 vote
1 answer
784 views

Why does GPU speed up inference?

I understand that GPU can speed up training for each batch multiple data records can be fed to the network which can be parallelized for computation. However, for inference, typically, each time the ...
marlon's user avatar
  • 113
2 votes
1 answer
329 views

Use a GPU to speed up neural net training in R

I'm currently training a neural net model in R and am wanting to use a GPU to speed up this process. I've looked into this but it appears that this is unavailable to Mac users as Apple no longer uses ...
Blake Lucey's user avatar
-1 votes
1 answer
36 views

How much GPUs are needed for Image ehancement? [closed]

I'm looking for a GPU to train my model. Most of the papers that I have followed used 2 or more gtx 1050ti card or higher. (MIRNet, EnlightenGAN) I need to that how much GPU power will it take to ...
Asel's user avatar
  • 99
4 votes
1 answer
8k views

CUDA compatibility of GTX 1650ti versus 1650

I am confused about CUDA compatibility. I am studying deep learning and looking for a laptop to buy. One laptop has GTX 1650ti and another has GTX 1650. Will both be able to use GPU for model training,...
user105772's user avatar