Questions tagged [gpu]

Graphics Processing Units (GPUs) within the context of Machine Learning often refer to the hardware requirements, design considerations, or level of parallelization for implementing and running various machine learning algorithms.

Filter by
Sorted by
Tagged with
0 votes
1 answer
222 views

Speed of training decrease by adding more GPUs

I am using the distributed Tensorflow with Mirror Strategy. I am training the VGG16 based on custom Estimator. However, by increasing the number of GPUs time of training is increased. As I check, the ...
1 vote
1 answer
947 views

Model Parallelism not working in Inception v3 with Keras and TensorFlow

I have been stuck with a problem like this for a while now. I have an AWS setup with 500 GB of RAM and about 7 GPUs. Now the issue is that each time I try to run my Keras with TensorFlow as back-end ...
0 votes
1 answer
314 views

GPU shows 0 utilization even when tensors and model are mounted on the gpu?

I am trying to run some PyTorch scripts on a remote GPU server. While calling the script in the ubuntu terminal i start as:...
1 vote
1 answer
53 views

"model.to('cuda:6')" becomes (nvidia-smi) GPU 4, same with any other "cuda:MY_GPU", only "cuda:0" becomes GPU 0. How do I get rid of this mapping?

Strange mapping: example In the following example, the first column is chosen in the code, second column is the one that does the work instead: 0:0 1234 MiB 1:2 1234 MiB 2:7 1234 MiB 3:5 2341 MiB 4:1 ...
0 votes
1 answer
94 views

Transformers Trainer: "RuntimeError: module must have its parameters ... on device cuda:6 (device_ids[0]) but found one of them on device: cuda:0"

I ask this since I could not fix it with the help of: Stack Overflow RuntimeError: module must have its parameters and buffers on device cuda:1 (device_ids[0]) but found one of them on device: cuda:2 ...
3 votes
2 answers
14k views

Supported GPU for Pytorch

Question Which GPUs are supported in Pytorch and where is the information located? Background Almost all articles of Pytorch + GPU are about NVIDIA. Is NVIDIA the only GPU that can be used by Pytorch? ...
1 vote
1 answer
77 views

Seeking guidance on understanding graphics card parameters for deep learning training

I am currently in the process of purchasing a new Nvidia graphics card for training deep learning models, and I have a few questions regarding the parameters involved and their relationship to the ...
0 votes
1 answer
205 views

How many video streams can single GPU handle for object detection

I need to detect objects from multiple video streams at realtime (or close to it, like 10 FPS). How many GPUs do I need to detect objects using YOLOv3 or MobileNet for, say, 10 video streams? Is it ...
2 votes
1 answer
2k views

Is GEMM used in Tensorflow, Theano, Pytorch

I know that Caffe uses GEneral Matrix to Matrix Multiplication (GEMM) which is part of Basic Linear Algebra Subprograms (BLAS) library for performing convolution operations. Where a convolution is ...
2 votes
3 answers
20k views

How do I install CUDA GPU for Visual Studio 2022 for windows 10?

I cannot find the visual studio 2019 version and every time I try to install CUDA 11.2.2 on my laptop, It warns me about not that I haven't installed Visual Studio. I've tried installing the C++ add-...
1 vote
1 answer
84 views

Coding a Content Addressable Memory on a GPU

I´m trying to code a CAM or more simply a dictionary storing the pointer of the data accessible by a key. I try to do it with a GPU but all attempts have been inefficient compared on using System....
1 vote
1 answer
406 views

Tensorflow MirroredStrategy() looks like it may only be working on one GPU?

I finally got a computer with 2 gpus, and tested out https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html and https://github.com/tensorflow/models/tree/master/tutorials/image/...
0 votes
1 answer
4k views

cuDNN isn't found FWD algo for convolution. How to TRAIN DARKNET ON GE FORCE GTX 1650

ISSUE: while training Darknet with GE FORCE GTX 1650 using following: CUDA 11.0 cuDNN 8.0.5 OPENCV 4.5 Model starts training with config file details as below for [net] section: ...
1 vote
1 answer
544 views

How to evenly distribute data to multiple GPUs using Keras

I am using Keras=2.3.1 with Tensorflow-gpu=2.0.0 backend. While I trained model on two RTX 2080 ti 11G GPUs, it allocates all data to '/gpu:0',and nothing changed with '/gpu:1'. Surely, the second GPU ...
0 votes
1 answer
48 views

Holding batch size constant, will a bigger dataset consume more GPU memory?

If you hold (mini) batch size constant (as well as everything else) but increase the number of examples (and therefore the number of training iterations), should you expect a (significant) increase in ...
1 vote
1 answer
505 views

Keras multi GPU in vast.ai

I am trying to run a keras model on vast.ai using multiple GPUs. For that I am using keras.utils.multi_gpu_model , however I keep having this error: ...
1 vote
2 answers
881 views

Using GPUs of other machines in the network with Keras

My situation is as follows: I have a rather cheap laptop with Ubuntu 18.04 running on it that unfortunately is not powerful enough (old, cheap GPU) to train deep learning models with. I am located in ...
0 votes
1 answer
63 views

How to run our python scripts utilizing our device's GPU?

My laptop has NVIDIA GeForce GTX1650 GPU. I want to utilize this GPU to run my Python script. Any help in the form of code would be really helpful. I mean tried researching this so much but I couldn't ...
6 votes
1 answer
6k views

What size language model can you train on a GPU with x GB of memory?

I'm trying to figure out what size language model I will be able to train on a GPU with a certain amount of memory. Let's for simplicity say that 1 GB = 109 bytes; that means that, for example, on a ...
1 vote
0 answers
76 views

Using gpu accelerated libSVM in python

I have been using libSVM in python notebook to classify my dataset and it takes approximately 5 hours for one run and for 5 fold cross validation, it will take almost a day+ time. I am planning to ...
0 votes
0 answers
158 views

LLM GPU Scalability for multiple inferences

New to LLMs and have a question on scalability. Supposing I take a pre-trained open-source LLM and only wish to perform inference (eg. a simple chatbot on a local machine). If it takes me 2 GPUs to ...
40 votes
8 answers
75k views

Using TensorFlow with Intel GPU

Is there any way now to use TensorFlow with Intel GPUs? If yes, please point me in the right direction. If not, please let me know which framework, if any, (Keras, Theano, etc) can I use for my Intel ...
3 votes
2 answers
99 views

Alternatives to GCP / AWS / Azure

Can anyone recommend an alternative to the big 3 cloud computing alternatives? I know they're the best but I found them overly complicated because they cater to massive enterprises. The amount of set ...
1 vote
1 answer
88 views

Cost Efficient Machine Learning Development Using Cloud GPUs

Let's say I have a Juypter Notebook I am working on where I am analyzing, visualizing, testing, etc. various Machine Learning Models with different hyperparameters on some arbitrary data set or I am ...
0 votes
2 answers
219 views

OOM memory after kernel restart, was working before

Ran my CNN on a SageMaker notebook and it started training, but I had to restart the kernel due to AWS disconnecting. However when I tried to rerun my code, I received an OOM error, and it never ...
13 votes
1 answer
42k views

How to make my Neural Netwok run on GPU instead of CPU

I have installed Anaconda3 and have installed latest versions of Keras and Tensorflow. Running this command : ...
2 votes
1 answer
6k views

Load an LLM in multiple GPUs

I am doing a POC on LLM text generation. I have one AWS p3.8x instance which has 4 GPUs each of 16 GB size. I am pretty new to use LLM and GPU. When I am trying load one LLM pertained model (WizardLM) ...
0 votes
0 answers
35 views

Pytorch fails to detect workstation installed nvidia GPUs

We are trying to execute a deep learning model on a Linux workstation that from all counts has 2 NVIDIA GPUs installed. The model runs fine on our HPC cluster, but when we try to run it locally on the ...
0 votes
2 answers
3k views

Tensorflow gpu not available for jupyter notebook

I am trying to make tensorflow work with gpu support. First I have tensorflow-gpu installed: However, there is no gpu available....
0 votes
1 answer
441 views

Dataloader is slow with mps - PyTorch

For some reason, when using mps the dataloader is much slower (to a point in which its better to use cpu). Any idea why? Code for reproduction: ...
0 votes
0 answers
123 views

How to do batch inference in pytorch on a local GPU?

I'm working on a version of SAM model which predicts the segmentation map using text prompts provided by the user. As of now, running a single image takes about 5 seconds. I want to run about 100 ...
0 votes
0 answers
14 views

optimization of multi-regression CNN architecture

I use T4 GPU on Google Colab to train a model for multi regression. I use to training 30k RGB images 256x256, 5k to validate, 7k to evaluate. The model has 11 outputs [0-1] range (the sum of outputs <...
2 votes
1 answer
6k views

How can I monitor the usage of NVLink connections?

If I'm running a Keras model on some Nvidia GPUs which are connected via NVLink, how can I monitor the usage of the NVLink connections? I want to be sure the NVLink connections are being used and see ...
2 votes
2 answers
509 views

Which desktop hardware is best for DL?

I will be building my home Deep Learning workstation. Right now, I'm digging for some time about the best HW to use for home conditions. The workstation will be used for my work as a developer, but I ...
1 vote
0 answers
133 views

Memory usage of a DNN model during inferencing

I am running a DNN model (YOLOv7) on GPU to detect objects in a video stream. The memory usage on my task manager shows that I am using up to 2GB of my RAM! The GPU Ram usage is almost 1 GB. I am ...
1 vote
0 answers
26 views

Reproducibility issue between GPU graphic cards for DNN Tensorflow models

I am facing reproducibility issues on my DNN (Tensorflow, Keras) models when using different GPU cards. For example, when I use two a100 cards, I would be able to reproduce my results. If I use one ...
2 votes
0 answers
16 views

Can the Apple M1's iGPU access the entire RAM as "video memory" when training with typical deep learning frameworks?

Can the Apple M1's iGPU access the entire RAM as "video memory" when training with typical deep learning frameworks (e.g., tensorflow_macos)? If not, what memory do they use as video memory?
1 vote
0 answers
93 views

How can I make my neural network learn faster?

I would like to train an LSTM-based variational autoencoder on a large dataset (37 million sentences). However, I have calculated that my training speed as of now is too slow (on Google Colab). I am ...
0 votes
0 answers
472 views

How to run list comprehensions on GPU?

Is there a way to run complex list comprehensions like the following on GPU? ...
0 votes
2 answers
447 views

mBART training "CUDA out of memory"

I want to train a network with mBART model in google colab , but I got the message of ...
0 votes
1 answer
371 views

Running Model on both GPUs and CPUs

I have access to a hpc node, of 3 GPU and maximum of 38 CPU. I have a transformer model which I run of a single GPU at the moment, I want to utilize all the GPUs and CPUs. I have seen couple of ...
1 vote
0 answers
1k views

Is RTX 2060 (12 GB) or RTX 3060 (12 GB) suitable for complex deep learning tasks? [closed]

After many months of saving, I can now afford to buy RTX 3060 (12GB). However, I am afraid to waste my money after saving them for months. I am preparing myself for starting my Ph.D. degree. It will ...
4 votes
3 answers
7k views

Alternatives with better GPU than Google Colab Pro

I am currently running/training MAchine learning models that are very GPU expensive, Google Colab Pro is not giving me enough GPU/RAM Is there any alternatives with better GPU and more RAM than ...
0 votes
0 answers
332 views

ImportError: DLL load failed while importing _pywrap_tfe when running exe file created by Pyinstaller

I have created the exe file of my code which uses tensorflow on gpu. It seems that building process using Pyinstaller goes fine. However, when I run the exe file in a system without GPU, I get the ...
0 votes
2 answers
55 views

Moving from macbook (without GPU) to linux system with Titan V, only getting a 4x speedup, what am I doing wrong?

I was prototyping a network architecture out on the macbook, and after finding something I was somewhat happy with, I wanted to test it out on a big data set on a system with a Titan V as the macbook ...
1 vote
1 answer
169 views

How does compute required scale with number of model parameters?

GPT-3 has 175 billion parameters, required ~$3.114 * 10^{23}$ FLOPS, and took approximately one month to train on ~10k Tesla V100 GPUs. It seems commonly stated that the brain has the equivalent of ~...
8 votes
3 answers
17k views

Why do I get an OOM error although my model is not that large?

I am a newbie in GPU based training and deep learning models. I am running cDCGAN (Conditional DCGAN) in TensorFlow on my 2 Nvidia GTX 1080 GPUs. My data set consists of around 320,000 images with ...
0 votes
0 answers
2k views

ValueError: Mixed precision training with AMP or APEX (`--fp16` or `--bf16`) and half precision evaluation (`--fp16) can only be used on CUDA devices

i’m fine tuning the wav2vec-xlsr model. i’ve created a virtual env for that and i’ve installed cuda 11.0 and tensorflow-gpu==2.5.0 but it gives the following error : ValueError: Mixed precision ...
1 vote
1 answer
2k views

BERT base uncased required gpu ram

I'm working on an NLP task, using BERT, and I have a little doubt about GPU memory. I already made a model (using DistilBERT) since I had out-of-memory problems with tensorflow on a RTX3090 (24gb gpu'...
1 vote
1 answer
745 views

How do NVIDIA GPU restrictions affect AI computational frameworks?

I know this question is very vendor specific and as time passes it might change but I am wondering how NVIDIA available GPU cards nowadays (2022) are restricted in any way license wise or hardware ...