Questions tagged [memory]

The tag has no usage guidance.

9 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2
votes
0answers
59 views

Running out of memory when calculating loss using SigmoidFocalCrossEntropy

Code Versions: python == 3.8 tensorflow == 2.2.0 tensorflow-addons == 0.11.2 Recently I've been using tensorflow addon's focal loss function for one of my models. ...
1
vote
1answer
63 views

Why is HistGradientBoostingRegressor in sklearn so fast and low on memory?

I trained multiple models for my problem and most ensemble algorithms resulted in lengthy fit and train time and huge model size on disk (approx 10GB for RandomForest) but when I tried ...
0
votes
0answers
8 views

How can I measure time complexity for a model?

How can I measure or find the time complexity for a model and memory complexity like VGG16 or Resnet50 also and does it will be different from one machine to another like using GTX or RTX?
0
votes
0answers
6 views

Does using torch.cuda.empty_cache can decrease performance?

We're struggling with memory usage in a project that deploys multiple models to the same GPU (models are usually built with PyTorch and TensorFlow). It was suggested that we could use ...
0
votes
1answer
20 views

determining size of batch, time of sending and memory in to send from scala to ML section

I have a time series (sampling time: 66.66 micro second, number of samples/sampling time=151), I would like to determine some anomalies in them, the inputs are made by scala customer message bus. ...
0
votes
0answers
11 views

Reducing dataset before computing similarity matrix

I'm writing my thesis and am trying to calculate a similarity matrix of houses. I currently have a dataset of 500,000 houses that I need to calculate the similarity between. I.e. I need to calculate ...
0
votes
0answers
53 views

Which memory usage is more accurate in nvidia-smi?

My nvidia-smi shows the following: ...
0
votes
0answers
32 views

Best practices for memory management in image processing

I'm relatively new to data science, and I'm trying to create a CNN model for the Kaggle melanoma competition. I've created the following two functions to transform a folder of JPEG images to a Numpy ...
0
votes
0answers
16 views

Can the Apple M1's iGPU access the entire RAM as "video memory" when training with typical deep learning frameworks?

Can the Apple M1's iGPU access the entire RAM as "video memory" when training with typical deep learning frameworks (e.g., tensorflow_macos)? If not, what memory do they use as video memory?