Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [transformer-model]

The tag has no usage guidance.

2 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
0 votes
0 answers
54 views

What are the ways to decrease the inference time for the Orca-mini v3_7b model on an Azure ML real-time endpoint?

What can be done to reduce inference time and achieve low latency for the deployed Orca-mini model in Azure ML? These are the steps that have been tried (more detailed info below): Different GPU SKUs ...
Demon's user avatar
  • 61
0 votes
1 answer
14 views

Understanding batching in pytorch models

I have following model which forms one of the step in my overall model pipeline: ...
Mahesha999's user avatar