There is a lot of discussion on google search about AI-custom-accelerators (like Intel's Gaudi) and GPUs.
Almost all of them say generic things like, a) AI Accelerator chip is for specialized AI processing whereas GPUs work for general AI models, and like b) if you want customize chip for specific AI workloads use AI-accelerator or else use GPU.
From what I understand GPUs are already great at large-scale dot-products done in batch-processing (throughput mode), and most of AI workloads are matmuls (which is essentially dot-product) so GPUs handle AI workloads very well.
Plus, I've also seen Intel's Gaudi being used for a "variety of AI workloads", not specialized for a single model. It can be used for general AI workloads just like GPU. So what's the difference.
What I don't understand is, "exactly" what specific features are built differently in Accelerator vs GPU. Both have ALUs and matmul engines that do very well on AI models. Both have large-cache/memory and DDR speed.
What exactly makes one better? For which AI workload would one choose accelerator over GPU?
AI accelerators have fixed-function for matmul. Do GPUs have fixed-function?
AI accelerators have software-managed cache (HBM) from what I understand. Is that the same with GPUs or is there a way cache is different between accelerators and GPUs that changes things?
I'm kind of unsure about the differences between GPUs and AI-accelerators with respect to Fixed-function and software-managed-cache.
