Specifically you need hardware with tons of high bandwidth memory that can be accessed by tons of very parallellised compute (so GPUs, though they’re only GPUs in architecture, they’re not actually that great for graphics), where single thread speed doesn’t matter at all.

For normal workloads this hardware wouldn’t be very efficient, you don’t need usually as much memory bandwidth usually and since not every problem is infinitely parallellisable, you’d want more single thread speed too. You’d get much better performance out of regular server CPUs for most tasks.