The costs might come down as hardware production chains are retooled and specialized, the hardware designs become more efficient, and economies of scale catch up.

The quality won’t drastically improve without some major paradigm shift in model training. All of the reasonably useful and accessible training data has already been collected and processed. There are probably still refinements to be made that will improve output quality incrementally, but after ingesting the entire Internet there’s nothing left to train on. They were already starting to feed LLMs their own output as training input two years ago, which just amplifies the instabilities.