The financial cost of running LLMs is astonishing. In response, the industry has rushed toward FinOps for AI, the practice of meticulously tracking and optimizing every dollar spent on computation.
Vitalik Buterin and Davide Crapis, the head of AI at the Ethereum Foundation, are proposing a new system to improve privacy when using large language models.