Cost Per DataPoint to Train Large Language Models
https://www.fuzzyflo.com/posts/training-cost-per-datapoint
Cost Per DataPoint to Train Large Language Models
https://www.fuzzyflo.com/posts/training-cost-per-datapoint
3 Comments
* The trend line suggests that costs are increasing from 2018, but I believe that there has been an AI revolution since 2020/2021 and thus the trend should be calulated on that line
* This data is from Epoch AI, an AI research firm, and these figures are estimates as the true cost of training models are often hidden
* I believe that AI spending will increase in the coming years but AI spending per datapoint will decrease in the coming years, possibly due to synthetic data
2024 is going to half 2023 or more as well. I guarantee it. What does that do to this graph?
The ghost of Dr. Zamenhof would never let me rest if I didn’t inquire as to the cost of Esperanto…