Rendered at 23:49:51 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
aspenmartin 1 hours ago [-]
I really wish more people skeptical of AI capabilities would read about scaling laws -- Lilian is always so marvelous at giving a deep overview of the technical side but the whole point of this is: there are scaling laws, and they hold and continue to hold. This is such a huge basis for the predictions about AI capabilities for the past like 5 years.
FromTheFirstIn 7 minutes ago [-]
And sitting right next to the data and compute factors in every cross entropy loss equation is the entropy of the language, which is just a fixed constant. There’s such a hard cap on cross entropy loss training and I never hear it come up!