Facebook

Large language models require orders of magnitude more words than humans: https://twitter.com/mcxfrank/status/1640379247373197313

But there are important caveats:

1. Humans possess innate knowledge in the form of an evolutionary prior that was fine-tuned over billions of years.
2. Humans benefit from transfer learning from multiple modalities.
3. Humans are grounded by and receive feedback through embodiment.
4. Human children can talk to adults. With access to “adult” language models, it gets vastly easier to train new models. By training a new model on the output of an existing model, performance similar to GPT-3 can be achieved at a total cost of $600: https://crfm.stanford.edu/2023/03/13/alpaca.html

Source : Facebook