Curated Resource ( ? )

Llama 2: an incredible open LLM

Llama 2: an incredible open LLM

my notes ( ? )

Summarises a recent Meta paper on Llama 2, "a continuation of the LLaMA... Big picture, this is a big step for the LLM ecosystem when research sharing is at an all-time low and regulatory capture at an all-time high" - so Meta continues its improbable position as good guy in the OS movement (at least when it comes to AI, but also possible in social, where they've committed to supporting ActivityPub in a future Threads upgrade), although "Technically, ... not open-source ... still useful for the open-source community, but it is only an open release / open innovation".

"base model seems ... beyond GPT3... fine-tuned chat models seem on the same level as ChatGPT ... (except in coding)". The author then provides "here's what you need to know", but it gets very technical very fast, except perhaps:

  • "model is available for commercial use unless your product has >= 700 million monthly active users"
  • Meta's political position may be why the paper "feels like an incredible double-down on the original LLaMA formula... leaning heavily into ... trust, accountability, and democratization¬†of AI via Open-Source...
  • make it exceedingly clear that they do not use user data, avoiding the Bard blunder ...
  • ... agrees with rumors ... from Anthropic and OpenAI friends: the reward model is the key of RLHF, which is also key to models... reward model accuracy is one of the most important proxies for the final performance of Llama 2-Chat...
  • two reward models trained to separate goals of helpful and safe...
  • [using] automatic benchmarks... LLAMA 2 is way better than any other open source model at all scales ...
  • safety aspects ... by far and away the biggest improvement over available open-source models ...

Read the Full Post

The above notes were curated from the full post www.interconnects.ai/p/llama-2-from-meta.

Related reading

More Stuff I Like

More Stuff tagged metrics , open source , llama , meta , llama2

Cookies disclaimer

MyHub.ai saves very few cookies onto your device: we need some to monitor site traffic using Google Analytics, while another protects you from a cross-site request forgeries. Nevertheless, you can disable the usage of cookies by changing the settings of your browser. By browsing our website without changing the browser settings, you grant us permission to store that information on your device. More details in our Privacy Policy.