Add OLMo November 2024 model

[2015aroras]

This PR implements the architecture for the soon-to-be-released OLMo November 2024 model. Tested with a model checkpoint that is through a large portion of pretraining. Fixes #10316.

Architecture differences from the original OLMo model:

• RMSNorm is used instead of standard layer norm.
• Norm is applied to attention queries and keys.
• Norm is applied after attention/feedforward rather than before.

Test (base model):

// Download OLMoE weights to models/olmo-nov-hf
python3 convert_hf_to_gguf.py models/olmo-nov-hf
./llama-quantize models/olmo-nov-hf/OLMo-7B-1124-hf-7.3B-F16.gguf models/olmo-hf/OLMo-7B-1124-hf-7.3B-Q4_K_M.gguf Q4_K_M
./llama-cli -m models/olmo-nov-hf/OLMo-7B-1124-hf-7.3B-Q4_K_M.gguf -p "Bitcoin is" -n 128

Output:

Bitcoin is not only a currency, but also a technology and a movement. It has been heralded as the currency of the future, but it has also been called a bubble. Despite its volatility, it is an asset that is highly sought after, and it is a technology that has been praised by many. But what is it that makes Bitcoin so special?

Bitcoin is a decentralized digital currency that was created in 2009. It is not issued by any government or central bank, and it is not backed by any physical asset. Bitcoin is a virtual currency that can be used to purchase goods and services, and it can also be exchanged for traditional

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[2015aroras]

This class is a lot simpler than OlmoModel because that model is using the wrong rope type and the OlmoModel compensates for it using modify_tensors.