argilla/ notus

💨 Notus 7B is an open source LLM released by Argilla, fine-tuned using Direct Preference Optimization (DPO) and AIF (AI Feedback) techniques. This model is fine-tuned with a better curated version of the Ultrafeedback dataset.

7B

67 Pulls Updated 5 months ago

13 Tags

5 months ago

4008f13b1b66 · 4.4GB

model
llama
·
7.2B
·
Q8_K
params
{"num_ctx":8192,"stop":["<|system|>","<|user|>","<|assistant|>","</s>"]}
template
"{{- if .System }}<|system|>\n{{ .System }}</s>\n{{- end }}\n<|user|>\n{{ .Prompt }}</s>\n<|assistant|>\n"

Readme

💨 Notus

A banner representing Notus, the wind god of the south, in a mythical and artistic style. The banner features a strong, swirling breeze, embodying the warm, wet character of the southern wind. Gracefully flowing across the scene are several paper planes, caught in the gentle yet powerful gusts of Notus. The background is a blend of warm colors, symbolizing the heat of the south, with hints of blue and green to represent the moisture carried by this wind. The overall atmosphere is one of dynamic movement and warmth.

Summary

Performance

Table adapted from Zephyr-7b-β and Starling’s original tables for MT-Bench and AlpacaEval benchmarks. Results are shown sorted by AlpacaEval win rates and ommit some >7B for brevity.

Notus stays on par with Zephyr on MT-Bench, while surpassing Zephyr and Claude 2 on AlpacaEval. Making Notus the most-competitive 7B commercial model on AlpacaEval.

Model Size Alignment MT-Bench (score) AlpacaEval (win rate %) License
GPT-4-turbo - ? 9.32 97.70 Proprietary
XwinLM 70b V0.1 70B dPPO - 95.57 LLaMA 2 License
GPT-4 - RLHF 8.99 95.03 Proprietary
Tulu 2+DPO 70B V0.1 70B dDPO 6.29 95.28 Proprietary
LLaMA2 Chat 70B 70B RLHF 6.86 92.66 LLaMA 2 License
Starling-7B 7B C-RLFT + APA 8.09 91.99 CC-BY-NC-4.0
Notus-7b-v1 7B dDPO 7.30 91.42 MIT
Claude 2 - RLHF 8.06 91.36 Proprietary
Zephyr-7b-β 7B dDPO 7.34 90.60 MIT
Cohere Command - RLHF - 90.62 Proprietary
GPT-3.5-turbo - RLHF 7.94 89.37 Proprietary

Usage

CLI

ollama run argilla/notus

API

Example:

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "notus",
  "prompt":"Here is a story about llamas eating grass"
}'

Context

You can find the entire process of the creation of Notus in our blogpost.

Notus is a collection of fine-tuned models using SFT, DPO, SFT+DPO, and/or any other RLAIF/RLHF techniques; following a data-first, human-centric approach, since that’s what we do best at Argilla.

Notus models are intended to be used as assistants via chat-like applications, and are evaluated with Chat (MT-Bench, AlpacaEval) and Academic (Open LLM Leaderboard) benchmarks for a direct comparison with other similar LLMs.

Notus name comes from the ancient Greek god Notus, as a wink to Zephyr, which comes from the ancient Greek god Zephyrus; with the difference that Notus is the god of the south wind, and Zephyr the god of the west wind. More information at https://en.wikipedia.org/wiki/Anemoi.

Being able to fine-tune LLMs while still keeping a data-first approach wouldn’t have been possible without the inestimable help of the open source community and all the amazing resources out there intended for the general public. We are very grateful for that, and we hope that our work can be useful for others as well.

🎩 h/t HuggingFace H4 team for their amazing work with alignment-handbook, and also for the fruitful discussions we had with them and their support.

News

  • December 1st, 2023: Notus 7B v1 is released! 🎉 Using the same DPO fine-tuning approach as Zephyr 7B Beta, but changing the data source from UltraFeedback to binarize it using the average of the different criterias, instead of the critique score. Notus 7B improved in both AlpacaEval and LM Eval Harness compared to Zephyr 7B Beta, while for MT-Bench the results were on par. More information at v1/.

Resources

🤗 HuggingFace Hub Collection

image

Available at: Hugging Face

💬 Chat UI

image

Chat with Notus at Hugging Face Spaces (powered by Hugging Face Chat UI)