Marek Rogala

Another huge release on the long-context LLMs front: Introducing Giraffe 70B, an open-source LLM fine-tuned from base Llama 2 with context length extended to 32K. The @abacusai team conducted an evaluation of the model assessing performance over long contexts. The 70B model improves significantly at the longest context windows (32k) for the document QA task vs. the 13B model, scoring 61% accuracy vs. the 18% accuracy of 13B on the AltQA dataset. I got excited with the earlier release of Giraffe because of the potential of these long context models to unlock unique applications. Even more exciting to see that the 70B Giraffe model shows the best performance of all the open-source models in key categories like extraction, coding, and math. Note that there is still a gap in performance to the best closed source models like GPT-4 but it's coming closer. I also wonder how these models could help with key capabilities like information extraction, large-form doc analysis, and reasoning. There is a lot to explore with these long-context models. Don't sleep on these. Great work by the Abacus team. Blog: blog.abacus.ai/blog/2023/09/… Model: huggingface.co/abacusai/Gira…

Open-source ML is at it again! Introducing ColossalChat - an open-source solution for cloning ChatGPT with a complete RLHF pipeline. Here is what you need to know: