Janas: Run LLMs larger than RAM by streaming experts from SSD

🌐 Hacker News (United States) —
Janas: Run LLMs larger than RAM by streaming experts from SSD

AI Summary

Janas is a software project designed to run large language models that exceed available RAM by streaming model experts from an SSD. The item was shared on GitHub and discussed on Hacker News.

Article URL: https://github.com/prabanta-dev/janas Comments URL: https://news.ycombinator.com/item?id=50063717 Points: 11 # Comments: 2

AI & Tech Janas large language models LLM inference SSD streaming RAM open-source software

Read original source →