John's Personal Website personal blog
Making my local LLM voice assistant faster and more scalable with RAG
If you read my previous blog post, you probably already know that I like my smart home open-source and very local, and that certainly includes any voice assistant I may have. If you watched the video demo, you have probably also found out that it’s… slow. Trust me, I did too. Prefix caching helps, but it feels like…
- Points
- 122
- HN comments
- 16
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningTools & Productivity2024
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Jun 15, 2024 | Making my local LLM voice assistant faster and more scalable with RAG by JohnTheNerd | 122 | 16 |
