How I run LLMs locally
I document how I run Large Language Models locally.
The latest blog posts to take off on Hacker News. 3,433 posts.
I document how I run Large Language Models locally.
I've often found myself uneasy when LLMs (Large Language Models) are asked to quote the Bible. While they can provide insightful discussions about faith, their tendency to hallucinate responses raises concerns when dealing with scripture, which we regard as the inspired Word of God. To explore these concerns, I
I will use this place to relate my geeky adventures in the world of gnu/linux, bsd, the internets, and other important things like windows managers and pretty colors. Basically, it's a place to expose and share with the world things I do on my computer.
How I built an ML rig for training LLMs locally, exploring hardware choices, setup tricks, and lessons learned along the way.
While looking for ways to activate the developer menus left over in Animal Crossing, including the NES emulator game selection menu, I found an interesting feature that exists in the original game that was always active, but never used by Nintendo. In addition to the NES/Famicom games that can be obtained in-game, it…
In 1993, Intel released the high-performance Pentium processor, the start of the long-running Pentium line. The Pentium had many improvement...
My long-time collaborator Richard Barnes [1] used to say that "in security, trust is a four letter word" , and yet the dominant experience of using any software-based system—which is, you know, pretty much anything electronic—is trusting the manufacturer. Not only is there no meaningful way to determine what software…
This morning while perusing the settings of a bunch of apps on my iPhone, I discovered a new setting for Photos that was enabled by default: Enhanced Visual Search. (I manually disabled it before taking the screenshot below.)
Google search results are becoming annoying to use and Open AI is about to eat Google's lunch.
A developer automated job applications, sending 250 in 20 minutes using Python scripts and GPT
The videos have been uploaded , thank-you notes have been sent, and the kiddo has decided to play a little Minecraft on her own, so now I get to write some quick notes on EmacsConf 2024 .
I have been doing Java in Neovim for quite a while at work, and it’s been a very pleasant experience. As Neovim usage grows (especially amongst the younger crowd), I want to share how I do it.
I recently picked up an embedded project in which I needed to build a highly customized full system image with minimal boot times and NetBSD was the best choice for it. Let's look at why its build system helps.
Mazes are typically constructed on regular grids: square, triangular, hexagonal, etc. But what if you generate a maze on top of a Penrose tiling, which is guaranteed to be irregular? (It is an aperiodic tiling .) The resulting mazes have a very characteristic appearance: they are filled with circles and stars, but the…
Tae Kim's terrific new book on the story of Nvidia and Jensen Huang
So I like to joke occasionally on the Fediverse that when looking at Wikipedia's list of programming languages supporting dependent types :
Learn How to Create HTML/ZIP/PNG Polyglot Files!
I built a Factorio belt balancer solver. From tiny 2x2 grids to a massive 16x16 monster, every step revealed new solver tricks, numeric hiccups, and devious debugging puzzles.
In 2023, we launched an AI-powered Chief of Staff for engineering leaders—an assistant that unified information across team tools and tracked critical project developments. Within a year, we attrac...
I am a simple enthusiast attempting to convey my love and enjoyment of radio, electronics, and computing and perhaps to inspire YOU to build something too.
In the context of out-of-order microarchitectures, I was under the impression that execution units remain occupied until the µop they’re processing is complete. This is often not the case.
An implemenation of the SuperMemo-2 algorithm.
Link to: https://www.codingfont.com/
No clustering algorithm is perfect and you must make a trade-off.
Sniffing USB HID traffic from the Stream Deck Plus to map its dial encoder and touchscreen events, building on the original Stream Deck reverse engineering.
In this tutorial, I will teach you how to write your own virtual machine (VM) that can run assembly language programs, such as my friend’s 2048 or my Roguelike . If you know how to program, but would like to gain a deeper understanding of what is going on inside a computer and better understand how programming…
Introduction The more I learn about Lua's design and implementation, the more impressed I am. It's very rare to see software that does so much with so lit...
ts_sms works similarly to ts_zip . It uses a specific padding system compatible with arithmetic coding so that the message length does not need to be explicitly encoded.
The Ivory Tower is a blog about software engineering and development philosophy by Anders Sundman.