Andrej Karpathy – It will take a decade to work through the issues with agents
"The problems are tractable, but they're still difficult”
Topics /
The most popular blog posts about ai & machine learning on Hacker News. 3,090 posts.
"The problems are tractable, but they're still difficult”
Unless you have been living under a rock for the last few years, you probably have seen the same massive surge I’ve seen in the application of artificial intelligence (AI) to pretty much every problem out there, in software testing, in software development, and in life in general.
offensive security & software engineering
I'm a software engineer, completing 10 years of professional experience this year. I started my career as a web frontend engineer (it was easier for me to de...
Not All Heroes Wear Capes
thoughts, talks, docs and unpopular opinions
Last week Google introduced Gemini Pro 1.5, an enormous upgrade to their Gemini series of AI models. Gemini Pro 1.5 has a 1,000,000 token context size. This is huge—previously that …
I'm about as pro-AI as you can be, but this is becoming a pet peeve of mine (and I'm not alone). That's why I love the AI;DR acronym as my new solution for ignoring the walls of slop.
A complete GPT2 implementation as a single SQL query in PostgreSQL.
Anthropic are strongly rumored to be about to have their first profitable quarter. Stories are circulating of companies surprised at how expensive their LLM bills are becoming from usage by …
Stephen Wolfram explores the broader picture of what's going on inside ChatGPT and why it produces meaningful text. Discusses models, training neural nets, embeddings, tokens, transformers, language syntax.
The Tetris effect is one of psychology’s most easy to reproduce experiments.Simply spend a bit of time playing the eponymous game every day for a few weeks.A...
A blog by Yoshi.
Microsoft knowingly released a broken product for short-term hype
Everybody has heard about the latest cool thing™, which is DALL·E 2 (henceforth called Dall-e). A few months ago, when the first previews started, it was basically everywhere. Now, a few weeks ago, the floodgates have been opened and lots of people on the waitlist got access - that group included me. I’ve spent a day…
The experiments that read raw weights and the ones that fix Qwen use completely different techniques.
Let’s fix these pesky Transformer outliers using Softmax One and QuietAttention.
Anthropic's Claude Fable policy turns coding assistants into non-neutral infrastructure for product companies.
AI makes projects with weak engineering culture fail much faster.
Neither OpenAI nor Anthropic have a chance at this point
Demystifying attention, the key mechanism inside transformers and LLMs.
Everyone is worried about Chinese models, but the frontier labs will be fine; we need to enable open U.S. alternatives.
Since the disastrous launch of the RTX 50 series, NVIDIA has been unable to escape negative headlines: scalper bots are snatching GPUs away from consumers before official sales even begin, power connectors continue to melt, with no fix in sight, marketing is becoming increasingly deceptive, GPUs are missing processing…
Deep learning has been at the forefront of the so called AI revolution for quite a few years now, and many people had believed that it is the silver bullet that will take us to
The fix was in, and Dario never had a chance.
Watching OpenAI’s new o3 model guess where a photo was taken is one of those moments where decades of science fiction suddenly come to life. It’s a cross between the …
A lot has happened in the world of Large Language Models over the course of 2024. Here’s a review of things we figured out about the field in the past …
My experience adopting any meaningful tool is that I've necessarily gone through three phases: (1) a period of inefficiency (2) a period of adequacy, then finally (3) a period of workflow and life-altering discovery.
PR by Xuan-Son Nguyen for llama.cpp: This PR provides a big jump in speed for WASM by leveraging SIMD instructions for qX_K_q8_K and qX_0_q8_0 dot product functions. Surprisingly, 99% of …