Paving the way to efficient architectures: StripedHyena-7B
StripedHyena-7B is an open-source model built beyond the Transformer architecture, matching Llama-2 and Mistral 7B while running faster on long sequences.
Topics /
The most popular blog posts about ai & machine learning on Hacker News. 6,325 posts.
StripedHyena-7B is an open-source model built beyond the Transformer architecture, matching Llama-2 and Mistral 7B while running faster on long sequences.
Text-to-image diffusion models can create stunning images from natural language descriptions that rival the work of professional artists and photographers. However, these models are large, with complex network architectures and tens of denoising iterations, making them computationally expensive and slow to run. As a…
The other day I saw the launch of Apple Vision Pro, the whole thing was very interesting, but one thing that caught my attention was the finger input. It seems very intuitive, by using the finger pinching as sort of like a cursor or mouse input. I figured I want to try it out, so I took it upon myself to create it.
How does the brain work? While neuroscience provides insights on small-scale processes and overall behavior, we don’t know much for sure about the learning and processing algorithms in between. In this blog post, I will give an introduction to what we know. Let’s take an engineering perspective and play evolution: How…
As a supporter, I would love not to feel this way
The most exciting part of Programming Languages (PL) research for me has always been in Programming Language Design.
Running Kimi K3 at ~952 tok/s/node, AMD continues to prove its case as the winner in performance per dollar.
Netlify now runs any OpenRouter model, including Kimi K3, GLM 5.2 and DeepSeek V4. We tested 11 of them on the same build prompt to see how they differ.
New context editing and memory tools enable Claude agents to handle long-running tasks without hitting context limits.
Kelly predicted AI heavyweights while advocating for a future of open weights
Burnout is on the rise again, with an ironic twist.
OpenAI’s mission is to build safe AI, and ensure AI’s benefits are as widely and evenly distributed as possible.
At Replit, our mission is to empower the next billion software creators. Yesterday, we strengthened our commitment by announcing that Replit AI is now free for all users. Over the past year, we’ve witnessed the transformative power of building software collaboratively with the power of AI. We believe AI will be part…
Posted by Christian Szegedy, Software EngineerThe ImageNet large-scale visual recognition challenge (ILSVRC) is the largest academic challenge in c...
Over the past few years, we've seen that next-token prediction scales, denoising diffusion scales, contrastive learning scales, and so on, all the way to the point where we can train models with billions of parameters with a scalable objective that can eat up as much data as we can throw at it. Then, what about…
Can you see the resemblance?
K/V context cache quantisation has been added to Ollama. This enables significant reductions in VRAM usage, allowing users to realise the potential of expanded context sizes and run larger models at their existing context sizes.
We need scientific and technical breakthroughs to steer and control AI systems much smarter than us. To solve this problem within four years, we’re starting a new team, co-led by Ilya Sutskever and Jan Leike, and dedicating 20% of the compute we’ve secured to date to this effort. We’re looking for excellent ML…
What is the function of a fire alarm?
Hey! Before we go any further — if you want to support my work, please sign up for the premium version of Where’s Your Ed At, it’s a $7-a-month (or $70-a-year) paid product where every week you get a premium newsletter, all while supporting my
I've been trying to make progress on a herculean task; I want decent looking programs for personal use which I can generate quickly using AI agents yet I'm a person without taste controlling an AI without taste. I found "one weird trick" that does enough for me, the results are not extra nice but it doesn't make me…
Voicebox is a state-of-the-art speech generative model based on a new method proposed by Meta AI called Flow Matching. By learning to solve a text-guided speech infilling task with a large scale of data, Voicebox outperforms single-purpose AI models across speech tasks through in-context learning.
Today, Kog AI launches a tech preview of the Kog Inference Engine (KIE): 3,000 output tokens/s per request on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 (FP16, no speculative decoding). This preview runs a 2B model, with support for large third-party MoE models coming next at similar speeds.