3961
RedPajama: Reproduction of LLaMA with friendly license
RedPajama launches to build fully open-source leading language models, starting by reproducing LLaMA's training dataset of over 1.2 trillion tokens.
New
The latest blog posts to take off on Hacker News. 6,334 posts.
3962
I used Stable Diffusion and Dreambooth to create an art portrait of my dog
In this post, we walk through my entire workflow/process for bringing Stable Diffusion to life as a high-quality framed art print. We’ll touch on making art with Dreambooth, Stable Diffusion, Outpainting, Inpainting, Upscaling, preparing for print with Photoshop, and finally printing on fine-art paper with an Epson…
3963
Understanding large language models: A cross-section of the relevant literature
A Cross-Section of the Most Relevant Literature To Get Up to Speed
3964
Batch computing and the coming age of AI systems
There’s a lot of excitement right now about human-in-the-loop systems supercharged by foundation models including chat assistants ( ChatGPT ), word processing ( Microsoft Office ), graphic design ( Stable Diffusion ), and code editing ( Copilot ). However, these systems only begin to scratch the surface of the role…
3965
33TB of text data for a 1T-parameter model
Important: This page summarizes data scaling only, using tokens to parameters as a ratio, and as derived from large language models like GPT-3, Chinchilla, and beyond, linked to the Compute-Optimal scaling laws like Kaplan and Hoffmann/Chinchilla. Please note that compute scaling laws are outside the scope of my…
3966
The Decade of Deep Learning
As the 2010’s draw to a close, it’s worth taking a look back at the monumental progress that has been made in Deep Learning in this decade.[1] Driven by the development of ever-more powerful comput
3967
What are transformer models and how do they work?
Explore the fundamentals of transformer models, which have revolutionized natural language processing.
3968
3969
Prompt injection: what’s the worst that can happen?
Activity around building sophisticated applications on top of LLMs (Large Language Models) such as GPT-3/4/ChatGPT/etc is growing like wildfire right now. Many of these applications are potentially vulnerable to prompt …
3970
Beyond automatic differentiation
Posted by Matthew Streeter, Software Engineer, Google Research Derivatives play a central role in optimization and machine learning. By locally app...
3971
Temporal quality degradation in AI models
A recent study from MIT, Harvard, The University of Monterrey, and other top institutions showed an experiment where 91% of their ML models degrade over time . This study is one of the first of its kind, where researchers focus on studying machine learning models' behavior after deployment and how their performance…
3972
Building LLM Applications for Production
[Hacker News discussion, LinkedIn discussion, Twitter thread]
3973
3974
Sorting waste and recyclables with a fleet of robots
Posted by Sergey Levine, Research Scientist, and Alexander Herzog, Staff Research Software Engineer, Google Research, Brain Team Reinforcement lear...
3975
Amazon CodeWhisperer, Free for Individual Use, Is Now Generally Available
Today, Amazon CodeWhisperer, a real-time AI coding companion, is generally available and also includes a CodeWhisperer Individual tier that’s free to use for all developers. Originally launched in preview last year, CodeWhisperer keeps developers in the zone and productive, helping them write code quickly and securely…
3976
Do you need a vector database?
Spoiler alert: the answer is maybe! Although, my inclusion of the word “actually” betrays my bias. Vector databases are having their day right now. Three different vector DB companies have raised money on valuations up to $700 million (paywall link). Surprisingly, their rise in popularity is not for their “original”…
3977
Scaffolded LLMs as natural language computers
Recently, LLM-based agents have been all the rage – with projects like AutoGPT showing how easy it is to wrap an LLM in a simple agentic loop and prompt it to achieve real-world tasks. More generally, we can think about the class of ‘scaffolded’ 1 LLM systems – which wrap...
3978
Experimenting with LLMs to Research, Reflect, and Plan
Also, shortcomings in document retrieval and how to overcome them with search & recsys techniques.
3979
Replacing my best friends with an LLM trained on 500k group chat messages
An exploration into customizing LLMs for personal use
3980
Free Dolly: First truly open instruction-tuned LLM
Introducing Dolly, the first open-source, commercially viable instruction-tuned LLM, enabling accessible and cost-effective AI solutions.
3981
GPT-4 gets a B on my quantum computing final exam
Update (April 14): On the suggestion of my TA Justin Yirka, I gave GPT-4 problem 2 from the final exam, along with its own answers and my and Justin’s grading comments, and asked it whether i…
3982
3983
3984
Maximizing the Potential of LLMs: A Guide to Prompt Engineering
Language models have rapidly improved in recent years, with large language models (LLMs) such as GPT-3 and GPT-4 taking center stage. These models have…
3985
Twitter showed us its algorithm – what does it tell us?
On March 31, Twitter open-sourced its recommendation algorithm. It turns out to be a standard engagement prediction algorithm of the kind most major platforms use; I explained how these algorithms work in a recent essay . The source code release makes for an interesting case study of social media transparency. Let’s…
3986
Emergent abilities of large language models
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
3987
Teaching ChatGPT to speak my son’s invented language
Becoming fluent in my son’s invented language was too difficult for me, so I decided to see how would AI do.
3988
Why is GPT-3 15.77x more expensive for certain languages?
How large language models process multilingual data differently
3989
From deep to long learning?
For the last two years , a line of work in our lab has been to increase sequence length. We thought longer sequences would enable a new era of machine learning foundation models: they could learn from longer contexts, multiple media sources, complex demonstrations, and more. All data ready and waiting to be learned…
3990
I wish GPT4 had never happened
Made in Notion, the connected workspace where AI agents and teams organize docs, projects and knowledge together.