511
Transformers from Scratch (2021)
I procrastinated a deep dive into transformers for a few years. Finally the discomfort of not knowing what makes them tick grew too great for me. Here is that dive.
New
The latest blog posts to take off on Hacker News. 789 posts.
512
513
The A.I. business influencer guys must be stopped
Please let's stop predicting the future of A.I.
514
Double Descent in Human Learning
The uncanny resemblance between double descent and a 50 year old theory from psychology
515
Squawk bots: Can generative AI lead us to understanding animals?
The Earth Species Project is using large language models to develop tools to decipher animal communication.
516
Capturing the Flag with GPT-4
This weekend I went to BSides SF 2023 and had a blast. I went to some really interesting talks (including an excellent one about adversarial machine learning), but mostly I spent my time solving CTF hacking challenges. And this time, I did it with the help of GPT-4, the
517
Bing Jail
A short story on how my website lost all indexed pages in Bing, affecting my presence in DuckDuckGo, secret blacklisting and classification as a splog by Bing AI
518
ProfileGPT: An Example of AI Agents Collaboration Architecture
A dive into the architecture and design of ProfileGPT, an app that allows users to analyze their user's profile and personality as seen by ChatGPT to raise awareness about data usage.
519
Prompt engineering vs. blind prompting
"Prompt Engineering" emerged from the growth of language models to describe the process of applying prompting to effectively extract information from language models, typically for use in real-world applications.
520
Finetuning Large Language Models
An introduction to the core ideas and approaches
521
GitHub Copilot emits GPL code
Demonstrating that GitHub Copilot trains on non-permissive licenses and cannot filter suggestions properly, while Codeium avoids legal risk.
522
Bard coding update with Colab exporting
Bard can now help with programming and software development tasks, across more than 20 programming languages.
523
Large language models: Scaling laws and emergent properties
More details on the science of training large autoregressive language models.
524
Building a ChatGPT-enhanced Python REPL
In this blog I share my experience in building a Python REPL augmented with ChatGPT. I explore how the application is built, and speculate on software engineering patterns and paradigms that might emerge in systems built on Large Language Models (LLMs). GEPL - Generate, Evaluate, Print, Loop Link to this section…
525
Google DeepMind
DeepMind and the Brain team from Google Research will join forces to accelerate progress towards a world in which AI helps solve the biggest challenges facing humanity.
526
Record labels panic as A.I. generated song becomes ‘hit single’
Welcome to the new world.
527
How to train your own large language models
From raw code to a model serving users: the data pipelines, training infrastructure, and evaluation work behind Replit’s language models.
528
Basic math related to computation and memory usage for transformers
We present basic math related to computation and memory usage for transformers
529
StableLM: A new open-source language model
Stability AI's open-source Alpha version of StableLM showcases the power of small, efficient models that can generate high-performing text and code locally on personal devices. Discover how StableLM can drive innovation and open up new economic opportunities while supporting transparency and accessi
530
NaturalSpeech 2: Zero-shot speech and singing synthesizers
Kai Shen*, Zeqian Ju*, Xu Tan*, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, Jiang Bian
531
532
MRI brain images become 64M times sharper
MRI technology from Duke-led effort reveals the entire mouse brain in the highest resolution
533
An example of LLM prompting for programming
Generated knowledge and chain of thought prompting of an LLM can generate useful code.
534
Community Is the Future of AI
To keep knowledge open and accessible to all, we must come together to build the future of AI.
535
RedPajama: Reproduction of LLaMA with friendly license
RedPajama launches to build fully open-source leading language models, starting by reproducing LLaMA's training dataset of over 1.2 trillion tokens.
536
I used Stable Diffusion and Dreambooth to create an art portrait of my dog
In this post, we walk through my entire workflow/process for bringing Stable Diffusion to life as a high-quality framed art print. We’ll touch on making art with Dreambooth, Stable Diffusion, Outpainting, Inpainting, Upscaling, preparing for print with Photoshop, and finally printing on fine-art paper with an Epson…
537
Understanding large language models: A cross-section of the relevant literature
A Cross-Section of the Most Relevant Literature To Get Up to Speed
538
Batch computing and the coming age of AI systems
There’s a lot of excitement right now about human-in-the-loop systems supercharged by foundation models including chat assistants ( ChatGPT ), word processing ( Microsoft Office ), graphic design ( Stable Diffusion ), and code editing ( Copilot ). However, these systems only begin to scratch the surface of the role…
539
33TB of text data for a 1T-parameter model
Important: This page summarizes data scaling only, using tokens to parameters as a ratio, and as derived from large language models like GPT-3, Chinchilla, and beyond, linked to the Compute-Optimal scaling laws like Kaplan and Hoffmann/Chinchilla. Please note that compute scaling laws are outside the scope of my…
540
The Decade of Deep Learning
As the 2010’s draw to a close, it’s worth taking a look back at the monumental progress that has been made in Deep Learning in this decade.[1] Driven by the development of ever-more powerful comput