• Chaos Theory
  • Posts
  • 🥟 Chao-Down #203 NVIDIA trains LLMs for chip design, PwC partners with OpenAI and Harvey to build legal domain-specific models, Britain calls for a sovereign rival to ChatGPT for national defense

🥟 Chao-Down #203 NVIDIA trains LLMs for chip design, PwC partners with OpenAI and Harvey to build legal domain-specific models, Britain calls for a sovereign rival to ChatGPT for national defense

Plus, MIT unveils Air-Guardian, an AI copilot for airline pilots leveraging Liquid Neural Networks.

October 31, 2023

As we’ve had more time to dive into the White House’s latest executive order on the safe and secure development of AI, one section that caught my eye was the reporting requirements for companies training models.

Specifically, in the section highlighted below, the executive order seems to imply that only giant models — those trained on tens of thousands of GPUs (~ 50K NVIDIA H100s or $50M+) or on any cluster with 10^20 FLOPS — need to be reported.

If the trend is for models to continue to get bigger, then eventually any company developing sophisticated AI models will need to comply. This is certainly the case for the tech giants like OpenAI, Anthropic, Google, etc. But another way to think about this is that the executive order might drive innovation for smaller, cheaper models to be built to bypass any US regulation.

@nearcyan for the H100 calculation

Some food for thought.

-Alex, your resident Chaos Coordinator.

What happened in AI? 📰

People are speaking with ChatGPT for hours, bringing 2013’s Her closer to reality (Ars Technica)

PwC partners with OpenAI and Harvey to build domain specific foundation models (PwC)

Britain needs sovereign rival to ChatGPT for the nation’s defense (telegraph.co.uk)

Moody’s to Start Using Tech Backed by AI Help Write Analytical Reports (Bloomberg)

Nvidia Trains LLM on Chip Design (EE Times)

Alibaba launches upgraded AI model to challenge Microsoft, Amazon (CNBC)

MIT’s copilot system can set the stage for a new wave of AI innovation (VentureBeat)

An isometric perspective view inside the cockpit of a commercial airplane mid-flight. In the foreground, a pilot sits calmly at the controls monitoring various screens and instruments. Behind the pilot, a semi-transparent digital overlay of an AI assistant known as "Air-Guardian" appears just above the pane of glass showing the view from the aircraft.

Always be Learnin’ 📕 📖

Researchers Discover Emergent Linear Structures in How LLMs Represent Truth (aimodels.fyi)

Once Upon a Brand: The Power of Storytelling in Marketing (Marketing Insider Group)

Why you should add friction to your onboarding (equals.com)

Projects to Keep an Eye On 🛠

Audio-AGI/AudioSep: Official implementation of "Separate Anything You Describe" (Github)

THUDM/AgentTuning: AgentTuning: Enabling Generalized Agent Abilities for LLMs (Github)

Flash-Decoding for long-context inference (PyTorch)

Vaibhavs10/insanely-fast-whisper - Transcribe 300 minutes (5 hours) of audio in less than 10 minutes - with OpenAI's Whisper Large v2. Blazingly fast transcription is now a reality! (Github)

The Latest in AI Research 💡

Proving Test Set Contamination in Black Box Language Models (arxiv)

FreeNoise - Tuning-Free Longer Video Diffusion via Noise Rescheduling (haonanqiu.com)

uni-medical/SAM-Med3D: SAM-Med3D: An Efficient 3D Model for Promptable Volumetric Medical Image Segmentation (Github)

The World Outside of AI 🌎

GM's biggest bets are running out of juice (axios.com)

Toyota, the World’s Biggest Carmaker, Made a Huge Bet on Tech. It Went Wrong Fast (WSJ)

Elon Musk to employees: In a year, X could replace bank accounts (Mashable)

Is Red Meat Bad for Your Health? Science Says Yes (Bloomberg)

California’s Ban on Red Food Dye Puts FDA on the Spot (The New York Times)

To obscure the risks of gas stoves, utilities used Big Tobacco tactics (Grist)

One Last Bite 😋

Are you buying or renting?