Chaos Theory
Posts
🥟 Chao-Down #290 Google Deepmind unveils a generalist AI agent for 3D virtual environments, EU passes the world's first comprehensive AI law, China uses AI to run the largest high-speed rail system

🥟 Chao-Down #290 Google Deepmind unveils a generalist AI agent for 3D virtual environments, EU passes the world's first comprehensive AI law, China uses AI to run the largest high-speed rail system

Plus, a look at the datacenter infrastructure powering Meta's push into GenAI.

March 14, 2024

Google Deepmind is back to their video game playing roots and this time around they are targeting all the games!

Researchers shared their latest work called SIMA (Scalable Instructable Multiworld Agent) an AI agent designed to play video games with human-like skills. Unlike typical AI for video games, SIMA aims to be a cooperative player, understanding natural language instructions and 3D environments and can go from pixels seen on the screen to controls on a keyboard.

From their post:

SIMA is an AI agent that can perceive and understand a variety of environments, then take actions to achieve an instructed goal. It comprises a model designed for precise image-language mapping and a video model that predicts what will happen next on-screen. We finetuned these models on training data specific to the 3D settings in the SIMA portfolio.

Our AI agent doesn’t need access to a game's source code, nor bespoke APIs. It requires just two inputs: the images on screen, and simple, natural-language instructions provided by the user. SIMA uses keyboard and mouse outputs to control the games’ central character to carry out these instructions. This simple interface is what humans use, meaning SIMA can potentially interact with any virtual environment.

SIMA has been trained on games like No Man’s Sky, Valheim, and Goat Simulator 3, learning from human gameplay and instructions. While Deepmind will say that they’ve only evaluated their AI on 600 basic skills and 10 second time horizons, the generalized approach definitely shows what could be the future of AI-based game playing.

Definitely a space to watch if you’re a gamer!

-Alex, your resident Chaos Coordinator.

What happened in AI? 📰

European Lawmakers Pass AI Act, World’s First Comprehensive AI Law (WSJ)

Intelligence officials warn pace of innovation in AI threatens US (CyberScoop)

Building Meta’s GenAI Infrastructure (Engineering at Meta)

SIMA generalist AI agent for 3D virtual environments (Google DeepMind)

DoorDash’s new AI-powered ‘SafeChat+’ tool automatically detects verbal abuse | TechCrunch

China using AI to run world's largest high-speed railway system (Interesting Engineering)

Always be Learnin’ 📕 📖

Turns out, Google is all about Links and Clicks (and Keywords) (seoforgooglenews.com)

Business-in-a-Box 2.0 - by Rex Woodbury (Digital Native)

Getting Machine Learning Projects from Idea to Execution (hbr.org)

Projects to Keep an Eye On 🛠

openai/transformer-debugger- Transformer Debugger (TDB) is a tool developed by OpenAI's Superalignment team with the goal of supporting investigations into specific behaviors of small language models. (Github)

koaning/drawdata: Draw datasets from within Jupyter. (Github)

Eventual-Inc/Daft: Distributed DataFrame for Python designed for the cloud, powered by Rust (Github)

The Latest in AI Research 💡

tinyBenchmarks: evaluating LLMs with fewer examples (arxiv)

Is Cosine-Similarity of Embeddings Really About Similarity? (arxiv)

Dialect prejudice predicts AI decisions about people's character, employability, and criminality (arxiv)

Design2Code: How Far Are We From Automating Front-End Engineering? (arxiv.org)

The World Outside of AI 🌎

Do some countries have better mental health than others? (Vox)

Pew survey finds that teens say they are happier when they go phone-free (Fast Company)

Microplastics in arteries linked with heart disease risk: Study (USA Today)

Boeing fails FAA safety checks over dish soap, hotel keys (qz.com)

Automakers Are Sharing Consumers’ Driving Behavior With Insurance Companies (The New York Times)

The Rich Are Starting to Have More Babies Than the Poor Again (Bloomberg)