- Chaos Theory
- Posts
- 🥟 Chao-Down #302 Apple releases a new AI system that can understand screen context and conversations, How AI is changing eye exams for the better, A look at the challenge of safely testing AI
🥟 Chao-Down #302 Apple releases a new AI system that can understand screen context and conversations, How AI is changing eye exams for the better, A look at the challenge of safely testing AI
Plus, researchers find that giving an "inner monologue" to AI greatly improves performance.
What can we expect from Apple for their AI ambitions? While recent headlines have shown the iPhone-maker to be willing to strike deals with other model providers like Google, the company is still looking for homegrown innovation.
Apple's AI research team recently released a new system called ReALM (Reference resolution as Langauge Modeling), which enhances voice assistant interactions by understanding screen context and conversational background. ReALM outperforms existing methods and even GPT-4 in being able to parse on-screen entities into a textual layout for language models to process.
From the researchers:
“Being able to understand context, including references, is essential for a conversational assistant… Enabling the user to issue queries about what they see on their screen is a crucial step in ensuring a true hands-free experience in voice assistants.”
Will we finally see the long-awaited upgrade to Siri with ReALM? Many eyes will be fixed on Apple as they announce their latest products at the upcoming Worldwide Developers Conference.
I know I’ll be tuning in.
-Alex, your resident Chaos Coordinator.
What happened in AI? 📰
Nobody Knows How to Safety-Test AI (TIME)
Apple researchers develop AI that can 'see' and understand screen context | (VentureBeat)
Why OpenAI and Other Data-Hungry AI Companies Need a Bigger Internet (WSJ)
AI is upending eye exams for patients and providers—in a good way: ‘It’s been a godsend for us’ (Fortune Well)
Larry Summers thinks AI could replace ‘almost all' forms of labor. Just don’t expect a 'productivity miracle' soon (Fortune Asia)
Researchers gave AI an 'inner monologue' and it massively improved its performance | (Live Science)
Always be Learnin’ 📕 📖
Your AI Product Needs Evals - How to construct domain-specific LLM evaluation systems. (hamel.dev)
Untangling concerns about consolidation in AI (Substack)
Putting Data in a Corner: LLMs and Financial Services Data | Andreessen Horowitz (a16z.com)
Projects to Keep an Eye On 🛠
topoteretes/cognee: Knowledge and Memory management for the AI Applications and AI Agents (Github)
mrphrazer/reverser_ai: Provides automated reverse engineering assistance through the use of local large language models (LLMs) on consumer hardware. (Github)
wandb/openui: OpenUI let's you describe UI using your imagination, then see it rendered live. (Github)
The Latest in AI Research 💡
AIOS: LLM Agent Operating System (arxiv)
Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Generation (arxiv)
HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption (arxiv)
The World Outside of AI 🌎
Half a million California workers will get $20 minimum wage, starting today (NPR)
Google to delete records from Incognito tracking (bbc.com)
Homebuyers need to earn 80% more than they did in 2020 to afford a home in today's market (Yahoo)
Microsoft unbundles Office and Teams globally following years-long criticism | (TechCrunch)
Japan: Fertility Startups Race to Meet Demand for IVF, Infertility Treatment - (Bloomberg)
Memories are made by breaking DNA — and fixing it (Nature)
One Last Bite 😋
