💡DishwasherBench: Load Dishes, Benchmark AI
Load dishes, score AI models
TL;DR
DishwasherBench is a new game that simulates loading a Bosch dishwasher to benchmark AI models. Players score points based on how well they clean dishes, with early versions showing promise but limitations.
DishwasherBench is a browser-based game that simulates loading a Bosch SMV4HAX40G dishwasher to benchmark AI models. Players score points based on how well they clean dishes, with the game calibrated on one real wash. While early versions show promise, they don't yet account for all cleaning factors. The simulation runs on a MacBook Pro and takes 4.9 minutes for a wash, with scores showing whether items are packed well but not if they stack for optimal cleaning. This tool is a unique way to test AI models in a practical, real-world context.
Key Points
DishwasherBench simulates a Bosch SMV4HAX40G dishwasher, with a wash taking 4.9 minutes on a MacBook Pro (Apple M3 Pro, 18 GB).
Players score points based on how well they clean dishes, with the game calibrated on one real wash. Scores show packing efficiency but not stacking for optimal cleaning.
The simulation runs on a MacBook Pro and takes 4.9 minutes for a wash, with scores showing whether items are packed well but not if they stack for optimal cleaning.
DishwasherBench uses Node 20.19 or newer and pnpm 9, with a GPU that Dawn can open for water simulation through WebGPU.
The game is an early release, version 0.1.0, with a score showing whether a player packed well and put cups, glasses, and pots in mouth down.
Why It Matters
DishwasherBench is a unique tool for AI developers to test models in a practical, real-world context. It simulates loading a Bosch dishwasher and scoring based on cleaning efficiency, offering insights into how AI models perform in everyday scenarios. However, it's still in early stages, with limitations on stacking for optimal cleaning.
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,570 builders reading daily.