🤖Google Adds Kernel Profiling to XProf for TPU Workloads
XProf now shows cycle-level details in custom kernels
TL;DR
Google has added Kernel Profiling to XProf, allowing developers to see cycle-level details in custom Pallas kernels on TPU v7. The new suite works at three levels and offers sub-microsecond capture with external event triggers.
Google has added Kernel Profiling to XProf, its open-source profiler for TPU workloads, enabling developers to see cycle-level details in custom Pallas kernels. This is a big deal for teams working on complex TPU applications, as it allows for deeper insights into performance bottlenecks and optimization opportunities. The suite works at three levels, offering detailed views through flags, Graph Viewer, and Trace Viewer. In a matmul case study, a memory-bound variant showed large spikes in sync_wait counters, which were reduced to 88µs from 125.5µs with improved buffering.
Key Points
XProf samples hardware performance counters during runtime on TPU v7 (Ironwood) with 1µs resolution.
New external event-triggered mode removes 1µs floor, enabling sub-microsecond capture with improved attribution.
Developers can configure up to 28 counters per core across up to four SparseCores (4x28 matrix).
Collection is enabled through jax.profiler.ProfileOptions, using tpu_enable_periodic_counter_sampling.
Perf Counters View lists over 16,000 raw counters in tabular form, reflecting maximum raw counter value.
Why It Matters
If you're working on TPU v7 applications, the new Kernel Profiling suite in XProf flips the script on optimization. It offers cycle-level details, crucial for pinpointing and fixing performance issues. However, the limited 4x28 counter budget means teams must carefully select counters for each investigation. This is a game-changer for teams with complex TPU workloads.
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,488 builders reading daily.