TAU: Taiwan Audio Understanding Benchmark

What is TAU?

Large audio–language models are advancing rapidly, yet most evaluations emphasize speech or globally sourced sounds, overlooking culturally distinctive cues. This gap raises a critical question: can current models generalize to localized, non-semantic audio that communities instantly recognize but outsiders do not? To address this, we present TAU (Taiwan Audio Understanding), a benchmark of everyday Taiwanese “soundmarks.” TAU curates everyday, locally distinctive Taiwanese non-speech sounds and evaluates models with multiple-choice questions (MCQs) that cannot be answered by semantic reasoning alone, steering evaluation toward timbre, rhythm, and iconic acoustic patterns. Beyond Taiwan, TAU illustrates how localized benchmarks can highlight cultural blind spots in audio–language models, include underrepresented communities in multimodal evaluation, and guide the design of more equitable and robust multimodal systems.

Dataset Snapshot

TAU contains 702 audio clips across 10 culturally distinctive categories. This diversity ensures that the dataset covers both highly frequent urban soundmarks (e.g., transit chimes, store jingles) and less common but socially important cues (e.g., emergency alarms, religious chants). The category distribution is intentionally imbalanced to reflect the natural frequency of sounds in everyday Taiwanese soundscapes, rather than enforcing artificial uniformity. Each clip is paired with up to four MCQs, resulting in 1794 evaluation items in total. The median clip length is 9.43 seconds, with a maximum of 30 seconds by design. This design balances realism with usability, allowing evaluation without excessive cognitive load. On average, each soundmark has 2.1 recording variants that differ by location, time, or background conditions, which increases robustness and reduces overfitting to specific contexts.