Audio elements have no rendering area for captions. However, in Chrome, if we have text tracks, we also render the captions toggle button in the audio controls. It's not a useful control and should be removed.
Since this was a Chrome-specific bug, at this point makes sense to mark it as WontFix.