From the CSS spec definition for scroll-padding (https://www.w3.org/TR/css-scroll-snap-1/#scroll-padding), scroll-padding should apply to any scroll container, not just scroll snap containers. This issue is particularly problematic when using scroll-padding to adjust for any fixed content (for example, a fixed header or footer).
This behavior exists on both iOS and macOS Safari, but not on Chrome or Firefox.
Note: this is particularly problematic when using iOS with VoiceOver on, as VoiceOver will trigger tapping on the location where its focus is, which means that if you're focused on an element that's underneath a fixed container, the click event will trigger on the fixed container and not the element that gets read out as having focus. As this issue isn't a problem in Chrome, I was able to successfully avoid this issue on an Android device with TalkBack enabled.
I created a really basic codepen (https://codepen.io/backwardok/pen/6980c6068b69618a90bd5b12716dde8b) that demonstrates this problem (full page version: https://cdpn.io/backwardok/debug/6980c6068b69618a90bd5b12716dde8b).
Thanks for the testcase!