There is a long standing sound issue in Debian and derivatives like Ubuntu, where the FIRST playback of a track to a digital eg: HDMI output misses the first second or two, or plays a short sound at low volume "muffled", or fails to play a very short sound at all. Subsequent plays will usually be OK, at least within the next 5 seconds to 20 minutes - then the issue recurrs. It happens with all sound apps and file types.
Because it has the worst effect on the shortest sounds, it is particularly annoying for desktop alerts or 'event' sounds, and the terminal bell (another can of worms in Xfce).
The issue can be forced by logging out and in again, or if your machine is attached to a KVM, possibly just by switching the KVM to another machine and back again. I have had this since Debian 6 Squeeze through to Xubuntu 20.04 LTS, on several machines, both Intel and AMD chipsets. It's not PulseAudio - the Debian machine only used ALSA, no PulseAudio. It was present in Debian with Gnome 2, Ubuntu with Gnome 3, and Xubuntu (Xfce 4).
Analogue output is not affected.
Backgound
Many people seem to have fixed a similar issue with various ways of disabling the 'power save' feature of the Intel HDA driver, but they didn't work for me. That issue seems to be associated with clicks or pops at the beginning/end of playback - which I don't get. However, my issues also seem to result from a delay in starting the sound system.
Others fixed this by playing continuous silence to keep the system turned on, eg: Sound output starts delayed and https://unix.stackexchange.com/questions/362223/short-audio-playback-is-muted-requires-warming-up-or-secondary-audio-in-backgro - but again this didn't work for me.