I recently got the s24 ultra and love the sound of the speakers. But I’m wondering if playing music too loud could break them.
No, you should be fine. Just turn it up and enjoy.
I don’t think the phone will let you turn it up loud enough to damage the speakers.
It’s made to last, it should hold up just fine. 
Be careful though, it gets pretty loud.
I’m sure the speakers can handle it. They seem built well. Just make sure you’re mindful of your surroundings and people around you. Hopefully, no distortion at max volume. Enjoy the music!
Yeah, the phone might even fly away with the music you’ll play!
Actually, the opposite might happen.
It’s your ears that might get hurt more than the speakers.
The speakers shouldn’t break, but sound quality usually drops after about 70-80% volume. Beyond that, you might notice some distortion.
If turning the volume up too much could break the speakers, they would’ve hidden the volume control somewhere else. Just crank it up, buddy.
It will break if you keep watching videos or playing music through the speakers. Always use headphones instead.