I think it really depends on the style of the watch, the size of the wearer's wrist, and personal preference. I have an 8" wrist, so larger watches look much better on my wrist. I have plenty of watches from the 70s and 80s that are 32-36mm, when that was the norm, and all just look silly on me now. My wife and daughters wear them now.
But it does also depend on the style of the watch. I prefer a dress watch to be 40mm, a diver to be 40-42mm and a chronograph to be 42-44mm for my wrist. But those sizes might be way too big for someone with a 6.5" wrist.
The biggest watch I own is 48mm and it rarely gets wrist time due to its size. The smallest watch I wear regularly is the 39.5mm Seiko Alpinist, which also wears much smaller than its size due to the dial design, so much so that I sometimes wonder if I will keep that watch in the long run.
In my opinion, there are many factors that go into what makes a watch the right size. I have several 40mm dress watches, but each appears to be a different size due to differences on the bezel design and thickness.
And yet I stubbornly keep, and occasionally wear, a 36mm Raymond Weil Tango I bought in 1997 because I absolutely love the design. Go figure.