Bdtyk

Oh my!

In the above example, to me, "oh" seems to suggest one should pronounce "o" as a short vowel, whereas "o", seems to suggest one should pronounce "o" as a long vowel. In other words, I would expect it to be spelled as:

O my!

I would expect "oh" to be used in this example, meaning to express "oops":

Oh o!

... or used an an expression of excited amazement, when watching beautiful fireworks for instance, like this:

Oh!

So why is "Oh my!" spelled this way?

There is no difference between the pronunciation of oh and o; in both the cases, the pronunciation is /oʊ/.

Oh is used to express surprise, anger, disappointment, or joy; it is also used when reacting to something just said.

Oh, shut up.

O is also an archaic spelling of oh; it was also used before a name in direct address.

Give peace in our time, O Lord.

I think the other answers miss mentioning that the spelling O is most often used in poetic language before a name/noun in direct address, not necessarily just in a religious context.

O CAPTAIN! my Captain! our fearful trip is done;     

The ship has weather’d every rack, the prize we sought is won;   

The port is near, the bells I hear, the people all exulting,     

While follow eyes the steady keel, the vessel grim and daring:

  But O heart! heart! heart!

    O the bleeding drops of red,     

      Where on the deck my Captain lies,     

        Fallen cold and dead.

— Walt Whitman

It is more common to see the spelling Oh as an interjection or expression of surprise, although the spelling O can be used and is not incorrect, even if it is falling out of favour. Note that there are some phrases where the spelling Oh is nearly always preferred.

Oh yeah? You want to make something of it?

'Why' questions are hard in spelling. The answer could be something like 'Noah Webster said so' or 'in the primary dialect of Wessex in AD XXXX, all words ending with a certain pronunciation were spelled a certain way (and those are the only source of such spellings in Modern English)', or 'somebody just made it up one day, and everybody started using it from then on' (oh, that last example is too much like the first).

Unfortunately nothing like that seems to be the case for 'oh'. It looks like it just is. There is a history to it (as others have noted) where 'oh' does alternate with 'o'. But it's just that 'oh' seems to be more popular nowadays. And there doesn't seem to be any traceable evidence that shows an event or historical trend explaining it.

O Lord! Why indeed? In fact it's not always spelt with an "h"...

...but in general the presence of the "h" does tend to make it more readable in other contexts.

"Oh" is an interjection expressing
any number of emotions, including surprise,
exasperation, desire, and gratitude.

"O" begins an address to a person or to God (thus it performs the same function that the
vocative case performs in certain
languages, for example in Latin; hence
the term "Vocative 'O'").

I feel that O and Oh are the same both in meaning and pronunciation so "Oh no" and "O no" are the same. The only case you will have to use Oh is that if the next word starts with vowel as in "Oh o". "O o" is confusing and doesn't please your eyes.