I can't replicate this on my laptop, but years ago (when we had good old DOS) I seem to remember that pressing either control and caps lock or alt control caps lock seemed to lock the caps so you got the punctuations at the top of the top row rather than the numbers. the same combination toggled this off, so that might be worth a try.