A switch on the keyboard to physically disable the Caps Lock key. This way users who don't use the key can't accidentally press it.

As is already mentioned in the list above: a programmable key.
The thing I would like to explicitly add is that it should also be possible to program it to function as the Caps Lock key.

1) Make Caps Lock only work in combination with Left Shift. This way the user always has to press 2 keys, and is less likely to enter Caps Lock mode.

2) Always put an LED on the Caps Lock key so the user can see he/she is in Caps Lock mode without having to look at the screen.