It is downright frustrating when you login to, for instance, Solaris 10 while you are in a hurry and keep pushing the wrong keys so your console looks like a greasy garbage dump filled with ^H^H^H^H and you just want to stangle yourself.
I shall never forget how to solve this problem by typing in this life saving coma^hmand:
stty erase <backspace key>
The command should look like this after you have hit your backspace key:
stty erase ^H
Now backspace key will function as expected, without starting bash.
To set-and-forget this on a global level:
echo stty erase <backspace key> >> /etc/profile