speakerphone annoyance!

This happens very randomly, but when it does it is quite annoying...  I never use speaker phone and sometimes when I make a call, it will automatically start the call with speaker phone on which is annoying because the phone will be against my ear and is LOUD!  I have yet to find a reasoning for why the speaker would begin a call in the ON position when I never use speakerphone.  help!