Where does the "rude frenchmen" stereotype come from?

Granted, I haven't been to France or had much exposure to the French, but the only rude French person I have met was a French speaker from Switzerland. So, should I count myself lucky I haven't encountered these people, should I ume that they don't really exist on the scale that popular culture portrays them as or do the xenophobic ones simply refuse to leave their country?