creeps and weirdoes

I realised when I was quite young that I had a "talent" for attracting weirdoes... I could be out with a group of friends, and the clearly socially-inept freak that everyone was avoiding would always single me out for their very own brand of special attention. In my youthful innocence, I was always polite, not wanting to upset and offend... this usually resulted in being plagued by unwanted attention!
As I grew older, and my sarcasm and cynicism developed (the sarcasm more than the cynicism), I learnt that I had to be cruel to be kind - as in I had to be cruel to them in order to be kind to myself! So I armoured myself with a wide variety of blunt and sarcastic comments, which has often worked to great effect. But this is a prevention and not a cure, and so I have often questioned "why me? why do these freaks always single me out for attention?" The modal answer (most popular answer - for the non-Maths whizzes) has consistently been:
"Your warm smile and friendly nature....they view you as someone too kind to reject their advances!"
So what now? Do I go around with a gloomy look, the typical trademark look of all londoners?
Plus add to that a new dimension to this dilemma, what if the freak is someone you work with?
This is my current predicament. I say work with, we don't work together, just in the same place. So I've gone back to being polite, because I don't want to get a reputation as a bitch. But he seriously creeps me out...
So what should I do? This is the thought that has been keeping me awake this week, and any advice would be very gratefully received...