somebody might want to pose this question to the RPh on the board, because as i understand the way alli and orlistat work is that they are not systemically absorbed, thus the only side effects we should see are those involving the GI tract. I'm a pharmacy tech myself and have done a lot of homework before starting alli.
i think it's worth remembering that most of sexual desire is mental, which sort of triggers the physical. i really think there is much to be said for the idea of being preoccupied with the idea of having a TE, or of not feeling the best because of other GI SEs that go along with taking the drug. it doesn't really put you in the mood.
so, check out and see what the pharmacists have to say about this, or talk to your MD. i personally haven't had any problems, but i know i've been getting less fat and calories than i'm supposed to, thus limiting the chance for TEs.