Your desire for sex can be affected by many different things. A low testosterone level is one of the most common. In fact, the first clue that you have low testosterone could be a loss in sex drive. Testosterone is the main male sex hormone. One of its most important jobs is to keep you (men and women) interested in having sex. But there are other causes of low sex drive that may surprise you.