Bob -
I can only assume your concern about other's being worried about your adderall use is that you are in a law-enforcement or medical/drug councilling job. I am still having trouble with why anyone but your HR people would even know if you haven't told them.
The problem is that non-amphetamine meds haven't worked with many people to date. It just isn't as suited for ADHD as our regular meds.
Remember you are protected in North America by LAW as to discrimination on these. Unless you have prior history of abuse no employer could hold your med usage against you!
I say stick up for your rights! If the meds work - don't quit!
Check with your local chapter of CHADD (www.chadd.org) for info on personal rights in this manner. I certainly would as I'd hate to see you get worse when things are going well for you.
I'm kind of in a situation with my job and the pills I'm on . Won't go into all the details, but the Adderall I'm on may not be a good suit for my job. Not that I've gotten into any trouble with it but some don't approve of it!