In short, no, except for phenylketonurics.

Aspartame is one of the most intensively studied food additives ever,
and the overwhelming scientific evidence is that it poses no danger.

The many claims of harm are all either anecdotal and not supported by
adequate observation, or are based on serious lack of understanding of
how to demonstrate facts scientifically. One of the most egregious is
the claim that studies with aspartame in capsules are invalid and that
it's only dangerous in solution. But d'oh -- if you administer
aspartame in solution, the patient will know whether he/she is getting
aspartame or not. This unblinds the experiment. Refer to Reid's Third
Law: Never Underestimate the Power of Suggestion.

An good set of links to web pages on aspartame is at
http://urbanlegends.about.com/library/blasp3.htm. (Unfortunately the
links open framed by about.com's heading, an unfair practice eschewed
by the vast majority of web sites. Ten demerits for about.com.)

The well known low-calorie sweeteners are pretty much all safe:
cyclamates, saccharin, aspartame, acesulfame, sucralose. Yes, even
cyclamates and saccharin -- the studies which resulted in their banning
turned out to be non-reproducible. I don't list stevia because it has
not been adequately studied, but I know of no significant indications
of danger.

If you don't like a given sweetener, try another. If you think you
respond badly in some way to a sweetener, try another. But unless you
have at least a heterozygous gene for phenylketonuria, it's unlikely
that you'll have a verifiable response to aspartame.