Wow. This one is hard. Well, my chivalrous side thinks he should let you keep the pet, but then the other half that loves animals says why should he have to part with a pet he's equally attached to just because he's a dude. If he's the one who initiated the break-up, it would be nice if he let you keep the pet. But if it's a mutual break-up, that makes it harder.
Ultimately, you should do what is best for the dog/cat/rabbit/ferret/etc. If one of you has a bigger house, or a backyard or lives closer to the vet or a park with other animals, then that person should keep the pet. BUT the other person is allowed visitation rights. I've known couples who split up and then basically both have joint custody of the pet. It's like a child -- you've both grown attached and, like you said, put in the time, money and love to nurture it.
Sure, telling your new boyfriend that you're going over to your ex's place to see your dog is a bit weird, but it can work if both of you are willing to be mature and think about the animal first and foremost. He/she has also grown attached to you, and doesn't understand that you're no longer a couple. So whoever ends up with the pet should allow the other person some quality time alone with Barky or Whiskers, whether it be taking him or her to the park, dog/cat-sitting when the owner is out of town, scheduling playdates with your new pet, etc. Divorced parents deal with split custody of children all the time. There's no reason why you can't be in the animal's life even if you've no longer in his.
If you had a mutual friend who wanted an animal it may make things easier! That way you'd both get to see it if you don't want to come into contact with your ex. Obviously it's not ideal and it depends on the state of things between you.
As a general rule of thumb, I think the person who initiates the breakup (the "dumper") should let the "dumpee" keep the pet. When you've just been abandoned by your lover, there's nothing more comforting than a cuddle with your furbaby. And when you're coming home to a *now* empty home instead of one filled by your lover, again, it's comforting to have a pet there keeping it from being totally empty. Obviously doesn't apply for mutual breakups, but really, when are breakups ever 100% mutual? Somebody has to initiate it.