You should ask him out. Life is short.
I'm not sure what his having a lot of female friends has to do with anything. If you like him, go for it. What's the worst that could happen? Rejection? There are worse things, trust me. You have little to lose and much to gain. It's a no-brainer.
It's like my buddy Virgil said a few* years back: Audentes fortuna iuvat. That's fancy talk for "fortune favors the bold."
Be bold.
* 20 B.C.
Or as the British Special Air Service puts it in their motto "who dares wins".