Approximate specs: quad-core xeon around 2.8 ghz each. About 20gb of fast ram. Average hard drive on an SATA interface. My pipe is verizon fios ... 25 Mb up and down.
Wow... I'm not surprised you have more RAM than my machine, but I wasn't expecting 20gb

I'd be really interested in seeing how a stress test runs on that. Of course, 20gb of RAM would mean that a machine we build would be that much more expensive.
I was telling a friend of mine about this project last night and he showed me a cool site for this:
http://canihostaminecraftserver.com/ Assuming it is accurate, the setup and internet connection you described could host 76 players. The 4gb of RAM on my computer bottlenecks it at 30, so RAM is going to be important. I'd play around with the site and find the sweet spot fr RAM, but I'm at work (and shouldn't be on the forums right now, anyway.)
It doesn't ask about CPU, so it is either assuming you are running it on a server with a pretty powerful CPU, or CPU stops being a bottleneck pretty quickly.