In other forums where I've seen such a system, it did nothing to encourage post quality. On the contrary, it encourages a sort of race to the bottom. Some people post-whore in an attempt to get upvotes, and other people go off the handle if they get a lot of downvotes. And it lets people hide behind the wall of anonymity that the button creates, so people invariably abuse the system by doing things like downvoting everything someone they don't like posts as a matter of course.
The folks who are the most intellectually engaged are generally be the ones who ignore the popularity contests and associated trifles that points systems are most useful for, and instead focus on the thing that actually conveys intellectual value: the words of the post.
Despite all of the site's faults, Slashdot probably has the best user mod/karma system I've seen. While I agree with those who think the forum is better off without such a system, if there
was one, I'd prefer it be modeled after that.
For those who aren't familiar with that site, it basically works as follows:
Users will occasionally get five mod points. How often they get them is dependent on a whole bunch of factors, but on the whole it tends to be a slightly uncommon thing. Once they have mod points, they have a limited time to use them (somewhere in the range of a couple days, I think?). Then they can spend them to up or downvote in any thread they haven't posted in. Posts start at 1 (2 for those with high karma, 0 for those with low) and a post's karma is capped at -2 to +5.
While this system isn't perfect, the rationed mod points
does seem to encourage mods to upvote genuinely good posts. Plus, the caps limit the value of karma whore posts (which isn't to say that it removes the value altogether).
On the whole, it's a pretty good system, and I wish more sites with karma systems would adopt it instead of giving everyone unlimited votes all the time.
