Thanks! I wanted something that would be useful to:

1. Pick the experts out of the crowd and elevate their posts.
2. Encourage civility/helpfulness/other community behaviors. (I've seen forums that would encourage flames and rudeness, so customizable ratings are important.)
3. Make the trolls easier to flag - without automatically suppressing their speech. (Some people really believe that crazy stuff. They need to be mocked, not silenced.)
4. Let moderate/quiet/scared people have a voice without requiring them to be vulnerable to backlash. The bad stuff is out there, and most people don't agree with it, but most people aren't going to post their disagreement - they'd rather just click a button.
5. Something a little more nuanced. Whee "Funny Troll"!

The problems with the above system that I can see are:

• Do you make aggregate post ratings public? That might convince some people not to post at all. (Is that good? More signal, less noise? What about the people who take pride in their Troll Level 9000?)
• How the heck could anyone make this scalable? I guess if anyone could though, you guys could!
• What tags do you permit? A list of tags could easily shape the discussion - people will be trying to earn specific ones, avoid others, etc - but you still have to make sure to cover "what people are going to really feel about a post", or they'll start using tags for other meanings and your measuring system will be out of whack.
  • Do you permit normal users to apply topic-specific tags to posts? So you could have users tag a post as being "about Linux" or "about free speech", whatever you choose to call those topics. That could help with catching the experts, but might subject the expert-finding system to easier gaming.
  • The absolute basic tags you'd need to apply to catch people's feelings (I think) would be something like:
    • The gut check stuff:
      • Agree
      • Disagree
      • Rude
      • Unhelpful
      • Funny
      • Serious
    • The nice stuff:
      • Polite
      • Helpful
      • Interesting
    • The factual stuff:
      • True
      • False
      • Mostly-true
      • Mostly-false
      • Half-true
      • Troll
    • I thought about including stuff like "stupid" and "hell yeah" but I figure if you're trying to encourage civility you probably shouldn't allow the silent majority to outright insult people in their tags. XD
    • There's probably stuff I'm missing here. This post is already way too huge, though.
• How can this system encourage people to read and tag more new posts?
  • Give a "reader rating"? "You have read and tagged 900+ posts! Congratulations!"
  • Give their tags more weight? So long as you don't let the bots take over.
  • Can you rate a tagger based on how many people agree with their tags? The only thing that seems good for is determining whether someone is with the group mind or against it on certain topics.
  • Let the good ones have custom tags? I have a bad feeling about that - but I'd love to earn a tagging rep so I could tag people "whackadoodle" or "window-licker". Maybe moderators would have to approve the custom tags to keep them civil or whatever. That could get overwhelming, but they're short, right?

I think Slashdot was headed this way but stopped when it got to be "good enough". I got the "multi-tagging" idea from Pipedot, though. I noticed how once multiple people agree that a post is off-topic, it gets flagged off-topic - unless more people agree that it's funny. When moderating, I've actually fished around until I found the adjective other moderators wanted to apply so I could make it show up right away - is this post "insightful" or "interesting", oh let's click that one since the word will show up faster. When I caught myself doing that, I realized that the site must store both tags - and I thought, why not show them?

Things to look out for:

• What if 9 bajillion people tag a controversial post 200 different things? I think you should only show the top 10 tags but then you run the risk of moderation being shouted down. Shown tags are more likely to receive metamoderation, though, so my expectation is that tags that get added quickly (or were added by the original poster) will win out, and tags that are outright wrong will be dropped quickly from the list.
• What about people specifically posting only in certain topics, trying to raise their rank to "expert" in those topics? I... guess you end up with a very informative site then?
• Find a way to handle a threaded conversation that is 1000+ comments gracefully - ha! Good luck! XD

See, toldja I put too much thought into this! But it is a very fun topic!