On any issue about which expert opinion is important, the Collaboratorium community needs to have a way of soliciting and sharing these expert opinions.

Here is a proposal for one way to do this:

  1. Anyone who wants to can put an item on a list (maintained publicly somewhere in the system) of items that need expert review. Types of items that could be added to such a list include:
    • A plan (but see discussion below about whether expert opinion is appropriate for an item)
    • A position in a debate (whether the debate is attached to a plan or is in the stand-alone debate section)
    • An argument in a debate
    • A model
    • A position embodied in a plan (not implemented yet)
  2. Any expert whose expertise is relevant can express "expert" opinions about any of the above types of items (whether the item is on the list of "items that need expert review" or not)
  3. Part of a moderator's job is to notice important items that are in need of review and solicit reviews of these items from relevant experts
  4. Expert opinion about an item is expressed as a rating on a "credibility" scale. We propose that this scale be a simple probability scale, from 0% - 100% probability. We could interpret the scale, then, as follows:
    0 - no possibility whatsoever that that item could be true
    50 - the item is just as likely to be true as false
    100 - there is no doubt whatsoever that the item is true
    Note that, if this proposal is adopted, we need capabilities in the system to let experts make these ratings and then share the ratings.
  5. As a default, let's assume that the system shows expert ratings in several ways (users can choose to see any of them):
    • average rating
    • histogram showing the spread of ratings
    • individual ratings by individual experts (we may eventually allow expert ratings to be anonymous, but the starting default would be that they are public).
  6. There should be an evolving, official list of tags for different kinds of expertise. Anyone should be able to indicate the type of expertise that is relevant for a specific item (including "none"). If someone else feels that the expertise tag on an item is wrong, there should be a way for them to appeal to a moderator. If someone feels a moderator's decision about this is wrong, there should be a way to appeal to some more extended process (e.g., votes by a number of moderators).
  • No labels