Versions Compared

Old Version 5

changes.mady.by.user W Benjamin Bray

Saved on

compared with

New Version Current

changes.mady.by.user W Benjamin Bray

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migration of unmigrated content due to installation of a new plugin
*

Full

Proposal

Peer

Review

*

includes

the

following

steps:

# Submitted full proposals are compiled by eSG Admins and uploaded to eSG via FMP, and accepted for review. # If your program uses peer review configuration #2 (see [FMP Interface Configuration - RFP|manuals:FMP Interface Configuration - RFP]), an evaluation criteria set for each call will have to be created in FMP. # Reviewers are added/edited, assigned to full proposals and emailed login and peer review instructions, via FMP. # Reviewers {link-window:https://seagrant.mit.edu/RFP/proposals/rev_login.php|width=1000px|height=850px}submit peer reviews{link-window}. # Reviews are viewed via the {link-window:https://seagrant.mit.edu/RFP/proposals/rev_admin_login.php\|width=1000px\|height=850px}Administrative Review Web Interface{link-window} or FMP, and compiled via FMP for distribution. {color:#ff0000}For a full description of the RFP FMP interface, go to :{color}

  1. Peer Review configuration and instructions are verified.
  2. Submitted full proposals are compiled by eSG Admins and uploaded to eSG via FMP, and accepted for review.
  3. Reviewers are added/edited, assigned to full proposals and emailed login and peer review instructions, via FMP.
  4. Reviewers .
  5. Reviews are viewed via the or FMP, and compiled via FMP for distribution.
Section


Wiki Markup

Peer Review Configuration; Instructions

Panel
borderColor#CCCCCC
bgColor#FFFFFF
borderWidth1
borderStylesolid

Peer Review is configured as either Simple or Complex; set in Call Management. 

  • Simple : Reviewers apply a single rating and comment to assigned proposals.
  • Complex : Reviewers apply scores for up to five evaluation criteria and a comment to assigned proposals. 
    If 'Complex' peer review is chosen, you must create an Evaluation Criteria Set for it in FMP (Reviews > Evaluation Criteria Sets).

Peer Review Instructions also need to be verified; see Instructions Sets for Calls and Reporting Periods.

Also, confirm that the Peer Review Deadline is correctly set for the call in Call Management.

Compiling and Uploading Full Proposals; Accepting for Review

Panel
borderColor#CCCCCC
bgColor#FFFFFF
borderWidth1
borderStylesolid

For each proposal,

  1. Go In FMP, go to Full Proposals > 90-2, and click the button: 'PDF: All parts for this proposal, incl. budget worksheets' under 'Reports'. This triggers creation of a folder on the user's desktop named for the related Proposal ID, into which the system deposits the full proposal coverpage(s), 90-2, narrative, budget worksheets or 90-4s, project time schedule, subcontract materials, support letters and CV / resumes, and any other miscellaneous uploads.
  2. Use a program like Adobe Acrobat to compile all of these pdfs into a single pdf named for the related Proposal ID (e.g. 'Bray_1234.pdf').
  3. Go to Full Proposals > Files, and upload the compiled full proposal pdf via the blue upload button. The uploaded full proposal will become available to PIs via the Control Panel, eSG Admins via the Administrative Review web interface, and to peer reviewers in the Peer Review web interface.
  4. At the top of Full Proposals, toggle the 'Accepted for Review?' to 'Yes'.

...

Select the Assignments tab.  To assign a reviewer to a specific proposal (among active calls and proposals accepted for review), click the green \[+\] button beneath 'Assigned Full Proposals for Peer Review', and select the proposal via the drop-down that appears


Reviewers may be assigned to proposals either from the Reviewers or Full Proposals layouts.

  1. (from Reviewers) Select the Assignments tab.  To assign a reviewer to a specific proposal (among active calls and proposals accepted for review), click the green [+] button beneath 'Assigned Full Proposals for Peer Review', and select the proposal via the drop-down that appears.
     
  2. (from Full Proposals) Select the Peer Reviewers tab. The list "Active Peer Reviewers" reflects all reviewers assigned to the role "Peer Reviewer", with a status of "active". To assign a reviewer in this list to the proposal in view, click the green [+] button.

Email Reviewers

  • Use the appropriate email template 'announcement of peer review process'customized for your program, which includes reviewer-specific login privileges and peer pre-proposal review instructions. Customize this template if necessary.[ see General Information - RFP - Email Tool ]
Panel
borderColor#CCCCCC
bgColor#FFFFFF
borderWidth1
borderStylesolid

Add / Edit Reviewers

  • Go to Reviewers > Suggested.  These reviewers were suggested by PIs during the pre-proposal submission phase.  To promote a Suggested Reviewer to serve as an actual Reviewer, click the 'Promote to Reviewer' button.  This will move a Suggested Reviewer's information into the Reviewers table.
  • At Add additional reviewers as necessary. At a minimum, the following fields must be populated when adding a reviewer: last name, first name, email, and password (use blue [+] button to generate).
  • Add the role of 'Peer Reviewer' via the Roles list at top-right, and set the role status to 'active'.

Roles are used for eSG system and organizational purposes. The following roles are used by eSG to permit access to web interfaces and SHOULD NOT BE ALTERED:

Pre-Proposal Reviewer
Peer Reviewer
Panel Reviewer
Admin
Sub-Admin

Other roles may be added and applied to reviewers, depending upon your program. For example, if you have two groups of peer reviewers, you'd assign everyone in the group to the 'Peer Reviewer' role, and then assign each group to its own organizational role.

Assign Peer Reviewers to Full Proposals

Wiki Markup

Peer Review Submission

The

{link-window:https://seagrant.mit.edu/RFP/proposals/rev_login.php|width=1000px|height=850px}Peer Review web interface{link-window}

presents

to

the

reviewer

all

full

proposals

assigned

to

them,

including

the

PI name, proposal title, and the

compiled

full

proposal

pdf

PDF uploaded

by

the

eSG

Admin.

Admin.  Evaluation fields reflect the full proposal peer review configuration (Simple or Complex) chosen for the call (see above). Reviewers are able to access the Peer Review web interface only if they have active full proposals assigned to them.

Panel
borderColor#CCCCCC
bgColor#FFFFFF
borderWidth1
borderStylesolid
Wiki Markup

Peer Review Management and Compilation

Reviews are viewable via the {link-window:https://seagrant.mit.edu/RFP/proposals/rev_admin_login.php\|width=1000px\|height=850px}Administrative Review Web Interface{link-window} or FMP, as they are added by reviewers. The Admin Review web interface shows reviews by proposal, while FMP shows reviews when browsing by full proposal, by reviewer, or as a list under 'Reviews'. You have several options for retrieving pdfs of peer reviews: # _By Proposal_ : {link-window:https://seagrant.mit.edu/RFP/proposals/rev_admin_login.php\|width=1000px\|height=850px}Administrative Review Web Interface{link-window} : click 'View Peer Reviews / Rebuttals' for a full proposal, and then export or print the pop-up window as a pdf # _By Proposal_ : FMP > Full Proposals > Reviews: Click 'View: Peer Reviews' and then 'Save as PDF'. # _By Proposal or Reviewer_ : FMP > Reviews > Reports: Select call to retrieve peer reviews for, click either 'View: Peer reviews, by proposal', or 'View: Peer reviews, by reviewer', and then 'Save as PDF'.
Panel
borderColor#CCCCCC
bgColor#FFFFFF
borderWidth1
borderStylesolid

Viewing

Reviews are viewable via the or FMP, as they are added by reviewers. Only full proposals whose "Accepted for Peer Review?" field is set to 'Yes' will appear in this interface. The Admin Review web interface shows reviews by proposal, while FMP shows reviews...

  • when browsing by proposal
  • by reviewer
  • as a list via Reviews (which includes the Rating of Review field supporting Reviewer evaluation, found in Reviewers > Evaluation)
  • via Reports > Review Reports.

Retrieving

You have several options for retrieving pdfs of peer proposal reviews:

  1. By Proposal :
    1. : click 'View Peer Reviews / Rebuttals' for a full proposal, and then export or print the pop-up window as a pdf 
    2. FMP > Full Proposals > Reviews: Click 'View: Peer Reviews (this proposal)' (blinded/unblinded) and then 'Save as PDF'.
  2. By Call, sorted by Proposal or Reviewer : FMP > Reports > Review Reports: Select call, then click either
    1. 'View: Peer reviews, by proposal', (blinded/unblinded) or 
    2. 'View: Peer reviews, by reviewer'
      and then 'Save as PDF'.
  3. By Review : FMP > Reviews: Switch to Form View of desired review and then save as PDF via the FMP main menu.

Review Tracking Grid

This feature (accessed via Reviews layout) lists scores/ratings, averages, and reviewers for submitted reviews in a 2D grid. Place the cursor over ratings to see the associated reviewer name.

Reviewer Evaluation

  • Each review may be rated via the Reviews layout, or through the 'Reviews' tab of the Reviewers layout.
  • Each reviewer may be evaluated or described using the 'Evaluation' tab of the Reviewers layout. You may include, keywords, comments about the reviewer and their performance, view the average quality of their reviews, and apply ratings.
Wiki Markup