From Docs // Creators.RockBand.com
Peer Review is the process by which members of the RBN Creators Club determine whether a song is ready to be approved for sale on the Xbox Live Marketplace. After a song has gone through at least one round of Playtesting, the author may promote it to Peer Review, where members with Premium XNA memberships submit their reviews via a web form.
Peer Review vs. Playtest
When a song is first submitted to the RBN Creators Club, it goes into a 14-day Playtesting period where members can provide feedback on all aspects of the authoring via the Playtest Process and the Advanced Playtesting form. A song can go through multiple rounds of Playtesting. When the author feels that the song has been sufficiently playtested and all the bugs & issues worked out, they will promote it to Peer Review.
Peer Review is the "last line of defense" before a song can be added to the Marketplace, so it is important both for authors to avoid submitting songs for review which haven't been thoroughly playtested, and for reviewers to be detailed and honest in their reviews.
Peer Review Votes
A song will remain in Peer Review for 30 days after it is submitted. During this time it must receive 8 positive reviews in order to be approved for the Marketplace.
The mechanics of what constitutes a positive or negative review are mostly hidden from the end user, to avoid manipulation of the system. Essentially, if you check any of the boxes on the review form you are giving a failing vote on that specific category. Each category has a different number of failing votes required to cause the song to actually fail Peer Review. Some can fail with a single vote. The majority need 2 votes. Difficulty differences require anywhere from 8 to 2 votes, depending on how different the reviewers say the difficulty is. A difference of one "dot" on a difficulty rating does not count as a negative vote against the song. If the song receives 8 reviews within the 30-day period without hitting that threshold of failing votes on any given category, it is approved.
A song's progress is tracked via a "Review Progress", displayed as a percentage. Because of how the system works behind-the-scenes, the percentage does not indicate the number of reviews received in an obvious way. The progress will change to 25% after one review and 50% after the second, with each subsequent review adding 7-9%. This is because the first two people who review typically satisfy both gameplay criteria and metadata criteria, and hence more of the behind-the-scenes checkboxes are marked off. The later reviews don't count for less, but some of the things they are voting on have already been satisfied.
Approval/Failure
If a song receives the required number of reviews without failing in any category, it enters a state called "Review Closed." Songs in the "Review Closed" state are subjected to a 48 hour quarantine, during which administrators will take a look at the song to make sure it is not in violation of any terms, check for copyrighted content, etc. During the quarantine period, the song will not be visible to end users on the site. Once it receives final approval from the administrators it will appear with a "Song Approved" status and be visible to all RBN Creators Club members, even those without Premium XNA memberships, and will be available for purchase in the RBN Store.
If a song receives enough failing reviews in a specific category or doesn't receive 8 total reviews within the 30-day period, it is removed from Peer Review. The author is notified when this occurs, including the reason for the failure, and must resubmit the song and start the review process over again. Generally this involves making changes to the authoring or metadata to address the issue(s) that caused reviewers to fail the song.
