Contents |
Protein-Protein Interaction Network Inference (DREAM2, Challenge 2)
This archival page describes the challenge exactly as it was presented to the participants. Go to the main DREAM2 Challenge 2 page to download data, view team rankings, cite this work, etc.
Synopsis
For many pairs of bait and prey genes, yeast protein-protein interactions were tested in an unbiased fashion using a high saturation, high-stringency variant of the yeast two-hybrid (Y2H) method. A high confidence subset of gene pairs that were found to interact in at least three repetitions of the experiment but that hadn’t been reported in the literature was extracted. There were 47 yeast genes involved in these pairs. Including self interactions, there are a tot"al of 47*48/2 possible pairs of genes that can be formed with these 47 genes. As mentioned above some of these gene pairs were seen to consistently interact in at least three repetitions of the Y2H experiments: these gene pairs form the "gold standard positive" set. A second set among these gene pairs were seen never to interact in repeated experiments and were not reported as interacting in the literature; we call this the "gold standard negative" set. Finally in a third set of gene pairs, which we shall call the "undecided" set, genes were seen to interact only once or twice in repeated experiments, or were seen never to interact but were reported as interacting in the literature. The challenge consists of predicting which gene pairs belong to the gold standard positive set, and which gene pairs belong to the gold standard negative set.
Dataset
File with Gene IDs: The file Prot-Prot_Genes.xls contains a list of 47 yeast genes (identified with ORF IDs). The challenge consists of determining the set of true positive and the set of true negative protein-protein interactions among all the pairwise interactions between these 47 genes.
Submission Information
Submit a ranked list of gene pairs, ordered according to the confidence you assign to your prediction that a pair interacts, from the most reliable (first row) to the least reliable (last row) prediction. Use a tab-separated 3 column format as in the example below:
- YeastGeneA \tab YeastGeneB \tab XYZ
where YeastGeneA and YeastGeneB are genes in the file Prot-Prot_Genes.xls, and XYZ is an interaction score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair interacts. (E.g., XYZ = 1 if the pair is deemed to interact with highest confidence and XYZ = 0 if the pair is deemed not to interact.) All pairs omitted from the list but that belong to the gold standard positive or gold standard negative set will be considered to appear randomly ordered at the end of the list with XYZ = 0. Submitted pairs that belong to the undecided set will not be scored. Save the file as unformatted text, and name it:
- TeamName_ProtProtSubnet.txt
where TeamName is the name of the team with which you registered for the challenge.
Scoring Metrics
The submitted list will be judged exclusively on the gold standard positive and gold standard negative sets. Submitted pairs that belong to the undecided set will not be scored. We will score the results using the area under the precision versus recall curve for the whole set of predictions. For the first k predictions (ranked by score, and for predictions with the same score, taken in the order they were submitted in the list of gene pairs), precision is defined as the fraction of correct gold standard positive predictions to k, and recall is the proportion of correct gold standard positive predictions out of all the possible gold standard positive interactions. Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated.