Dream Initiative - New pages [en]

XD4c3full

Summary:

__NOTOC__
= Predictive Signaling Network Modeling =
== DREAM4, Challenge 3 ==

'''Note: Both the data and pathway map cannot be used for purposes other than this challenge without the explicit permission of the data providers. (See below for contact information.)'''

== Synopsis ==

This challenge explores the extent to which our current knowledge of signaling pathways, collected from a variety of cell types, agrees with cell-type specific high-throughput experimental data. Specifically, we ask the challenge participants to create a cell-type specific model of signal transduction using the measured activity levels of signaling proteins in HepG2 cell lines. The model, which can leverage prior information encoded in a generic signaling pathway provided in the challenge, should be biologically interpretable as a network, and capable of predicting the outcome of new experiments.

== The challenge ==

It is an open question how to make use of the accumulated body of knowledge of signaling pathways to create mechanistic, predictive signaling network models. The network depicted in Figure 1 is representative of the type of information about the topology of signaling pathways that can be culled from the literature [1]. Figure 1 depicts canonical pathways downstream of major receptors to four ligands (represented by green nodes): two inflammatory (TNFa, IL1a), one insulin (IGF-I), and one growth factor (TGFa). Note that this pathway map is not cell-type specific.

In addition to the topology of "canonical" signaling pathways based on the accumulation of evidence from multiple cell-types, we have at our disposal a data set consisting of measurements of phosphoprotein activity levels in the HepG2 cell line using the Luminex xMAP sandwich assay. Measurements of certain phosphoprotein activities were measured under various perturbations of the signaling pathways [2]. The signaling pathways were stimulated with one or more of the ligands mentioned above. The pathways were also perturbed with chemical inhibitors of specific phosphoproteins. In Figure 1, blue and magenta nodes indicate the phosphoproteins measured by the xMAP assay; red and magenta nodes indicate the phosphoproteins that were inhibited.

[[Image:DREAM4_Challenge3_Figure1.png|center]]

<blockquote><blockquote>Figure 1. Pathway map summarizing the current public knowledge of the signaling pathway pertaining to this challenge, simplified from [1]. Green nodes represent stimuli. Red nodes represent inhibited proteins. Blue nodes indicate proteins whose phosphorylation is measured. Magenta nodes represent proteins that are both inhibited and measured. Grey nodes represent proteins considered to be involved in the relevant pathways. This figure was created with Cytoscape http://www.cytoscape.org/) from data obtained from the Ingenuity knowledgebase.</blockquote></blockquote>

The Hep2G data set is plotted in Figure 2, which is organized into panels corresponding to the various ligands. Upon pretreatment with an inhibitor (or no inhibitor), measurements (phosphoprotein activities) of seven proteins at three time points (0, 30 minutes, and 3 hours post stimulus) were acquired.

The challenge entails "customizing" the provided pathway map (Figure 1) so that it is an accurate representation of the provided data set (Figure 2). Specifically, we are soliciting

# A revised network specific to the HepG2 cell line. The revised network could be produced by removal of links that are not supported by the provided data set from the pathway map of Figure 1, and/or, addition of links that are supported by data, but absent from the pathway map of Figure 1.
# The predicted values of the 7 measured phosphoproteins for all 20 possible pairwise combinations of the following stimuli and inhibitors which comprise a "test set."

Stimuli: IL1a, IGF1, TGFa, and TGFa+IGF1
<br>Inhibitors: pp38+MEK, PI3K+MEK, p38+PI3K, p38+IKK, and PI3K+IKK

In the above list, TGFa+IGF1 indicates that both TGFa and IGF1 were simultaneously applied to the cells. The same is true for simultaneous application of inhibitors such as PI3K+IKK.
The answer to the challenge should entail some interplay between predictive modeling and network reconstruction. Any modeling formalism may be used as long as the model is amenable to be interpreted as a network. A wide range of modeling formalisms can be applied and model relevance will be ascertained by how close the model predicts the response to the set of test stimuli and inhibitors.

[[Image:DREAM4_Challenge3_Figure2.png|center]]

<blockquote><blockquote>Figure 2. Training data set. Time courses for the phosphorylation of 7 key proteins (rows) in the cancer cell line HepG2 treated with 5 different protein inhibitors (including no inhibitor) under 5 different conditions of cytokine stimulation (panels, including no cytokine stimulus) [2]. When the measured molecule is inhibited the measurement cannot be used (denoted with a big red X). The numbers at the right of the figure indicate the maximum value for the signals across all conditions (i.e., the maximum value of the corresponding row) and it is in arbitrary units (fluorescent intensity). The figure was created using DataRail [3].</blockquote></blockquote>

== Data ==

The canonical pathway map (Figure 1) is provided in several formats with filenames
* '''DREAM4_SignalingNetwork.ext'''
where ext is one of the following: pdf, gml, xgmml, cys, and sif. All formats were created with Cytoscape. The sif (simple interaction file ) format contains the human readable list of edges in source/target format; Gml (Graph markup language ) and xgmml (extensible graph markup and modeling language) provide additional information about the network visualization (not relevant for the analysis), and cys is the intern Cytoscape format.

The data set (Figure 2) is provided as comma-separated-value files in two formats, DataRail’s MIDAS (Minimum Information for Data Analysis in Systems Biology) format [3], and a simple table, with the filenames:
* '''SignalingNetworkChallenge_TrainingData_MIDAS-format.csv'''
* '''SignalingNetworkChallenge_TrainingData.csv'''

=== Important information regarding measurements ===

(a) Data integrity / linearity. Significant effort was dedicated to data integrity. The data are reported as arbitrary (fluorescence) units in the range between 0 and ~29000. The upper limit (~29000) corresponds to the saturation limit of the detector. Experiments were performed in such a way that measurements are as much as possible within the linear range of the detector. In general, data can be considered linear but there are a few cases that measurements are closer to the upper detection limit of ~29000 (e.g. some AKT measurements) where linearity might have been lost.

(b) Detection limits/Repeatability. The coefficient of variation for repeated measurements was found to be ~8% (mostly due to biological variability). With our current experimental design the instrument detector can report data with accuracy as low as ~300. For example, changes from 55 fluorescence units (FU) to 110 FU cannot be considered "2 fold increase" because values lie within the noise error of the detector. On the contrary, data from 1000 to 2000 are significant.

(c) Comparability between phosphoproteins. In the xMAP sandwich assays used to collect the data for this challenge, the fluorescence measured for one phosphoprotein is not directly comparable to that of another phosphoprotein. For example, the same readout of 1000 in AKT and ERK signal does not imply that the concentration of AKT and ERK are the same. The reason is that even at the same concentration, the amount of light detected for different phosphoproteins depends on the affinity of the antibodies to the phosphoproteins.

(d) Comparability between training and test sets. The lysate concentration used for the measurements of the training data set (contained in the file SignalingNetworkChallenge_TrainingData.csv) was different from the lysate concentration used for the test data set. This was done to keep the measurement values within the linear range of the detector. Therefore, even for the same phosphoprotein and under the same conditions, the measurement in the training and test data sets could be different. This is why we give the value of the measurement at t=0 in the file DREAM4_TeamName_SignalingNetworkPredictions_Test.csv, as these values could, in principle, be different from the values at t=0 for similar conditions in the training set. Therefore, the predictions at t=30 min have to take into account the baseline value at t=0 of the test set rather than equivalent measurements in the training set. For clarifications on this important aspect of the data, please feel free to contact the DREAM organizers or the data providers.

== Submission ==

Challenge participants will submit three files:

(1) Predictions of the 7 phosphoprotein activities under the various perturbations in the test set. These predictions should be submitted within the template file:
* '''DREAM4_TeamName_SignalingNetworkPredictions_Test.csv'''
provided with the data. At submission, replace TeamName with the name of your team, and the entries containing the text "PREDICT" with your numerical predictions.

(2) A list of edges of the network underlying your predictions. (The model used to produce your prediction must be interpretable as a network.) Submit this list as the file
* '''DREAM4_TeamName_Edges.txt'''
replacing TeamName with the name of your team. Your network must be submitted as a tab delimited list of node pairs, which represent edges in the network. Only edges supported by your model should be included in the submitted edge list, and the order of the list is inessential. Edges such as tgfa→ erk12 and igf1→ hsp27, for example, should be encoded as:

tgfa \tab erk12
igf1 \tab hsp27
...
Identify the nodes using ONLY the following node labels:
tgfa, igf1, tnfa, il1a, akt, jnk12, erk12, ikb, hsp27, mek12, p38, pi3k, ikk

These are the colored nodes in Figure 1. Your network submission may not have self-loops or node labels other than those provided above. (See section Network compression for submission for additional information.)

(3) A one to two page write-up explaining how the predictions are produced from the network. The write-up helps enforce the purpose of the challenge: to develop a predictive network model. This write-up can contain pseudo-code describing the algorithm used. Submit the write-up as the file
* '''DREAM4_TeamName_Writeup.ext'''
replacing TeamName with the name of your team and the file extension (ext) with your choice of txt, doc, rtf, or pdf.
Network compression for submission. Only some of the nodes in the pathway map of Figure 1 are measured or manipulated in the HepG2 cell line data. However, a model may contain a representation of additional proteins that are not measured or manipulated in the assays (latent variables). To facilitate scoring and comparison of models from different teams, we ask that the network edges be reported using only the nodes that we provide for the explicit purpose of submission of the network. For example, if your model has a pathway A-->B-->C, but B is a latent variable, then this pathway should be "compressed" and reported as A-->C for the purpose of submission of the network.

== Scoring ==

The submissions will be scored by the prediction error in the test set and the parsimony of the submitted network. More specifically the prediction cost function will be scored as a sum of squared errors over all the predictions. The teams that have a prediction error below a threshold (determined by a low p-value) will be further evaluated according to the number of edges in the submitted network. Of the most significant predictive models, the team with the sparsest network will be considered a best performer.

We realize that "cheating teams" might use a prediction model that is not associated with any network interpretation, and submit a network without edges. We strongly discourage this, as the idea of this challenge is to explore the possibility of using predictive models that are biologically interpretable, and could lead to the formulation of new hypotheses.

== References ==

[1] Saez-Rodriguez J, Alexopoulos L, Epperlein J, Samaga R, Lauffenburger DA, Klamt S, Sorger PK. Submitted.

[2] Alexopoulos L, Saez-Rodriguez J, Cosgrove B, Lauffenburger DA, Sorger PK. Net-
works reconstructed from cell response data reveal profound differences in signaling by
Toll-like receptors and NF-κB in normal and transformed human hepatocytes. Submitted.

[3] Saez-Rodriguez J, Goldsipe A, Muhlich J, Alexopoulos LG, Millard B, Lauffenburger DA, Sorger PK. Flexible informatics for linking experimental data to mathematical models via DataRail, Bioinformatics. 2008 Mar 15;24(6):840-7. (http://code.google.com/p/sbpipeline/wiki/DataRail).

== Authors ==

The challenge was generously provided before publication by Julio Saez-Rodriguez, Leonidas Alexopoulos*, and Peter Sorger, from the Department of Systems Biology, Harvard Medical School and Biological Engineering Department, M.I.T. The challenge has been designed in collaboration with Robert Prill and Gustavo Stolovitzky from the IBM T.J. Watson Research Center in New York.

*Present Address: Department of Mechanical Engineering, National Technical University of Athens

== Download ==

* [{{link}}/data/DREAM4/ Download Data]

Don't hesitate to post a question in the DREAM [{{link}}/discuss discussion board] if you need any clarification on this challenge.

XD4c2full

Summary:

__NOTOC__
= Predictive Signaling Network Modeling (DREAM4, Challenge 2) =

== Download / Upload ==

* [{{link}}/data/DREAM4/ Download Data]

XD4c1full

Summary:

__NOTOC__
= Peptide Recognition Domain (PRD) Prediction =
== DREAM4, Challenge 1 ==

== Synopsis ==

Many important protein-protein interactions are mediated by peptide recognition domains (PRD), which bind short linear sequence motifs in other proteins. For example, SH3 domains typically recognize proline-rich motifs, PDZ domains recognize hydrophobic C-terminal tails, and kinases recognize short sequence regions around a phosphorylatable residue [1].

Given the sequence of the domains, the challenge consists of predicting a position weight matrix (PWM) that describes the specificity profile of each of the given domains to their target peptides. Any publicly accessible peptide specificity information available for the domain may be used.

== Background ==

Ideally, PRD specificity could be predicted directly from the sequence of the domain itself. This will enable the prediction of protein-protein interaction networks directly from the genome sequence.

The specificity of selected human SH3, synthetic PDZ and kinase PRDs were experimentally mapped using phage display and combinatorial peptide libraries. The peptide libraries contain many short peptides with diverse sequences, around ten amino acids in length. The domain is used to select peptides from the library that bind to it. The set of peptides that bind to a domain defines a short, linear sequence pattern that the domain is expected to recognize. This pattern can be represented probabilistically as a position weight matrix (PWM).

Publicly available information about the domain family that may be useful for prediction includes known ligands of members of the domain family from the literature or databases like DOMINO[2] or PDZBase[3] and structures from the PDB[4].

== The Challenge ==

Peptides bound by SH3, PDZ, and kinase PRDs were experimentally identified. These data constitute an unpublished "gold standard" for the binding specificity of the selected PRDs.

Given the sequence of the domains, the challenge consists of predicting a position weight matrix (PWM) that describes the specificity profile of each of the given domains to their target peptides. Any publicly accessible peptide specificity information available for the domain may be used.

== Data ==

* '''DREAM4_DomainSequences.txt''' contains 5 human SH3 domain sequences, 3 serine/threonine kinase sequences and 5 synthetic PDZ domain sequences modeled on Erbin (Erbb2 interacting protein).

== Submission ==

Using the provided tab delimited template file

* '''DREAM4_TeamName_PWM.txt'''

and keeping the formatting of this file, submit a ten-column PWM for each domain. An example PWM is illustrated below. Each row corresponds to an amino acid, each column corresponds to the probability that the given amino acid is found at that position. Each of the ten columns must sum to 1.0. (Note that the amino acids are ordered alphabetically by IUPAC single letter code. Please keep this template format.)

<center>
<pre>
A 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
C 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
D 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
E 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
F 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
G 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
H 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
I 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
K 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
L 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
M 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
N 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
P 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
Q 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
R 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
S 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
T 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
V 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
W 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
Y 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
</pre>
</center>

* If a column is not predicted, enter 0.05 for all rows in that column, signifying uniform background frequency.

* If a PWM is not predicted, leave 0.05 for all columns and all rows for that PWM.

* All PWM predictions must be placed in one text file according to the template, keeping the order of the template file as it is.

* A best performer will be identified for each of the three domain types (SH3, PDZ, and kinase). You must submit predictions for at least one of the domain types. All the instances of the PRD in a given domain type must be predicted in order for your submission to be scored in that domain type.

* Replace TeamName in the filename "DREAM4_TeamName_PRD.txt" with the name of your team before submitting.

== Scoring Metrics ==

The submitted PWM predictions will be judged exclusively by similarity to the experimentally mapped PWM using the distance induced by the Frobenius Norm (http://mathworld.wolfram.com/FrobeniusNorm.html).

Domain specific notes:

*Kinase: Column 6 in the PWM must correspond to the phosphorylatable S/T residue in the peptide that binds to the kinase.

* PDZ: Column 10 in the PWM must correspond to C-terminus of the peptide that binds to the PDZ domain.

* SH3: No anchor position in the PWM is defined. Every possible alignment of the predicted SH3 peptide specificity PWM with the experimentally mapped SH3 peptide specificity PWM of length >=5 will be tried and the final score will be equal to the highest similarity found.

== References ==

# Pawson T, Nash P (2003) Assembly of cell regulatory systems through protein interaction domains. Science 300: 445-452.
# Ceol A, Chatr-aryamontri A, Santonico E, Sacco R, Castagnoli L, et al. (2007) DOMINO: a database of domain-peptide interactions. Nucleic Acids Res 35: D557-560.
# Beuming T, Skrabanek L, Niv MY, Mukherjee P, Weinstein H (2005) PDZBase: a protein-protein interaction database for PDZ-domains. Bioinformatics 21: 827-828.
# Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat TN, et al. (2000) The Protein Data Bank. 28: 235-242.

== Download / Upload ==

* [{{link}}/data/DREAM4/ Download Data]

Don't hesitate to post a question in the DREAM [{{link}}/discuss discussion board] if you need any clarification on this challenge.

Secret

Summary:

__NOTOC__


{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:30%; color:#000"| 
{| style="width:500px; border:solid 0px; background:none;"
|style="width:400px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">Prediction Submission Deadlines
</div>
|}

|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="padding:.3em;">
* Sept 15, 2009 Prediction Submission OPENS
* Oct 15, 2009 Prediction Submission DEADLINE
* Nov 15, 2009 Scores Released, Best Performers Notified
* Dec 2-Dec 6, 2009 [[DREAM4conf | RECOMB Systems Biology / Regulatory Genomics / DREAM4 Conference]] (The DREAM4 conference track is on Friday December 4th.)
</div>
|}


Nov 15, 2009 Notifications to Predictors of their Scores and Ranks.
Dec 2-Dec 6, 2009 RECOMB Systems Biology/Regulatory Genomics/DREAM4 conference. (The DREAM4 conference track is on Friday December 4th.)

= Secret =

[[Image:Signaling_Cascade_Model_and_Challenge_Figure.jpg|center]]
[[Image:FigureLegend.jpg|center]]

[[Image:Signaling_Cascade_Model_and_Challenge_Figure.jpg|thumb|example image caption]]


{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:30%; color:#000"| 
{| style="width:500px; border:solid 0px; background:none;"
|style="width:400px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">[[Challenges | DREAM4 Challenges Are Posted]]</div>
|}

|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="padding:.3em;">
* [{{link}}/register Register] to download the challenges
* 2 of 3 challenges have been posted (another will be posted soon)
* Ask questions on the [{{link}}/discuss discussion board]
* The [[DREAM4conf | DREAM4 Conference]] page has preliminary information about the next meeting
</div>
|}


== DREAM4 Challenges ==

These pages will be renamed when we go public

* [[xD4c1full]] DREAM4, Challenge 1: Peptide Recognition Domain (PRD) Specificity Prediction
* [[xD4c2full]] DREAM4, Challenge 2: Predictive Signaling Network Modeling
* [[xD4c3full]] DREAM4, Challenge 3: DREAM4 In Silico Gene Regulation Network Inference

DREAM4Conf

Summary:

* Regulatory Genomics: Orly Alter, Nitin S Baliga, Panayiotis (Takis) Benos,
Mathieu Blanchette, Michael R. Brent, Albert Erives, Eleazar Eskin,
Ernest Fraenkel, Nir Friedman, Mikhail Gelfand, Sridhar Hannenhalli,
Tim Hughes, Uri Keich, Christina Leslie, Hao Li, Adam Margolin, Uwe Ohler,
Mireille regnier, Eran Segal, Ron Shamir, Saurabh Sinha, Mona Singh,
Christopher Workman

* Systems Biology: M. Madan Babu, Joel Bader, Diego di Bernardo, Jim Collins,
Joaquin Dopazo, Eleazar Eskin, Igor Jurisica, Pascal Kahlem, Andre Levchenko,
Avi Ma'ayan, Adam Margolin, Satoru Miyano, Theodore Perkins, Timothy Ravasi,
Frederick Roth, Roded Sharan, Mona Singh, Pavel Sumazin, Ioannis Xenarios,

DREAM4conf

Summary:

__NOTOC__
= DREAM4 Conference =

This year, the DREAM conference is partnering with two related events in systems and regulatory biology in what promises to be a very exciting joint meeting. Our partners are the 6th Annual RECOMB Satellite on Regulatory Genomics and the 5th Annual RECOMB Satellite on Systems Biology. The joint conference on systems and regulatory biology will be held at MIT, in the Broad Institute, on Dec 2-Dec 6, 2009, between Wed at 5pm and Sun at 1pm.

For information on the joint conference, including invited speakers, venue, hotels, etc., please visit the site for the [http://compbio.mit.edu/recombsat/ joint conferences of RECOMB Systems Biology/Regulatory Genomics/DREAM4]

As in the previous DREAM conferences, DREAM4 will feature posters and oral presentations of papers accepted by our Program Committee.
Besides this usual conference format, the DREAM4 Conference will feature the discussion of predictions of a set of four DREAM4 challenges. Each challenge is composed of one or more datasets which originated from a biological system or ''in-silico'' representation of a biological system that have some aspects unknown to the challenge participants. The predictors are invited to infer as best as possible the unknown data or underlying network associated with the challenge. The actual measurements and networks along with the results of the predictions will be disclosed at the conference.

== Conference Dates And Venue==

<div> Dec 2 - Dec 6, 2009 </div>
<div> The Broad Institute of MIT and Harvard</div>
<div> Cambridge, MA </div>
<div>The conference will be at the corner of Main Street and Vassar St, the new biotech corner of MIT, at the Broad Institute Auditorium. </div>

== Conference Registration==

<div> Registration to the DREAM4 conference and RECOMB Regulatory Genomics and Systems Biology is now open at [http://www.regonline.com/recombsat09 regonline.com/recombsat09]. Participants are encouraged to register for the entire meeting, but also have the option to attend any one or two of the meetings at a reduced registration. The early registration deadline is Oct 1st, 2009.

</div>

== Conference Program ==

<div> The Program of the joint RECOMB Regulatory Genomics, RECOMB Systems Biology and DREAM4 features an exciting group of Invited Speakers, the presentation of papers accepted by our Program Committee, poster sessions and the presentation of the best network predictions from our DREAM4 data challenges.</div>

== Timeline for Oral Presentations, Papers and Posters==
For information on submission, formatting, partner journals, etc., please go the site for the [http://compbio.mit.edu/recombsat/ joint conferences of RECOMB Systems Biology/Regulatory Genomics/DREAM4].

* Paper submission deadline (Reg/Sys/DREAM): Fri July 31, 2009 (CLOSED)
* Abstract submission deadline: Oct 1st, 2009
* Notice of conf. acceptance/journal assignment by: Sep 15, 2009
* Final notification of journal acceptance: Oct 15, 2009
* Notification of Talk/Poster Selection: Oct 15, 2009
* Papers published in Partner Journals: Dec 1st, 2009

== Call for Predictions and Network Inference from Data== '
To download the data to participate in the challenge or to upload your predictions follow the links [{{link}}/index.php/The_DREAM_Project here]

* DREAM challenge response deadline: Oct 15, 2009
* Notification of challenge performance: Nov 15, 2009 (at the latest)

== Invited Speakers ==
===Regulatory Genomics: Confirmed Speakers===

* Naama Barkai (Weizmann Institute), Mark Biggin (Berkeley LBNL JGI), Bob Waterston (U Washington), Kevin White (U Chicago), Rick Young (Whitehead Institute)

===Systems Biology and DREAM4*: Confirmed Speakers===
* Walter Fontana (Harvard University), Nevan Krogan (UCSF), Ihor Lemischka (Mount Sinai), Franziska Michor (Sloan Kettering), Gary Nolan* (Stanford), Michael Yaffe* (MIT)

== Program Committees ==

===Regulatory Genomics===
* Orly Alter, Nitin S Baliga, Panayiotis (Takis) Benos, Mathieu Blanchette, Michael R. Brent, Albert Erives, Eleazar Eskin, Ernest Fraenkel, Nir Friedman, Mikhail Gelfand, Sridhar Hannenhalli, Tim Hughes, Uri Keich, Christina Leslie, Hao Li,Adam Margolin, Uwe Ohler, Aviv Regev, Mireille Regnier, Eran Segal, Ron Shamir, Saurabh Sinha, Mona Singh, Christopher Workman.

===Systems Biology and DREAM4===
* M. Madan Babu, Gary Bader, Joel Bader, Diego di Bernardo, Jim Collins, Joaquin Dopazo, Eleazar Eskin, Igor Jurisica, Pascal Kahlem, Andre Levchenko, Avi Ma'ayan, Adam Margolin, Satoru Miyano, Dana Pe'er, Theodore Perkins, Timothy Ravasi, Frederick Roth, Michael S Samoilov, Roded Sharan, Mona Singh, Pavel Sumazin, Denis Thieffry, Ioannis Xenarios.

== Information on Lodging==
<div> Several local hotels have offered us conference rates:

* Marriott Cambridge, 6 Cambridge Center, Cambridge, MA, (617) 494-1885.
* Le Meridien (formerly Hotel@MIT), 20 Sidney St, Cambridge MA, (866) 716-8119.
* Doubletree Club Boston, Bayside, on the T red line, 240 Mount Vernon St, Boston MA, (617) 583-1131.
* Best Western Terrace Inn, on the T green line, 1650 Commonwealth Ave, Boston MA, (617) 566-6260.
* Holiday Inn Boston-Somerville, 30 Washington Street, Somerville, MA, (617) 628-1000, van or taxi, 617-616-1970.

To find other hotels in the area, visit: http://maps.google.com/maps?q=hotel&near=02139

== DREAM4 Sponsors ==
<div>Columbia University Center for Multi scale Analysis Genomic and Cellular Networks (MAGNet)</div>
<div>NIH Roadmap Initiative</div>
<div>IBM Computational Biology Center</div>

{| style="width:100%; background:#fff; text-align:center; margin-top:+.9em; border:1px solid #ccc;"
|
<div style="font-size:150%; border:20px; margin:1; color:#000;"> [http://www.regonline.com/recombsat09 Conference Registration] </div>
|}

News

Summary:

== DREAM in the news ==
*In [http://www.bio-itworld.com/archive/silicobio/index_11132006.html BioIT World | Systems Biology ].

*In [http://www.bio-itworld.com/newsitems/2007/july/07-30-07-dream2 BioIT World.com ].

*In [http://www.genengnews.com/news/bnitem.aspx?name=21826072 GEN news].

*[http://www.bio-itworld.com/issues/2008/july-august/russell-transcript.html <font color="ff0000">July 14, 2008</font>: In BioIT-World.com]

*[http://www.bio-itworld.com/full_newsletter.aspx?id=78714 <font color="ff0000">July 16, 2008</font>: In John Russell's Systems Biology newsletter]

*[http://www.genengnews.com/articles/chitem.aspx?aid=2630 <font color="ff0000">October 15, 2008</font>: "Deciphering Biological Networks", in Genetics Engineering and Biotechnology News]

*[http://www.bio-itworld.com/pb/2008/10/23/dream3-results.html <font color="ff0000">October 23, 2008</font>: "DREAM3 Predictions (and Their Grades) Are In", In Predictive Medicine newsletter]

*[http://www.bio-itworld.com/pb/2008/12/04/interpreting-dream3.html <font color="ff0000">December 4, 2008</font>: " Interpreting DREAM3", In Bio-ITWorld.com]

*[http://www.bio-itworld.com/pb/2009/04/16/dream3-update.html <font color="ff0000">April 16, 2009</font>: "DREAM Project Vision Expands", In Bio-IT World’s Predictive Biomedicine newsletter]

About

Summary:

DREAM is a Dialogue for Reverse Engineering Assessments and Methods. The main objective is to catalyze the interaction between experiment and theory in the area of cellular network inference. The fundamental question for DREAM is simple: How can researchers assess how well they are describing the networks of interacting molecules that underlie biological systems? The answer is not so simple. Researchers have used a variety of algorithms to deduce the structure of very different biological and artificial networks, and evaluated their success using various metrics. What is still needed, and what DREAM aims to achieve, is a fair comparison of the strengths and weaknesses of the methods and a clear sense of the reliability of the network models they produce.

DREAM2conf

Summary:

__NOTOC__
= DREAM2 Conference =

The [[Media:Program.pdf| program]] features an exciting group of [http://wiki.c2b2.columbia.edu/dream/index.php/2nd_DREAM_Conference#Invited_Speakers Invited Speakers] and the presentation of papers accepted by our [http://wiki.c2b2.columbia.edu/dream/index.php/2nd_DREAM_Conference#Program_Committee Program Committee].
Besides this usual conference format, the second DREAM Conference will feature the discussion of predictions of a set of five DREAM challenges. Each challenge is composed of one or more dataset which originated from a network unknown to the participants to the challenge. The predictors are invited to infer as best as possible the network from which the data originated. The actual networks, and the results of the predictions will be disclosed at the conference.

== <font style="color:#f00"> Give us Feedback </font>==
{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:30%; color:#000"| 
{| style="width:400px; border:solid 0px; background:none;"
|style="width:800px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:0.5em; color:#2f0;">Provide feedback on the</div>
<div style="font-size:150%; border:none; margin:1; padding:0.5em; color:#2f0;">DREAM2 conference </div>
<div style="font-size:150%; border:none; margin:1; padding:0.5em; color:#2f0;">by adding a comment in </div>
<div style="font-size:150%; border:none; margin:1; padding:0.5em; color:#2f0;">the DREAM
[http://wiki.c2b2.columbia.edu/dream/discuss Discussion Forum] </div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="font-size:100%; padding:.3em;"> This forum is intended for discussion of anything related to Reverse Engineering in biological systems. Please feel free to comment on recent papers, interesting conferences or anything else related to pathway inference.
</div>
|}

== <font style="color:#f00"> Explore the results of the DREAM challenges </font>==
{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:30%; color:#000"| 
{| style="width:500px; border:solid 0px; background:none;"
|style="width:400px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">See [http://wiki.c2b2.columbia.edu/dream/results Results and scores] of the </div>
<div style="font-size:150%; border:none; margin:1; padding:.5em .5em 0 .5em; color:#000;">DREAM2 Network Inference Challenge</div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="padding:.3em;"> </div>
*Results are described for each challenge and sub-challenge.
*Team names have been anonymized in the results
*The best performers for each challenge will be announced
*Precision at different recall values are listed for each prediction
*Area under the precision-recall and ROC are listed for each prediction
|}

== Conference Dates And Venue==

<div> December 3 and 4, 2007 </div>
<div> [http://www.nyas.org/ New York Academy of Sciences]</div>
<div> New York, NY </div>

== Conference Registration==

<div> [http://www.nyas.org/events/eventDetail.asp?eventID=10232&date=12/3/2007 Registration] to the DREAM2 conference is open. Registration is processed by the NY Academy of Sciences. </div>

== Conference [[Media:Program.pdf| Program]] ==

<div> The [[Media:Program.pdf| program]] of the 2nd DREAM COnference features an exciting group of [http://wiki.c2b2.columbia.edu/dream/index.php/2nd_DREAM_Conference#Invited_Speakers Invited Speakers], the presentation of papers accepted by our [http://wiki.c2b2.columbia.edu/dream/index.php/2nd_DREAM_Conference#Program_Committee Program Committee], and the presentation of the best network predictions from our [[DREAM2 Challenges|data challenges]].</div>

== Information on Lodging==
<div> Hotel rates in Manhattan are extremely variable, going from expensive to very expensive. To get an idea of the range in prices, please take a look at the following [[Media:Hotel Lists.xls|list of hotels and rates]]. Note that the list contains some hotels in Brooklyn and some in Jersey City (the latter just one train station accross the the Hudson River away from the NYAS). We are providing this list only as a reference. There may be many other more convenient and less expensive hotels not included in this list. Unfortunately we couldn't find hotels that give special conference rates. If you need to know how to get from your hotel to the DREAM2 conference at the NYAS you can use the [http://tripplanner.mta.info New York City Transit Trip Planner], or the [http://www.newyorkontap.com/subways.asp New York City subway map].

== Call for Oral Presentations, Papers and Posters==
{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:40%; color:#000"| 
{| style="width:400px; border:solid 0px; background:none;"
|style="width:400px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">We are requesting papers and posters</div>
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">for the DREAM2 conference</div>
<div style="font-size:150%; border:none; margin:1 1 0 1; padding:.5em .5em 0 .5em; color:#000;">[http://www.easychair.org/DREAM2/ Submit Papers]</div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="padding:.3em;"> </div>
*Download the [[Media:dream2-submission_instructions.doc|instructions for paper submissions]].
*Papers and posters [http://www.easychair.org/DREAM2/ submission site ]
*Papers will be reviewed by members of the [[2nd_DREAM_Conference#Program_Committee | Program Committee]]
*A subset of papers papers will be fast-tracked to Molecular Systems Biology
*Accepted papers will be published in the Annals of the NYAS

|}

== Call for Predictions of Networks from Data==
{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:30%; color:#000"| 
{| style="width:500px; border:solid 0px; background:none;"
|style="width:400px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">The DREAM2 Network Inference Challenge</div>
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">[[DREAM2 Challenges|Download Challenges]]</div>
<div style="font-size:150%; border:none; margin:1 1 0 1; padding:.5em .5em 0 .5em; color:#000;">[http://wiki.c2b2.columbia.edu/dream/data Upload Predictions]</div>
<small>( PREDICTION UPLOADING AVAILABLE THROUGHOUT OCT 22 )</small>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="padding:.3em;"> </div>
*A list of proteins some of which interact
*A synthetic biology circuit transfected to an in-vivo organism
*The targets of a TF, to be inferred from gene expression data
*The GRN of a model organism to be inferred from gene expression data
*An in-silico generated network with simulated "gene expression data"
|}

== Important Dates==
=== Regular Paper Submissions ===

{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:10%; color:#000"| 
{| style="width:400px; border:solid 0px; background:none;"
|style="width:800px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:0.5em; color:#2f0;">Submission Deadline Extension</div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="font-size:100%; padding:.3em;"> The new deadline for paper and poster submission is Monday October 8th.
</div>
|}

{| style="color;black;background-color:#eeeeee;" border="3"
|Week of July 23:
|First call for papers and poster abstracts.
|-
|Oct 1:
|Papers and poster abstract <font style="color:#f00"> '''OLD''' </font> submission deadline.
|-
|Oct 8:
|Papers and poster abstract <font style="color:#f00"> '''NEW''' </font> submission deadline.
|-
|Nov 1:
|Paper acceptance decisions issued. Early registration deadline.
|-
|Dec 3-4:
|Conference dates.
|-
|Jan 18, 2008:
|Revised papers due to Molecular Systems Biology. If the authors participated in the challenge, the revisions may include the results of the challenge.
|-
|Feb 1, 2008:
|Final decision for publication in Molecular Systems Biology.
|-
|Feb 1, 2008:
|Revised papers due for those papers invited to publish in the Proceedings of the DREAM2 (Annals of the NYAS). If the authors participated in the challenge, the revisions may include a section with the results of the challenge.
|-
|}

=== Predictions Submissions ===

{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:10%; color:#000"| 
{| style="width:400px; border:solid 0px; background:none;"
|style="width:800px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:0.5em; color:#2f0;">Challenge Submission Deadline Extension</div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="font-size:100%; padding:.3em;"> The new deadline for submissions of challenge predictions is Monday October 22nd.
</div>
|}

{|style="color;black;background-color:#eeeeee;" border="3"
|Week of July 23:
|Call for participation in the challenge. Challenges posted.
|-
|Oct 15:
|<font style="color:#f00"> '''OLD''' </font> Submission deadline.
|-
|Oct 22:
|<font style="color:#f00"> '''NEW''' </font> Submission deadline.
|-

|Nov 1:
|Deadline for early registration to conference.
|-
|Nov 15:
|Notifications to predictors of their scores and ranks.
|-
|Dec 3-4:
|Conference Dates.
|-
|Feb 1, 2008:
|Deadline for submission of best predictions as papers to be included as contributions in the Proceedings of the 2nd DREAM Conference (Annals of the NYAS.)
|-
|}

== Invited Speakers ==
Andrea Califano.
Tim Gardner,
Ravi Iyengar,
Fritz Roth,
Chris Sander,
Ron Shamir,
Ilya Shmulevich,
Mike Snyder,
Peter Sorger,
Dennis Vitkup,
Ioannis Xenarios.

== Program Committee ==
Andre Levchenko,
Avi Ma'ayan,
Brandilyn Stigler,
Christina Leslie,
Eduardo Sontag,
Fritz Roth,
Harel Weinstein,
Igor Jurisica,
Ilya Shmulevich,
Ioannis Xenarios,
Ivet Bahar,
Jean-Loup M. Faulon,
Jesper Tegner,
Jim Collins,
Kathleen Marchal,
Luis Mendoza,
Madan Babu Mohan,
Mark Gerstein,
Monah Singh,
Rune Linding,
Satoru Miyano,
Seungchan Kim,
T Ravasi,
Ted Perkins,
Tim Elston,
Vesteinn Thorsson,
Winfried Just,
Yannis Kevrekidis,
Yuval Kluger,
Alex Hartemink,
Pablo Iglesias,
Adam Margolin,
Edward R. Dougherty,
Frank Doyle,
Gyan Bhanot,
Joaquin Dopazo,
Michael Samoilov,
Pascal Kahlem,
Mike Mackey,
Denis Thieffry,
John Moult,
Jeremy Rice,
John Wagner,
Jose Vilar,
Gary Bader,
Ilya Nemenman,
Leon Glass,
Yuhai Tu.

== DREAM2 Sponsors ==
<div>Columbia University Center for Multi scale Analysis Genomic and Cellular Networks (MAGNet)</div>
<div>NIH Roadmap Initiative</div>
<div>IBM Computational Biology Center</div>
<div>The New York Academy of Sciences</div>
<div>Merck Research Laboratories</div>

{| style="width:100%; background:#fff; text-align:center; margin-top:+.9em; border:1px solid #ccc;"
|
<div style="font-size:150%; border:20px; margin:1; color:#000;">[http://www.nyas.org/events/eventDetail.asp?eventID=10232&date=12/3/2007 Conference Registration ] </div>
|}

== Data for Reverse Engineering ==

One of the main thrusts of DREAM is to make available data to contribute to the Reverse Engineering efforts. There is no censensus as to what the best data modality is to infer cellular pathways from high throughput data. Therefore we expect that the novel uses of old data such as gene expression, as well as the use of newer data such as ChIP-chip, or phosphoproteomics, will be good drivers to the field. The objective of this site is to provide links to other sites with available datased aimed at reverse engineering cellular networks, as well as to provide data that came for our own DREAM challenges. As usual, you can add you comments and suggestions in the DREAM [http://wiki.c2b2.columbia.edu/dream/discuss Discussion Forum]

== Data and Gold Standards from the[[DREAM2:_The_2nd_DREAM_Conference | DREAM2 Conference]] network inference challenges ==

{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:40%; color:#000"| 
{| style="width:400px; border:solid 0px; background:none;"
|style="width:500px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#2f0;">Important Notice</div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="padding:.3em;"> </div>
Datasets of Challenges 1 and 3 cannot be used for external publication without explicit permission from the data owners. For the other datasets, please refer to the original data providers and the DREAM project in your publication.
|}

Step 1. [http://wiki.c2b2.columbia.edu/dream/register/index.php?dream=2 Register your team to download the data].
Once you have registered to receive the data, a password will be sent to the e-mail address you provided in the registration.

Step 2. Download the datasets. You will need to use the password provided in Step 1 above.
* Challenge 1: BCL6-Targets Challenges. [[BCL6 Transcriptional-Target Challenge. Description | Data Description]]. [http://wiki.c2b2.columbia.edu/dream/data Data Download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards Gold-standard Download].
* Challenge 2: The Protein-Protein Subnetwork Challenge. [[ The Protein-Protein Subnetwork Challenge. Description| Data Description]]. [http://wiki.c2b2.columbia.edu/dream/data Data download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards Gold-standard Download].
* Challenge 3: The Five-Gene-Network Challenges. [[The Five-Gene Network Challenge. Description | Data Description]].[http://wiki.c2b2.columbia.edu/dream/data Data Download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards Gold-standard Download].
* Challenge 4: The In-Silico-Network Challenges. [[The In-Silico-Network Challenges. Description | Data Description]]. [http://wiki.c2b2.columbia.edu/dream/data Data Download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards Gold-standard Download].
* Challenge 5: The Genome-Scale Network Challenge. [[The Genome-Scale Network Challenge. Description | Data Description]]. [http://wiki.c2b2.columbia.edu/dream/data Data download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards Gold-standard Download].

== Test your predictions for the DREAM2 Conference network inference challenges ==
As the Gold Standard for Challenge 1 (the BCL6 target challenge) has not yet been released, we created at web tool to facilitate the [http://wiki.c2b2.columbia.edu/dream/data/upload/roc testing of the predictions for Challenge 1]. However, to avoid the releasing the whole set of targets before the data owners could publish their data, we are only allowed to provide results for 5 predictions per team. (Better than nothing...)

You can use a web tool to easily [http://www.bioinformatica.crs4.org/dream-eval/ test your predictions] for Challenges 3, 4 and/or 5. (Courtesy of Alberto De la Fuente and his team at the [http://www.bioinformatica.crs4.org/ CRS4 Bioinformatics Research & Development Lab]. )

DREAM3conf

Summary:

= DREAM3 Conference =

This year, the DREAM conference is partnering with two related events in systems and regulatory biology in what promises to be a very exciting joint meeting. Our partners are the 5th Annual RECOMB Satellite on Regulatory Genomics and the 4th Annual RECOMB Satellite on Systems Biology. The joint conference on systems and regulatory biology will be held at MIT, in the Broad Institute, on Oct 29-Nov 2nd, 2008, between Wed at 5pm and Sun at 1pm.

For information on the joint conference, including invited speakers, venue, hotels, etc., please visit the site for the [http://compbio.mit.edu/recombsat/ joint conferences of RECOMB Systems Biology/Regulatory Genomics/DREAM3]

As in the previous DREAM conferences, DREAM3 will feature posters and oral presentations of papers accepted by our [http://wiki.c2b2.columbia.edu/dream/index.php/2nd_DREAM_Conference#Program_Committee Program Committee].
Besides this usual conference format, the DREAM3 Conference will feature the discussion of predictions of a set of four DREAM3 challenges. Each challenge is composed of one or more datasets which originated from a biological system or ''in-silico'' representation of a biological system that have some aspects unknown to the challenge participants. The predictors are invited to infer as best as possible the unknown data or underlying network associated with the challenge. The actual measurements and networks along with the results of the predictions will be disclosed at the conference.

== Give us Feedback ==
{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:10%; color:#000"| 
{| style="width:400px; border:solid 0px; background:none;"
|style="width:800px; text-align:center; white-space:nowrap; color:#000;"|
<div style="font-size:150%; border:none; margin:1; padding:0.5em; color:#2f0;">Provide feedback on the DREAM3 challenges </div>
<div style="font-size:150%; border:none; margin:1; padding:0.5em; color:#2f0;">by adding a comment in the DREAM
[http://wiki.c2b2.columbia.edu/dream/discuss Discussion Forum] </div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="font-size:100%; padding:.3em;"> This forum is intended for discussion of anything related to Reverse Engineering in biological systems. Please feel free to comment on recent papers, interesting conferences or anything else related to pathway inference.
</div>
|}

== Explore the results of the DREAM3 challenges ==
{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:30%; color:#000"| 
{| style="width:500px; border:solid 0px; background:none;"
|style="width:400px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">See [http://wiki.c2b2.columbia.edu/dream/results/DREAM3/ Results and scores] of the </div>
<div style="font-size:150%; border:none; margin:1; padding:.5em .5em 0 .5em; color:#000;">DREAM3 Network Inference Challenge</div>
|}
{| style="width:500px; border:solid 0px; background:none;"
|style="width:400px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">[[The_DREAM3_Challenges|Download Challenges]] and [http://wiki.c2b2.columbia.edu/dream09/data/gold-standards/DREAM3/ Gold-Standards]</div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="padding:.3em;"> </div>
*A Signaling-Cascade Network Inference and Ligand Identification Challenge (flow cytometry data)
*A Signaling-Pathway Response to Perturbation Prediction Challenge in Normal and Cancer Hepatocytes (phosphoproteomics data)
*A Prediction of Gene-Expression data in Yeast Strains Challenge (GeneChip data)
*A Network inference Challenge from in silico generated data (in Silico gene expression data)
|}

=== Important Dates===
{|style="color;black;background-color:#eeeeee;" border="3"
|Mid June:
|Call for participation in the DREAM3 challenges. Challenges posted.
|-
|Sep 15:
|Deadline for submissions of predictions for challenge.
|-
|Oct 15:
|Notifications to predictors of their scores and ranks.
|-
|Oct 31
|DREAM3 track of joint conferences date.
|-
|}

== Invited Speakers ==
===Regulatory Genomics===

Uri Alon (Weizmann Institute),
Chris Burge (MIT),
Mark Gerstein (Yale),
Tim Hughes (U. Toronto),
Daphne Koller (Stanford University),
Bing Ren (UC San Diego),
Gerry Rubin (Janelia Farms),
Eddy Rubin (Berkeley/JGI),
Thomas Tuschl (Rockefeller)

===Systems Biology and DREAM3===

Aviv Bergman (A. Einstein College),
David Botstein (Princeton),
George Church (MIT/Harvard),
Todd Golub (Broad Institute),
Boris Kholodenko (T.Jefferson U),
Doug Lauffenburger (MIT),
Leona Samson (MIT BE),
Pamela Silver (Harvard SysBio),
John Tyson (Virginia Tech)

== Program Committee ==

M. Madan Babu,
Gary Bader,
Andrea Califano,
Jim Collins,
Joaquin Dopazo,
Peicheng Du,
Jean-Loup Faulon,
Igor Jurisica,
Winfried Just,
Pascal Kahlem,
Manolis Kellis,
Yannis Kevrekidis,
Seungchan Kim,
Andre Levchenko,
Rune Linding,
Kathleen Marchal,
Adam Margolin,
Satoru Miyano,
Theodore Perkins,
Raul Rabadan,
Timothy Ravasi,
John Rice,
Michael Samoilov,
Ilya Shmulevich,
Eduardo Sontag,
Brandilyn Stigler,
Gustavo Stolovitzky,
Jesper Tegner,
Yuhai Tu,
Ioannis Xenarios

== Information on Lodging==
<div> Several local hotels have offered us conference rates:

* Marriott Cambridge, 6 Cambridge Center, Cambridge, MA, (617) 494-1885, $229.00
* Le Meridien (formerly Hotel@MIT), 20 Sidney St, Cambridge MA, (866) 716-8119, $209.00/night
* Doubletree Club Boston, Bayside, on the T red line, 240 Mount Vernon St, Boston MA, (617) 583-1131, $189.00/night
* Best Western Terrace Inn, on the T green line, 1650 Commonwealth Ave, Boston MA, (617) 566-6260, $189.00/ night
* Holiday Inn Boston-Somerville, 30 Washington Street, Somerville, MA, (617) 628-1000, van or taxi, 617-616-1970, $169.00

To find other hotels in the area, visit: http://maps.google.com/maps?q=hotel&near=02139

== DREAM3 Sponsors ==
<div>Columbia University Center for Multi scale Analysis Genomic and Cellular Networks (MAGNet)</div>
<div>NIH Roadmap Initiative</div>
<div>IBM Computational Biology Center</div>
<div>Merck Research Laboratories</div>

{| style="width:100%; background:#fff; text-align:center; margin-top:+.9em; border:1px solid #ccc;"
|
<div style="font-size:150%; border:20px; margin:1; color:#000;">[http://www.regonline.com/Checkin.asp?EventId=608438 Conference Registration ] </div>
|}

== A few rules==
*Each participant or team of participants will choose a "Team Name".
*The Team Name cannot have blank spaces or special characters.
*Each team will be able to submit only one prediction per category.
*In order to download the data the participant has to [http://wiki.c2b2.columbia.edu/dream/register/index.php?dream=3 register for the challenge]. This registration is independent of the registration for the DREAM3 conference.

== Data Download ==

{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:40%; color:#000"| 
{| style="width:400px; border:solid 0px; background:none;"
|style="width:500px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:250%; border:none; margin:1; padding:.5em; color:#2f0;">Important Notice</div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="padding:.3em;"> </div>
These datasets of Challenges 1-3 cannot be used for external publication without explicit permission from the data owners.
|}

Step 1. [http://wiki.c2b2.columbia.edu/dream/register/index.php?dream=3 Register your team for the challenge].
Once you have registered your Team for the challenge, a password will be sent to the e-mail address you provided in the registration.

Step 2. Download the datasets. You will need to use the password provided in Step 1 above.
* Challenge 1: The Signaling-Cascade Challenges. [[The Signalling-Cascade Challenges. Description | Data Description]]. [http://wiki.c2b2.columbia.edu/dream/data/DREAM3 Data Download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold-Standards].
* Challenge 2: The Signaling-Response Prediction Challenges. [[ The Signaling-Response Prediction Challenge. Description| Data Description]]. [http://wiki.c2b2.columbia.edu/dream/data/DREAM3/ Data download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold-Standards].
* Challenge 3: The Gene-Expression Prediction Challenge. [[The Gene-Expression Prediction Challenge. Description | Data Description]].[http://wiki.c2b2.columbia.edu/dream/data/DREAM3 Data Download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold-Standards].
* Challenge 4: The DREAM3 In-Silico-Network Challenges. [[The DREAM3 In-Silico-Network Challenges. Description | Data Description]]. [http://wiki.c2b2.columbia.edu/dream/data/DREAM3 Data Download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold-Standards].

D2c5full

Summary:

= Genome-Scale Network Inference (DREAM2, Challenge 5) =

<div style="background-color:#FFFF99;width:100%">
This archival page describes the challenge exactly as it was presented to the participants. Go to the main [[D2c5|DREAM2 Challenge 5]] page to download data, view team rankings, cite this work, etc.
</div>

== Synopsis ==

A panel of single-channel microarrays was collected for a particular microorganism, including some already published and some in-print data. The data was appropriately normalized (to the logarithmic scale). The challenge consists of reconstructing a genome-scale transcriptional network for this organism. The accuracy of network inference will be judged using chromatin precipitation and otherwise experimentally verified Transcription Factor (TF)-target interactions.

== Dataset ==

This challenge dataset consists of two files. One file, '''data.csv''' contains the experimental data, and the other, '''tfs.csv''', lists the transcription factors.

*'''data.csv''' This file contains a 3456 genes x 300 experiments dataset. The names of both genes and experiments have been withheld, and operon information is not provided. As described above, the experiments represent both published and not-yet-released data from a variety of sources. The 3456 genes include all known and putative transcription factors and all genes whose interactions will be used for testing, as well as a number of other recognized coding sequences. This file is comma-separated and is easily imported into Excel or any other program.

*'''tfs.csv''' This file contains the indices of rows belonging to transcription factors in the matrix from '''data.csv''', one per line.

== Submission Information ==

Submit one network prediction in one or more of the following categories: DIRECTED-UNSIGNED, DIRECTED-SIGNED. Use the 3 tab-separated column format as in the example below:

:row#1 \tab row#2 \tab XYZ

where row#1 is the index of the row of a transcription factor and row#2 is the index of the row of one of its putative targets in the same order as the rows in the file '''data.csv'''. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a the TF at row#1 regulates the gene at row#2. The value of XYZ will be different for UNSIGNED, SIGNED-EXCITATORY and SIGNED-INHIBITORY, and will be discussed below.

Note that row#1 has to be one of the values of the file '''tfs.cvs'''. Interactions for which row#1 does not correspond to a transcription factor will not be judged for scoring.

Only DIRECTED networks will be accepted. If the transcription factor at row#1 regulates the transcription factor at row#2, and also the transcription factor at row#2 regulates the transcription factor at row#1, then both lines should be included. Participants whose algorithms produce UNDIRECTED networks can submit their predicitons provided they directionalize their UNDIRECTED network into a DIRECTED one. To do that, simply assume that only transcription factors may be regulators. This will make all edges directed, except for TF1 --> TF2 edges. These edges must then be submitted twice as: TF1 --> TF2, and TF2 --> TF1, in effect as if predicting a feedback loop. Naturally, this strategy will produce no false positive if the feedback loop is correctly predicted, one false positive even when one edge is correctly predicted, and two false positives when it's not.

'''''For UNSIGNED submissions:'''''

:XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a TF regulates a target gene, regardless of the sign of the regulation. (E.g., XYZ = 1 if the pair TF-gene is deemed to be connected with highest confidence, and XYZ = 0 if the pair is deemed not to interact.) Order your predictions in decreasing order of XYZ values, i.e., from the most reliable prediciton (highest XYZ value) in the first row and the least reliable prediction (lowest XYZ value) in the last row. Save the file as text, and name it:

::'''TeamName_UNSIGNED_GenomeScale.txt'''

:where TeamName is the name of the team with which you registered for the challenge.

'''''For SIGNED submissions:'''''

:Submit one network predictions for excitatory connections and one for inhibitory connections.

:'''''For EXCITATORY connections:'''''

::XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a TF upregulates a target gene. (E.g., XYZ = 1 if the TF is deemed to upregulate the target gene with the highest confidence, and XYZ = 0 if the pair is deemed to be disconnected, or the TF is deemed to downregulate the target gene.) Order your predictions in decreasing order of XYZ values, i.e., from the most reliable prediction (highest XYZ value) in the first row, and the least reliable prediction (lowest XYZ value) in the last row. Save the file as text, and name it:

:::'''TeamName_SIGNED_EXCITATORY_GenomeScale.txt'''

::where TeamName is the name of the team with which you registered for the challenge

:'''''For INHIBITORY connections:'''''

::XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a TF downregulates a target gene. (E.g., XYZ = 1 if the TF is deemed to downregulate the target gene with the highest confidence, and XYZ = 0 if the pair is deemed to be disconnected, or the TF is deemed to upregulate the target gene.) Order your predictions in decreasing order of XYZ values, i.e., from the most reliable prediction (highest XYZ value) in the first row and the least reliable prediction (lowest XYZ value) in the last row. Save the file as text, and name it:

:::'''TeamName_SIGNED_INHIBITORY_GenomeScale.txt'''

::where TeamName is the name of the team with which you registered for the challenge.

== Scoring metrics ==

We will score the results using the area under the precision versus recall curve for the whole set of predicitons. No threshold need be applied to your predicitons, since even low precisions at increasing recall will contribute to the final score. All pairs omitted from the list in your prediction files will be considered to appear randomly ordered at the end of the list with XYZ = 0. For the first ''k'' predictions (ranked by connectivity score, and for predictions with the same score, taken in the order they were submitted in the prediction files), precision is defined as the fraction of correct predictions to ''k'', and recall is the proportion of correct predictions out of all the possible true connections (with the approperiate sign, if the category is SIGNED). Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated.

D2c5

Summary:

__NOTOC__
= Genome-Scale Network Inference (DREAM2, Challenge 5) =

== Synopsis ==

A panel of single-channel microarrays was collected for a particular microorganism, including some already published and some in-print data. The data was appropriately normalized (to the logarithmic scale). The challenge consists of reconstructing a genome-scale transcriptional network for this organism. The accuracy of network inference will be judged using chromatin precipitation and otherwise experimentally verified Transcription Factor (TF)-target interactions.
== About the Data ==

* Data generously provided by [http://gardnerlab.bu.edu/ Tim Gardner], Boston University
* '''Reference:''' Faith JJ, Hayete B, Thaden JT, Mogno I, Wierzbowski J, Cottarel G, Kasif S, Collins JJ, Gardner TS. Large-scale mapping and validation of Escherichia coli transcriptional regulation from a compendium of expression profiles. PLoS Biol. 2007 Jan;5(1). [http://www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed&cmd=search&term=17214507 PubMed]
* [http://m3d.bu.edu/ Many Microbe Microarrays Database (M3D)]

== Best Performer ==

* Team 1: Dimitris Anastassiou, John Watkinson, Kuo-ching Liang, Xiadong Wang, and Tian Zheng, Columbia University.

== Take Action ==

* [[d2c5full|Full Challenge Description]] (archival)
* [{{link}}/results/DREAM2/?c=5 Team Rankings] (results)
* [{{link}}/data/DREAM2/ Download Training Data]
* [{{link}}/data/gold-standards/DREAM2/ Download Gold Standard]
* [{{link}}/data/scripts/DREAM2 Download Evaluation Scripts]
* Download Team Predictions (anonymous)

D2c4full

Summary:

__NOTOC__
= DREAM2 In Silico Network Challenge (DREAM2, Challenge 4) =

<div style="background-color:#FFFF99;width:100%">
This archival page describes the challenge exactly as it was presented to the participants. Go to the main [[D2c4|DREAM2 Challenge 4]] page to download data, view team rankings, cite this work, etc.
</div>

== Synopsis ==

Three in-silico networks were created and endowed with a dynamics that simulate biological interactions. The challenge consists of predicting the connectivity and some of the properties of one or more of these three networks.

== Datasets ==

Three datasets, named InSilico1, InSilico2 and InSilico3, were generated using simulations of biological interactions, as described below. The data from InSilico1 and InSilico2 correspond to mRNA levels of gene networks with qualitatively different topologies. InSilico3 corresponds to a full biochemical network, including metabolites, proteins and mRNA concentrations.

----

=== Dataset InSilico1 ===

This datasets were produced from a gene network with 50 genes, where the rate of synthesis of the mRNA of each gene is affected by the level of mRNA of other genes.

*'''InSilico1-heterozygous.xls''' contains steady state levels for the wild-type and 50 heterozygous knock-down strains for each gene (+/-). Values of gene expression are provided for a standard condition (steady states).

*'''InSilico1-null-mutants.xls''' contains steady state levels for the wild-type and 50 null mutant strains for each gene (-/-). Values of gene expression are provided for a standard condition (steady states).

*'''InSilico1-trajectories.xls''' contains time courses (trajectories) of the network recovering from several external perturbations. There are 23 different perturbations and 26 time points for each one.

----

=== Dataset InSilico2 ===

The structure of the dataset and submission information for InSilico2 are similar to those of the InSilico1 dataset.
However, the topology of the InSilico2 network is qualitatively different from the topology of the InSilico1 network. The InSilico2 datasets were produced from a gene network with 50 genes, where the rate of synthesis of the mRNA of each gene is affected by the level of mRNA of other genes.

*'''InSilico2-heterozygous.xls''' contains steady state levels for the wild-type and 50 heterozygous knock-down strains for each gene (+/-). Values of gene expression are provided for a standard condition (steady states).

*'''InSilico2-null-mutants.xls''' contains steady state levels for the wild-type and 50 null mutant strains for each gene (-/-). Values of gene expression are provided for a standard condition (steady states).

*'''InSilico2-trajectories.xls''' contains time courses (trajectories) of the network recovering from several external perturbations. There are 23 different perturbations and 26 time points for each one.

----

=== Dataset InSilico3 ===

The InSilico3 dataset was produced from a full in-silico biochemical network, that includes 24 metabolites, 23 proteins and 20 genes. The network has transcription, translation, some signaling, and metabolism. Variables are named Mxx for metabolites, Pyy for proteins (more specifically protein forms), and Gzz for mRNA (where xx, yy and zz are numbers between 1 and 24, 23 adn 20 respectively).

*'''InSilico3-heterozygous.xls''' contains steady state levels of metabolites, proteins and mRNA for the wild-type and the 20 heterozygous knock-down strains for each gene (+/-). Values are provided for a standard condition (steady states).

*'''InSilico3-null-mutants.xls''' contains steady state levels of metabolites, proteins and mRNA for the wild-type and the 20 null mutant strains for each gene (-/-). Values are provided for a standard condition (steady states). NOTE: the knockout of G14 was "lethal".

*'''InSilico3-trajectories.xls''' contains time courses (trajectories) of the network recovering from several external perturbations. There are 22 different perturbations and 26 time points for each one.

'''Notation''': Besides binary reactions, dataset InSilico3 contains biochemical reactions that involve 3 or more molecular species. In order to represent these reactions as the usual networks, we have to agree on a notation to represent these reactions in terms of binary interactions. If we indicate the interaction A represses B, or A consumes B with A ---| B, and the interaction A activates B as A ==> B then we will write the three species reaction x ---> y, catalyzed by enzyme E as the following set of binary interactions

<div> x ==> y</div>
<div>E ==> y</div>
<div>E ---| x</div>

If the reaction between x and y were reversible, x <--> y, then the interaction
<div>y ==> x</div>

is also present. Similarly, the notation for a reaction of the form: w + x ---> y + z catalyzed by enzyme E should be

<div>w ==> y</div>
<div>x ==> y</div>
<div>w ==> z</div>
<div>x ==> z</div>
<div>E ---| w</div>
<div>E ---| x</div>
<div>E ==> y</div>
<div>E ==> z</div>
<div>w ---| x</div>
<div>x ---| w</div>

for the forward reaction. If it the reverse reaction is also present in the system, then the following lines should be added

<div>y ==> w </div>
<div>y ==> x </div>
<div>z ==> w </div>
<div>z ==> x </div>
<div>y ---| z </div>
<div>z ---| y </div>

----

== Submission Information ==

Predictions for datasets InSilico1, inSilico2 and InSilico3 can be submitted in one or more of the following categories: UNDIRECTED-UNSIGNED, UNDIRECTED-SIGNED, DIRECTED-UNSIGNED, DIRECTED-SIGNED.

'''''For UNSIGNED submissions:'''''

:Submit network predictions the corresponding dataset in one or both of the following categories: UNDIRECTED-UNSIGNED, DIRECTED-UNSIGNED. Submit a ranked list of pairs of molecular species ordered according to the confidence you assign to your prediction that a pair is connected, from the most reliable (first row) to the least reliable (last row) prediction. Use a 3 tab-separated column format as in the example below:

::A \tab B \tab XYZ

:where A and B are mRNA species in datasets InSilico1 and InSilico2, but can independently be a metabolite, a protein or an mRNA in dataset InSilico3.

:If the category is DIRECTED, the molecular species in the first column causes a change in the molecular species in the second column. (If both A regulates B and B regulates A, then both lines should be included.) If the category is UNDIRECTED, the order of the molecular species is irrelevant. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair is connected. (E.g., XYZ = 1 if the pair is deemed to be connected with highest confidence and XYZ = 0 if the pair is deemed not to interact.) All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

::'''TeamName_Category_Dataset.txt'''

:where TeamName is the name of the team with which you registered for the challenge, Category can be one of the following types of predictions: UNDIRECTED-UNSIGNED, DIRECTED-UNSIGNED, and Dataset can be InSilico1, InSilico2 or InSilico3.

'''''For SIGNED submissions:'''''

:Submit one network predictions for excitatory connections and one for inhibitory connections for dataset FiveGeneNet1 in one or both of the following categories: UNDIRECTED-SIGNED, DIRECTED-SIGNED.

:'''''For EXCITATORY connections:'''''

::Submit a ranked list of pairs of molecular species, ordered according to the confidence you assign to your prediction that a pair is connected with an excitatory connection, from the most reliable (first row) to the least reliable (last row) prediction. Use a 3 tab-separated column format as in the example below:

:::A \tab B \tab XYZ

::If the category is DIRECTED, the molecular species in the first column causes a change in the molecular species in the second column. (If both A regulates B and B regulates A, then both lines should be included.) If the category is UNDIRECTED, the order of the molecular species is irrelevant. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair is connected with an excitatory connection. (E.g., XYZ = 1 if one element of the pair is deemed to activate the other element with the highest confidence and XYZ = 0 if the pair is deemed to be disconnected, or deemed to interact with an inhibitory connection.) All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

:::'''TeamName_Category_EXCITATORY_Dataset.txt'''

::where TeamName is the name of the team with which you registered for the challenge, and category can be one of the following types of predictions: UNDIRECTED-SIGNED, DIRECTED-SIGNED, and Dataset can be InSilico1, InSilico2 or InSilico3.

:'''''For INHIBITORY connections:'''''

::Submit a ranked list of pairs of molecular species, ordered according to the confidence you assign to your prediction that a pair is connected with an inhibitory connection, from the most reliable (first row) to the least reliable (last row) prediction. Use a 3 tab-separated column format as in the example below:

:::A \tab B \tab XYZ

::If the category is DIRECTED, the molecular species in the first column causes a change in the molecular species in the second column. (If both A regulates B and B regulates A, then both lines should be included.) If the category is UNDIRECTED, the order of the molecular species is irrelevant. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair is connected with an inhibitory connection. (E.g., XYZ = 1 if one element of the pair is deemed to inhibit the other element with the highest confidence and XYZ = 0 if the pair is deemed to be disconnected, or deemed to interact with an excitatory connection.) All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

:::'''TeamName_Category_INHIBITORY_Dataset.txt'''

::where TeamName is the name of the team with which you registered for the challenge, and category can be one of the following types of predictions: UNDIRECTED-SIGNED, DIRECTED-SIGNED, and Dataset can be InSilico1, InSilico2 or InSilico3.

== Scoring Metrics ==

We will score the results using the area under the precision versus recall curve for the whole set of predicitons. For the first ''k'' predictions (ranked by score, and for predictions with the same score, taken in the order they were submitted in the prediction files), precision is defined as the fraction of correct predictions to ''k'', and recall is the proportion of correct predictions out of all the possible true connections (with the approperiate sign, if the category is SIGNED). Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated.

D2c4

Summary:

__NOTOC__
= DREAM2 In Silico Network Inference (DREAM2, Challenge 4) =

== Synopsis ==

Three in-silico networks were created and endowed with a dynamics that simulate biological interactions. The challenge consists of predicting the connectivity and some of the properties of one or more of these three networks.

== About the Data ==

* Data generously provided by [http://www.comp-sys-bio.org/tiki-index.php Pedro Mendes], Virginia Bioinformatics Institute, Virginia Tech
* Data cannot be used for external publication without explicit permission from the data provider
* Reference: TODO

== Best Performer ==

* Team 1: Alan Scheinine, Wieslawa Mentzen, G. Fotia, E. Pieroni, F. Maggio, G. Mancosu, and Alberto de la Fuente, CRS4 Bioinformatica, Italy
* Team 2 (and 2s): Mario Lauri, Francesco Iorio, and Diego di Bernardo, Telethon Institute of Genetics and Medicine, University of Naples Federico II, and University of Salerno, Italy.
* Team 3: Mika Gustafsson, Michael Hornquist, Jesper Lundstrom, Johan Bjorkegren, and Jesper Tegner, from Linkoping University and Karolinska Universitetssjukhuset, Sweden
* Team 4: Tejaswi Gowda, Sarma Vrudhul, and Seungchan Kim, from Arizona State University, Tempe, Arizona, and Translational Genomics Research Institute, Phoenix, Arizona.

== Take Action ==

* [[d2c4full|Full Challenge Description]] (archival)
* [{{link}}/results/DREAM2/?c=4 Team Rankings] (results)
* [{{link}}/data/DREAM2/ Download Training Data]
* [{{link}}/data/gold-standards/DREAM2/ Download Gold Standard]
* [{{link}}/data/scripts/DREAM2 Download Evaluation Scripts]
* Download Team Predictions (anonymous)

D2c3full

Summary:

= Synthetic Five-Gene Network Inference (DREAM2, Challenge 3) =

<div style="background-color:#FFFF99;width:100%">
This archival page describes the challenge exactly as it was presented to the participants. Go to the main [[D2c3|DREAM2 Challenge 3]] page to download data, view team rankings, cite this work, etc.
</div>

== Synopsis ==

A synthetic-biology network consisting of 5 interacting genes was created and transfected to an in-vivo model organism. The challenge consists of predicting the connectivity of the five-gene network from in-vivo measurements.

== Datasets ==

Two datasets were generated using qPCR (Dataset FiveGeneNet1) and gene expression arrays (Dataset FiveGeneNet2), from which the five gene network could in principle be inferred independently. The predictions from each of these datasets will be evaluated independently.

-----

=== FiveGeneNet1 ===

'''File name: FiveGene_qPCR.xls'''

'''Description:''' This dataset contains two time series for the 5 genes of the network. In each of the time series, the organism was treated with the same initial perturbation, and cell cultures were collected at different times from qPCR measurements. The two time series correspond to samples taken at regular intervals for 3 hr (time series qPCR_A) and for 5 hr (time series qPCR_B).

'''Submission Information:''' Predictions for dataset FiveGeneNet1 can be submitted in one or more of the following categories: UNDIRECTED-UNSIGNED, UNDIRECTED-SIGNED, DIRECTED-UNSIGNED, DIRECTED-SIGNED.

'''''For UNSIGNED submissions:'''''

:Submit network predictions for dataset FiveGeneNet1 in one or both of the following categories: UNDIRECTED-UNSIGNED, DIRECTED-UNSIGNED. Submit a ranked list of gene pairs, ordered according to the confidence you assign to your prediction that a pair is connected, from the most reliable (first row) to the least reliable (last row) prediction. Use the following 3 tab-separated column format as in the example below:

::gene_A \tab gene_B \tab XYZ

:If the category is DIRECTED, the gene in the first column regulates the gene in the second column. (If both gene_A regulates gene_B and gene_B regulates gene_A, then both lines should be included.) If the category is UNDIRECTED, the order of the genes is irrelevant. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair is connected. (E.g., XYZ = 1 if the pair is deemed to be connected with highest confidence and XYZ = 0 if the pair is deemed not to interact.) All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

::'''TeamName_Category_FiveGene_qPCR.txt'''

:where TeamName is the name of the team with which you registered for the challenge, and category can be one of the following types of predictions: UNDIRECTED-UNSIGNED, DIRECTED-UNSIGNED.

'''''For SIGNED submissions:'''''

:Submit one network predictions for excitatory connections and one for inhibitory connections for dataset FiveGeneNet1 in one or both of the following categories: UNDIRECTED-SIGNED, DIRECTED-SIGNED.

:'''''For EXCITATORY connections:'''''

::Submit a ranked list of gene pairs, ordered according to the confidence you assign to your prediction that a pair is connected with an axcitatory connection, from the most reliable (first row) to the least reliable (last row) prediction. Use the following 3 tab-separated column format as in the example below:

:::gene_A \tab gene_B \tab XYZ

::If the category is DIRECTED, the gene in the first column regulates the gene in the second column. (If both gene_A regulates gene_B and gene_B regulates gene_A, then both lines should be included.) If the category is UNDIRECTED, the order of the genes is irrelevant. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair is connected with excitatory connection. (E.g., XYZ = 1 if one element of the pair is deemed to activate the other element with the highest confidence and XYZ = 0 if the pair is deemed to be disconnected, or deemed to interact with an inhibitory connection.) All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

:::'''TeamName_Category_EXCITATORY_FiveGene_qPCR.txt'''

::where TeamName is the name of the team with which you registered for the challenge, and category can be one of the following types of predictions: UNDIRECTED-SIGNED, DIRECTED-SIGNED.

:'''''For INHIBITORY connections:'''''

::Submit a ranked list of gene pairs, ordered according to the confidence you assign to your prediction that a pair is connected with an inhibitory connection, from the most reliable (first row) to the least reliable (last row) prediction. Use the following 3 tab-separated column format as in the example below:

:::gene_A \tab gene_B \tab XYZ

::If the category is DIRECTED, the gene in the first column regulates the gene in the second column. (If both gene_A regulates gene_B and gene_B regulates gene_A, then both lines should be included.) If the category is UNDIRECTED, the order of the genes is irrelevant. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair is connected with an inhibitory connection. (E.g., XYZ = 1 if one element of the pair is deemed to inhibit the other element with the highest confidence and XYZ = 0 if the pair is deemed to be disconnected, or deemed to interact with an excitatory connection.) All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

:::'''TeamName_Category_INHIBITORY_FiveGene_qPCR.txt'''

::where TeamName is the name of the team with which you registered for the challenge, and category can be one of the following types of predictions: UNDIRECTED-SIGNED, DIRECTED-SIGNED.

'''Scoring Metrics:''' We will score the results using the area under the precision versus recall curve for the whole set of predicitons. For the first ''k'' predictions (ranked by score, and for predictions with the same score, taken in the order they were submitted in the prediction files), precision is defined as the fraction of correct predictions to ''k'', and recall is the proportion of correct predictions out of all the possible true connections (with the approperiate sign, if the category is SIGNED). Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated.

-----

=== Dataset FiveGeneNet2 ===

'''File name:''' '''FiveGene_chip.xls'''

'''Description:''' This dataset contains two time series corresponding to two different treatments. 588 genes from the original Affymetrix microarray data were selected, which include the 5 genes in the synthetic network plus genes known in the literature to be regulated by some of these 5 genes. The 5-gene network, which is a subnet of the bigger network, is oscillating with the cell cycle.

'''Submission Information:''' Predictions for dataset FiveGeneNet2 can be submitted in one or more of the following categories: UNDIRECTED-UNSIGNED, UNDIRECTED-SIGNED, DIRECTED-UNSIGNED, DIRECTED-SIGNED.

'''''For UNSIGNED submissions:'''''

:Submit network predictions for dataset FiveGeneNet2 in one or both of the following categories: UNDIRECTED-UNSIGNED, DIRECTED-UNSIGNED. Submit a ranked list of gene pairs, ordered according to the confidence you assign to your prediction that a pair is connected, from the most reliable (first row) to the least reliable (last row) prediction. Use the following 3 tab-separated column format as in the example below:

::gene_A \tab gene_B \tab XYZ

:If the category is DIRECTED, the gene in the first column regulates the gene in the second column. (If both gene_A regulates gene_B and gene_B regulates gene_A, then both lines should be included.) If the category is UNDIRECTED, the order of the genes is irrelevant. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair is connected. (E.g., XYZ = 1 if the pair is deemed to be connected with highest confidence and XYZ = 0 if the pair is deemed not to interact.) All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

::'''TeamName_Category_FiveGene_chip.txt'''

:where TeamName is the name of the team with which you registered for the challenge, and category can be one of the following types of predictions: UNDIRECTED-UNSIGNED, DIRECTED-UNSIGNED.

'''''For SIGNED submissions:'''''

:Submit one network predictions for excitatory connections and one for inhibitory connections for dataset FiveGeneNet2 in one or both of the following categories: UNDIRECTED-SIGNED, DIRECTED-SIGNED.

:'''''For EXCITATORY connections:'''''

::Submit a ranked list of gene pairs, ordered according to the confidence you assign to your prediction that a pair is connected with an excitatory connection, from the most reliable (first row) to the least reliable (last row) prediction. Use the following 3 tab-separated column format as in the example below:

:::gene_A \tab gene_B \tab XYZ

::If the category is DIRECTED, the gene in the first column regulates the gene in the second column. (If both gene_A regulates gene_B and gene_B regulates gene_A, then both lines should be included.) If the category is UNDIRECTED, the order of the genes is irrelevant. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair is connected with excitatory connection. (E.g., XYZ = 1 if one element of the pair is deemed to activate the other element with the highest confidence and XYZ = 0 if the pair is deemed to be disconnected, or deemed to interact with an inhibitory connection.) All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

:::'''TeamName_Category_EXCITATORY_FiveGene_chip.txt'''

::where TeamName is the name of the team with which you registered for the challenge, and category can be one of the following types of predictions: UNDIRECTED-SIGNED, DIRECTED-SIGNED.

:'''''For INHIBITORY connections:'''''

::Submit a ranked list of gene pairs, ordered according to the confidence you assign to your prediction that a pair is connected with an inhibitory connection, from the most reliable (first row) to the least reliable (last row) prediction. Use the following 3 tab-separated column format as in the example below:

:::gene_A \tab gene_B \tab XYZ

::If the category is DIRECTED, the gene in the first column regulates the gene in the second column. (If both gene_A regulates gene_B and gene_B regulates gene_A, then both lines should be included.) If the category is UNDIRECTED, the order of the genes is irrelevant. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair is connected with an inhibitory connection. (E.g., XYZ = 1 if one element of the pair is deemed to inhibit the other element with the highest confidence and XYZ = 0 if the pair is deemed to be disconnected, or deemed to interact with an excitatory connection.) All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

:::'''TeamName_Category_INHIBITORY_FiveGene_chip.txt'''

::where TeamName is the name of the team with which you registered for the challenge, and category can be one of the following types of predictions: UNDIRECTED-SIGNED, DIRECTED-SIGNED.

'''Scoring Metrics:''' Out of all the predicted pairs of genes, we will select those pairs in which both genes are a subset of the five genes in our five gene network. The score will then be computed as explained in Dataset FiveGeneNet1.

D2c1

Summary:

__NOTOC__
= BCL6 Transcriptional Target Prediction (DREAM2, Challenge 1) =

== Synopsis ==

A number of potential transcriptional targets of BCL6, a gene that encodes for a transcription factor active in B cells, have been identified with ChIP-on-chip data and functionally validated by perturbing the BCL6 pathway with CD40 and anti-IgM, and by over-expressing exogenous BCL6 in Ramos cell. We subselected a number of targets found in this way (the "gold standard positive" set), and added a number decoys (genes that have no evidence of being BCL6 targets, named the "gold standard negative" set), compiling a list of 200 genes in total. Given this list of 200 genes, the challenge consists of identifying which ones are the true targets and which ones are the decoys, using an independent panel of gene expression data.

== About the Data ==

* Data generously provided by [http://icg.cpmc.columbia.edu/faculty_Dalla-Favera.htm Riccardo Dalla-Favera] and [http://wiki.c2b2.columbia.edu/califanolab/index.php/Califano_Info Andrea Califano], Columbia University
* '''Reference:''' Klein U, Tu Y, Stolovitzky GA, Keller JL, Haddad J Jr, Miljkovic V, Cattoretti G, Califano A, Dalla-Favera R. Transcriptional analysis of the B cell germinal center reaction. Proc Natl Acad Sci U S A. 2003 Mar 4;100(5):2639-44. [http://www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed&cmd=search&term=12604779 PubMed]

== Best Performer ==

* Team 1: W.K. Sung, W.H. Lee, V. Narang, H. Xu, F. Lin, and K.C. Chin, Genome Institute of Singapore
* Team 2: Neil D. Clarke, Vinsensius B. Vega, Xing Yi Woo, Habib Hamidi, Hock Chuan Yeo, Zhen Xuan Yeo, and Guillaume Bourque, Genome Institute of Singapore
* Team 3: Ilya Shmulevich, Matti Nykter, Harri Lahdesmaki, Alistair Rust, and Vesteinn Thorsson, Institute for Systems Biology

== Take Action ==

* [[d2c1full|Full Challenge Description]] (archival)
* [{{link}}/results/DREAM2/?c=1 Team Rankings] (results)
* [{{link}}/data/DREAM2/ Download Training Data]
* [{{link}}/data/gold-standards/DREAM2/ Download Gold Standard]
* [{{link}}/data/scripts/DREAM2 Download Evaluation Scripts]
* Download Team Predictions (anonymous)

D2c2

Summary:

__NOTOC__
= Protein-Protein Interaction Network Inference (DREAM2, Challenge 2) =

== Synopsis ==

For many pairs of bait and prey genes, yeast protein-protein interactions were tested in an unbiased fashion using a high saturation, high-stringency variant of the yeast two-hybrid (Y2H) method. A high confidence subset of gene pairs that were found to interact in at least three repetitions of the experiment but that hadn’t been reported in the literature was extracted. There were 47 yeast genes involved in these pairs. Including self interactions, there are a total of 47*48/2 possible pairs of genes that can be formed with these 47 genes. As mentioned above some of these gene pairs were seen to consistently interact in at least three repetitions of the Y2H experiments: these gene pairs form the "gold standard positive" set. A second set among these gene pairs were seen never to interact in repeated experiments and were not reported as interacting in the literature; we call this the "gold standard negative" set. Finally in a third set of gene pairs, which we shall call the "undecided" set, genes were seen to interact only once or twice in repeated experiments, or were seen never to interact but were reported as interacting in the literature. The challenge consists of predicting which gene pairs belong to the gold standard positive set, and which gene pairs belong to the gold standard negative set.

== About the Data ==

* Data generously provided by Marc Vidal, Dana Farber Cancer Institute
* '''Reference:''' Yu H, et. al. High-quality binary protein interaction map of the yeast interactome network. Science. 2008 Oct 3;322(5898):104-10 [http://www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed&cmd=search&term=18719252 PubMed]
* [http://interactome.dfci.harvard.edu/S_cerevisiae/ Yeast Interactome Database]

== Best Performer ==

* BioRG_FIU: Hon Nian Chua, Willy Hugo, Guimei Liu, Xiaoli Li, Limsoon Wong, and See-KiongNg, Singapore’s Institute for Infocomm Research and the National University of Singapore.

== Take Action ==

* [[d2c2full|Full Challenge Description]] (archival)
* [{{link}}/results/DREAM2/?c=2 Team Rankings] (results)
* [{{link}}/data/DREAM2/ Download Training Data]
* [{{link}}/data/gold-standards/DREAM2/ Download Gold Standard]
* [{{link}}/data/scripts/DREAM2 Download Evaluation Scripts]
* Download Team Predictions (anonymous)

D2c3

Summary:

__NOTOC__
= Synthetic Five-Gene Network Inference (DREAM2, Challenge 3) =

== Synopsis ==

A synthetic-biology network consisting of 5 interacting genes was created and transfected to an in-vivo model organism. The challenge consists of predicting the connectivity of the five-gene network from in-vivo measurements.

== About the Data ==

* Data generously provided by [http://www.tigem.it/researchers/cosma/maria-pia-cosma.html Maria Pia Cosma] and [http://dibernardo.tigem.it/ Diego di Bernardo], TIGEM Telethon Institute of Genetics and Medicine, Naples, Italy
* '''Reference:''' Cantone I, Marucci L, Iorio F, Ricci MA, Belcastro V, Bansal M, Santini S, di Bernardo M, di Bernardo D, Cosma MP. "A yeast synthetic network for in vivo assessment of reverse-engineering and modeling approaches." Cell. 2009 Apr 3;137(1):172-81. [http://www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed&cmd=search&term=19327819 PubMed]

== Best Performer ==

* Team 0: Alberto de la Fuente, Angela Baralla, and Wieslaws Mentzen, CRS4 Bioinformatica, Universita degli Studi di Sassari, Italy
* Team 1: Daniel Marbach, Claudio Mattiussi, and Dario Floreano, from the Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland.

== Take Action ==

* [[d2c3full|Full Challenge Description]] (archival)
* [{{link}}/results/DREAM2/?c=3 Team Rankings] (results)
* [{{link}}/data/DREAM2/ Download Training Data]
* [{{link}}/data/gold-standards/DREAM2/ Download Gold Standard]
* [{{link}}/data/scripts/DREAM2 Download Evaluation Scripts]
* Download Team Predictions (anonymous)

D4c3

Summary:

__NOTOC__
= Predictive Signaling Network Modeling =
== DREAM4, Challenge 3 ==

'''Note: Both the data and pathway map cannot be used for purposes other than this challenge without the explicit permission of the data providers. (See below for contact information.)'''

== Synopsis ==

This challenge explores the extent to which our current knowledge of signaling pathways, collected from a variety of cell types, agrees with cell-type specific high-throughput experimental data. Specifically, we ask the challenge participants to create a cell-type specific model of signal transduction using the measured activity levels of signaling proteins in HepG2 cell lines. The model, which can leverage prior information encoded in a generic signaling pathway provided in the challenge, should be biologically interpretable as a network, and capable of predicting the outcome of new experiments.

== The challenge ==

It is an open question how to make use of the accumulated body of knowledge of signaling pathways to create mechanistic, predictive signaling network models. The network depicted in Figure 1 is representative of the type of information about the topology of signaling pathways that can be culled from the literature [1]. Figure 1 depicts canonical pathways downstream of major receptors to four ligands (represented by green nodes): two inflammatory (TNFa, IL1a), one insulin (IGF-I), and one growth factor (TGFa). Note that this pathway map is not cell-type specific.

In addition to the topology of "canonical" signaling pathways based on the accumulation of evidence from multiple cell-types, we have at our disposal a data set consisting of measurements of phosphoprotein activity levels in the HepG2 cell line using the Luminex xMAP sandwich assay. Measurements of certain phosphoprotein activities were measured under various perturbations of the signaling pathways [2]. The signaling pathways were stimulated with one or more of the ligands mentioned above. The pathways were also perturbed with chemical inhibitors of specific phosphoproteins. In Figure 1, blue and magenta nodes indicate the phosphoproteins measured by the xMAP assay; red and magenta nodes indicate the phosphoproteins that were inhibited.

[[Image:DREAM4_Challenge3_Figure1.png|center]]

<blockquote><blockquote>Figure 1. Pathway map summarizing the current public knowledge of the signaling pathway pertaining to this challenge, simplified from [1]. Green nodes represent stimuli. Red nodes represent inhibited proteins. Blue nodes indicate proteins whose phosphorylation is measured. Magenta nodes represent proteins that are both inhibited and measured. Grey nodes represent proteins considered to be involved in the relevant pathways. This figure was created with Cytoscape http://www.cytoscape.org/) from data obtained from the Ingenuity knowledgebase.</blockquote></blockquote>

The Hep2G data set is plotted in Figure 2, which is organized into panels corresponding to the various ligands. Upon pretreatment with an inhibitor (or no inhibitor), measurements (phosphoprotein activities) of seven proteins at three time points (0, 30 minutes, and 3 hours post stimulus) were acquired.

The challenge entails "customizing" the provided pathway map (Figure 1) so that it is an accurate representation of the provided data set (Figure 2). Specifically, we are soliciting

# A revised network specific to the HepG2 cell line. The revised network could be produced by removal of links that are not supported by the provided data set from the pathway map of Figure 1, and/or, addition of links that are supported by data, but absent from the pathway map of Figure 1.
# The predicted values of the 7 measured phosphoproteins for all 20 possible pairwise combinations of the following stimuli and inhibitors which comprise a "test set."

Stimuli: IL1a, IGF1, TGFa, and TGFa+IGF1
<br>Inhibitors: pp38+MEK, PI3K+MEK, p38+PI3K, p38+IKK, and PI3K+IKK

In the above list, TGFa+IGF1 indicates that both TGFa and IGF1 were simultaneously applied to the cells. The same is true for simultaneous application of inhibitors such as PI3K+IKK.
The answer to the challenge should entail some interplay between predictive modeling and network reconstruction. Any modeling formalism may be used as long as the model is amenable to be interpreted as a network. A wide range of modeling formalisms can be applied and model relevance will be ascertained by how close the model predicts the response to the set of test stimuli and inhibitors.

[[Image:DREAM4_Challenge3_Figure2.png|center]]

<blockquote><blockquote>Figure 2. Training data set. Time courses for the phosphorylation of 7 key proteins (rows) in the cancer cell line HepG2 treated with 5 different protein inhibitors (including no inhibitor) under 5 different conditions of cytokine stimulation (panels, including no cytokine stimulus) [2]. When the measured molecule is inhibited the measurement cannot be used (denoted with a big red X). The numbers at the right of the figure indicate the maximum value for the signals across all conditions (i.e., the maximum value of the corresponding row) and it is in arbitrary units (fluorescent intensity). The figure was created using DataRail [3].</blockquote></blockquote>

== Data ==

The canonical pathway map (Figure 1) is provided in several formats with filenames
* '''DREAM4_SignalingNetwork.ext'''
where ext is one of the following: pdf, gml, xgmml, cys, and sif. All formats were created with Cytoscape. The sif (simple interaction file ) format contains the human readable list of edges in source/target format; Gml (Graph markup language ) and xgmml (extensible graph markup and modeling language) provide additional information about the network visualization (not relevant for the analysis), and cys is the intern Cytoscape format.

The data set (Figure 2) is provided as comma-separated-value files in two formats, DataRail’s MIDAS (Minimum Information for Data Analysis in Systems Biology) format [3], and a simple table, with the filenames:
* '''SignalingNetworkChallenge_TrainingData_MIDAS-format.csv'''
* '''SignalingNetworkChallenge_TrainingData.csv'''

=== Important information regarding measurements ===

(a) Data integrity / linearity. Significant effort was dedicated to data integrity. The data are reported as arbitrary (fluorescence) units in the range between 0 and ~29000. The upper limit (~29000) corresponds to the saturation limit of the detector. Experiments were performed in such a way that measurements are as much as possible within the linear range of the detector. In general, data can be considered linear but there are a few cases that measurements are closer to the upper detection limit of ~29000 (e.g. some AKT measurements) where linearity might have been lost.

(b) Detection limits/Repeatability. The coefficient of variation for repeated measurements was found to be ~8% (mostly due to biological variability). With our current experimental design the instrument detector can report data with accuracy as low as ~300. For example, changes from 55 fluorescence units (FU) to 110 FU cannot be considered "2 fold increase" because values lie within the noise error of the detector. On the contrary, data from 1000 to 2000 are significant.

(c) Comparability between phosphoproteins. In the xMAP sandwich assays used to collect the data for this challenge, the fluorescence measured for one phosphoprotein is not directly comparable to that of another phosphoprotein. For example, the same readout of 1000 in AKT and ERK signal does not imply that the concentration of AKT and ERK are the same. The reason is that even at the same concentration, the amount of light detected for different phosphoproteins depends on the affinity of the antibodies to the phosphoproteins.

(d) Comparability between training and test sets. The lysate concentration used for the measurements of the training data set (contained in the file SignalingNetworkChallenge_TrainingData.csv) was different from the lysate concentration used for the test data set. This was done to keep the measurement values within the linear range of the detector. Therefore, even for the same phosphoprotein and under the same conditions, the measurement in the training and test data sets could be different. This is why we give the value of the measurement at t=0 in the file DREAM4_TeamName_SignalingNetworkPredictions_Test.csv, as these values could, in principle, be different from the values at t=0 for similar conditions in the training set. Therefore, the predictions at t=30 min have to take into account the baseline value at t=0 of the test set rather than equivalent measurements in the training set. For clarifications on this important aspect of the data, please feel free to contact the DREAM organizers or the data providers.

== Submission ==

Challenge participants will submit three files:

(1) Predictions of the 7 phosphoprotein activities under the various perturbations in the test set. These predictions should be submitted within the template file:
* '''DREAM4_TeamName_SignalingNetworkPredictions_Test.csv'''
provided with the data. At submission, replace TeamName with the name of your team, and the entries containing the text "PREDICT" with your numerical predictions.

(2) A list of edges of the network underlying your predictions. (The model used to produce your prediction must be interpretable as a network.) Submit this list as the file
* '''DREAM4_TeamName_SignalingNetworkPredictions_Edges.txt'''
replacing TeamName with the name of your team. Your network must be submitted as a tab delimited list of node pairs, which represent edges in the network. Only edges supported by your model should be included in the submitted edge list, and the order of the list is inessential. Edges such as tgfa→ erk12 and igf1→ hsp27, for example, should be encoded as:

<pre>
tgfa \tab erk12
igf1 \tab hsp27
...
</pre>

Identify the nodes using ONLY the following node labels:
tgfa, igf1, tnfa, il1a, akt, jnk12, erk12, ikb, hsp27, mek12, p38, pi3k, ikk

These are the colored nodes in Figure 1. Your network submission may not have self-loops or node labels other than those provided above. (See section Network compression for submission for additional information.)

(3) A one to two page write-up explaining how the predictions are produced from the network. The write-up helps enforce the purpose of the challenge: to develop a predictive network model. This write-up can contain pseudo-code describing the algorithm used. Submit the write-up as the file
* '''DREAM4_TeamName_SignalingNetworkPredictions_Writeup.ext'''
replacing TeamName with the name of your team and the file extension (ext) with your choice of txt, doc, rtf, or pdf.

===Network compression for submission===

Only some of the nodes in the pathway map of Figure 1 are measured or manipulated in the HepG2 cell line data. However, a model may contain a representation of additional proteins that are not measured or manipulated in the assays (latent variables). To facilitate scoring and comparison of models from different teams, we ask that the network edges be reported using only the nodes that we provide for the explicit purpose of submission of the network. For example, if your model has a pathway A-->B-->C, but B is a latent variable, then this pathway should be "compressed" and reported as A-->C for the purpose of submission of the network.

== Scoring ==

The submissions will be scored by the prediction error in the test set and the parsimony of the submitted network. More specifically the prediction cost function will be scored as a sum of squared errors over all the predictions. The teams that have a prediction error below a threshold (determined by a low p-value) will be further evaluated according to the number of edges in the submitted network. Of the most significant predictive models, the team with the sparsest network will be considered a best performer.

We realize that "cheating teams" might use a prediction model that is not associated with any network interpretation, and submit a network without edges. We strongly discourage this, as the idea of this challenge is to explore the possibility of using predictive models that are biologically interpretable, and could lead to the formulation of new hypotheses.

== References ==

[1] Saez-Rodriguez J, Alexopoulos L, Epperlein J, Samaga R, Lauffenburger DA, Klamt, S and Sorger PK (2009) "Discrete logic modeling as a means to link protein signaling networks with functional analysis of mammalian signal transduction." Mol Syst Biol in press.

[2] Alexopoulos L, Saez-Rodriguez J, Cosgrove B, Lauffenburger DA, Sorger PK. Net-
works reconstructed from cell response data reveal profound differences in signaling by
Toll-like receptors and NF-κB in normal and transformed human hepatocytes. Submitted.

[3] Saez-Rodriguez J, Goldsipe A, Muhlich J, Alexopoulos LG, Millard B, Lauffenburger DA, Sorger PK. Flexible informatics for linking experimental data to mathematical models via DataRail, Bioinformatics. 2008 Mar 15;24(6):840-7. (http://code.google.com/p/sbpipeline/wiki/DataRail).

== Authors ==

The challenge was generously provided before publication by Julio Saez-Rodriguez, Leonidas Alexopoulos*, and Peter Sorger, from the Department of Systems Biology, Harvard Medical School and Biological Engineering Department, M.I.T. The challenge has been designed in collaboration with Robert Prill and Gustavo Stolovitzky from the IBM T.J. Watson Research Center in New York.

*Present Address: Department of Mechanical Engineering, National Technical University of Athens

== Download ==

* [{{link}}/data/DREAM4/ Download Data]

Don't hesitate to post a question in the DREAM [{{link}}/discuss discussion board] if you need any clarification on this challenge.

D2c2full

Summary:

= Protein-Protein Interaction Network Inference (DREAM2, Challenge 2) =

<div style="background-color:#FFFF99;width:100%">
This archival page describes the challenge exactly as it was presented to the participants. Go to the main [[D2c2|DREAM2 Challenge 2]] page to download data, view team rankings, cite this work, etc.
</div>

== Synopsis ==

For many pairs of bait and prey genes, yeast protein-protein interactions were tested in an unbiased fashion using a high saturation, high-stringency variant of the yeast two-hybrid (Y2H) method. A high confidence subset of gene pairs that were found to interact in at least three repetitions of the experiment but that hadn’t been reported in the literature was extracted. There were 47 yeast genes involved in these pairs. Including self interactions, there are a tot"al of 47*48/2 possible pairs of genes that can be formed with these 47 genes. As mentioned above some of these gene pairs were seen to consistently interact in at least three repetitions of the Y2H experiments: these gene pairs form the "gold standard positive" set. A second set among these gene pairs were seen never to interact in repeated experiments and were not reported as interacting in the literature; we call this the "gold standard negative" set. Finally in a third set of gene pairs, which we shall call the "undecided" set, genes were seen to interact only once or twice in repeated experiments, or were seen never to interact but were reported as interacting in the literature. The challenge consists of predicting which gene pairs belong to the gold standard positive set, and which gene pairs belong to the gold standard negative set.

== Dataset ==

''File with Gene IDs:'' The file '''Prot-Prot_Genes.xls''' contains a list of 47 yeast genes (identified with ORF IDs). The challenge consists of determining the set of true positive and the set of true negative protein-protein interactions among all the pairwise interactions between these 47 genes.

== Submission Information ==

Submit a ranked list of gene pairs, ordered according to the confidence you assign to your prediction that a pair interacts, from the most reliable (first row) to the least reliable (last row) prediction. Use a tab-separated 3 column format as in the example below:

:YeastGeneA \tab YeastGeneB \tab XYZ

where YeastGeneA and YeastGeneB are genes in the file '''Prot-Prot_Genes.xls''', and XYZ is an interaction score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair interacts. (E.g., XYZ = 1 if the pair is deemed to interact with highest confidence and XYZ = 0 if the pair is deemed not to interact.) All pairs omitted from the list but that belong to the gold standard positive or gold standard negative set will be considered to appear randomly ordered at the end of the list with XYZ = 0. Submitted pairs that belong to the undecided set will not be scored. Save the file as unformatted text, and name it:

:'''TeamName_ProtProtSubnet.txt'''

where TeamName is the name of the team with which you registered for the challenge.

== Scoring Metrics ==

The submitted list will be judged exclusively on the gold standard positive and gold standard negative sets. Submitted pairs that belong to the undecided set will not be scored. We will score the results using the area under the precision versus recall curve for the whole set of predictions. For the first ''k'' predictions (ranked by score, and for predictions with the same score, taken in the order they were submitted in the list of gene pairs), precision is defined as the fraction of correct gold standard positive predictions to ''k'', and recall is the proportion of correct gold standard positive predictions out of all the possible gold standard positive interactions. Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated.

D4c2

Summary:

__NOTOC__
= In Silico Network Challenge =
== DREAM4, Challenge 2 ==

== Synopsis ==

The goal of the ''in silico'' network challenge is to reverse engineer gene regulation networks from simulated steady-state and time-series data. Participants are challenged to infer the network structure from the given ''in silico'' gene expression datasets. Optionally, participants may also predict the response of the networks to a set of novel perturbations that were not included in the provided datasets.

== The three sub-challenges ==

There are three ''in silico'' sub-challenges called

* InSilico_Size10
* InSilico_Size100
* InSilico_Size100_Multifactorial

The sub-challenges differ in the size of the network and the type of data provided. Predictions are assessed independently for each sub-challenge. Thus, teams may choose to submit predictions to all three or only some of the challenges.

Each sub-challenge consists of five networks (the so-called gold standard networks). In order to participate in a challenge, predictions for all five networks of this sub-challenge must be submitted. The rational is that in this way it will be possible to assess how consistently a method predicts the topology in five independent networks of the same type and size.

===InSilico_Size10 sub-challenge===

In the first sub-challenge, we provide all of the datasets described in the next section (wild-type, knockouts, knockdowns, multifactorial perturbations, and time series) for five networks of size 10. Participants are challenged to predict the directed unsigned topology of these networks. In addition, participants can choose to predict the network response to previously unseen perturbations in the bonus round described below. Note that the best performer of the sub-challenge will be determined solely based on the prediction of the network topologies, and participation in the bonus round is optional.

'''Bonus round.''' Whereas some inference methods focus on predicting only network structures, others reverse engineer (potentially) predictive dynamical models, which could be used to predict the network response to novel perturbations that were not included in the original datasets. We invite participants that tackle inference of such models to predict, in addition to the network structure, also the steady-state levels of dual knockout experiments (knockout of two genes simultaneously, as described in the next section).

===InSilico_Size100 sub-challenge===

The second sub-challenge is similar to the first one, except that the five networks are of size 100. Furthermore, only the wild-type, knockout, knockdown, and time-series datasets are provided (the multifactorial perturbation datasets are not included as they are the subject of another sub-challenge). The primary goal is to predict the network structures, but there is an optional bonus round where participants can evaluate whether their inferred models correctly predict the effect of dual knockouts.

===InSilico_Size100_Multifactorial sub-challenge===

The third sub-challenge consists of five networks of size 100. In this challenge, we assume that extensive knockout / knockdown or time series experiments can't be performed. Instead, different variations of the network can be observed (e.g., samples from different patients). Thus, only the multifactorial perturbation dataset described below is provided. The goal is prediction of the network structure. There is no bonus round in this challenge.

==The datasets==

The data are given for each of the three sub-challenges in the following three files ([{{link}}/data/DREAM4/ download here]):

* DREAM4_InSilico_Size10.zip
* DREAM4_InSilico_Size100.zip
* DREAM4_InSilico_Size100_Multifactorial.zip

We will now describe the types of experiments that we simulated to produce gene expression datasets, and the name of the files where this data is included. In all cases, the data corresponds to noisy measurements of mRNA levels, which have been normalized such that the maximum normalized gene expression value in the datasets of a given network is one.

===Wild-type===

The files '''*wildtype.tsv''' contain the steady-state levels of the wild-type (the unperturbed network).

===Knockouts===

The files '''*knockouts.tsv''' contain the steady-state levels of single-gene knockouts (deletions). An independent knockout is provided for every gene of the network. A knockout is simulated by setting the transcription rate of this gene to zero. The k'th data line of the file *knockouts.tsv is the steady-state of the network after knockout of gene k.

===Knockdowns===

The files '''*knockdowns.tsv''' contain the steady-state levels of single-gene knockdowns. A knockdown of every gene of the network is simulated. Knockdowns are obtained by reducing the transcription rate of the corresponding gene by half. The k'th data line of the file *knockdowns.tsv is the steady state of the network after knockdown of gene k.

===Multifactorial perturbations===

The files '''*multifactorial.tsv''' contain steady-state levels of variations of the network, which are obtained by applying multifactorial perturbations to the original network. Each line gives the steady state of a different perturbation experiment, i.e., of a different variation of the network. One may think of each experiment as a gene expression profile from a different patient, for example. We simulate multifactorial perturbations by slightly increasing or decreasing the basal activation of all genes of the network simultaneously by different random amounts.

===Time series===

The files '''*timeseries.tsv''' contain time courses showing how the network responds to a perturbation and how it relaxes upon removal of the perturbation. For networks of size 10 we provide 5 different time series, for networks of size 100 we provide 10 time series. Each time series has 21 time points. The initial condition always corresponds to a steady-state measurement of the wild-type. At t=0, a perturbation is applied to the network as described below. The first half of the time series (until t=500) shows the response of the network to the perturbation. At t=500, the perturbation is removed (the wild-type network is restored). The second half of the time series (until t=1000) shows how the gene expression levels go back from the perturbed to the wild-type state.

In contrast to the multifactorial perturbations described in the previous section, which affect all the genes simultaneously, the perturbations applied here only affect about a third of all genes, but basal activation of these genes can be strongly increased or decreased. For example, these experiments could correspond to physical or chemical perturbations applied to the cells, which would cause (via regulatory mechanisms not explicitly modeled here) some genes to have an increased or decreased basal activation. The genes that are directly targeted by the perturbation may then cause a change in the expression level of their downstream target genes.

===Dual knockouts===

Dual knockouts consist of simulating each of the five networks in which two gene are knocked-out simultaneously. Gene expression data for dual knockouts is not provided to the participants. Instead, participants may predict steady-state levels for dual knockouts in the bonus round described in the previous section. The files '''*dualknockouts_indexes.tsv''' indicate the pairs of genes for which a dual knockout should be predicted. For example, the line "6 8" means that participants should predict the steady-state of the network after knocking out genes 6 and 8. For networks of size 10 we ask for predictions for 5 dual knockout experiments, for networks of size 100 we ask for 20 predictions.

==Submission Information==

===Network predictions===

Network predictions must be directed and unsigned. There are no self-interactions (auto-regulatory loops) in the gold standard networks. Predictions of self-loops are ignored by the scoring.

Submit a ranked list of regulatory link predictions ordered according to the confidence you assign to the predictions, from the most reliable (first row) to the least reliable (last row) prediction. Use a 3 tab-separated column format as in the example below:

A \tab B \tab XYZ

where A and B are two different genes (no self-interactions). Links are directed: the gene in the first column regulates the gene in the second column. (If both A regulates B and B regulates A, then both lines should be included.) XYZ is a score between 0 and 1 that indicates the confidence level you assign to the prediction. (E.g., XYZ = 1 if gene A is deemed to regulate gene B with highest confidence and XYZ = 0 if A is deemed not to directly regulate B). All pairs omitted from the list will be considered to appear randomly ordered at the end of the list. Save the file as text, and name it:

* '''DREAM4_TeamName_SubChallenge_Network.txt'''

where TeamName is the name of the team with which you registered for the challenge, SubChallenge is either InSilico_Size10, InSilico_Size100, or InSilico_Size100_Multifactorial, and Network is one of the five networks of the indicated challenge (1,2,...,5). As mentioned above, to participate in a challenge you need to submit predictions for all five networks of this challenge.

===Bonus round predictions===

Predictions for double knockouts in the bonus round should be submitted in the following format. The file should have M lines, where M is the number of double knockouts to be predicted (5 for networks of size 10 and 20 for networks of size 100). Line k should contain the steady-state levels of all genes in the k'th double knockout experiment

x_1 \tab x_2 \tab x_3 \tab ... x_N \newline

where x_i is the predicted expression level of gene i, and N is the size of the network. The two genes that should be knocked out in the k'th experiment are indicated in the file *doubleknockout_indexes.tsv, as described in the previous section. If the pair of genes (u, v) are knocked out in the k'th experiment, x_u and x_v must be equal zero in that line (we will verify this to check that the file format is correct).

Please submit a separate file for every network. Use the same naming convention as explained above for the network predictions and append _dualknockouts to the filename:

* '''DREAM4_TeamName_SubChallenge_Network_dualknockouts.txt'''

==Scoring Metrics==

We will score the results using the area under the precision versus recall curve for the whole set of link predictions for a network. For the first k predictions (ranked by score, and for predictions with the same score, taken in the order they were submitted in the prediction files), precision is defined as the fraction of correct predictions to k, and recall is the proportion of correct predictions out of all the possible true connections. Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated. Teams will be ranked according to their overall performance over the five networks of a challenge.

Predictions for dual knockouts in the bonus round will be evaluated by comparing them to the true, noise-free gene expression values (e.g. using a sum of square error).

==How were the ''in silico'' benchmarks generated?==

'''Network structures'''. Network topologies were obtained by extracting subnetworks from transcriptional regulatory networks of ''E. coli'' and ''S. cerevisiae''. We adapted the subnetwork extraction method to preferentially include parts of the network with cycles. Auto-regulatory interactions were removed, i.e., there are no self-interactions in the ''in silico'' networks.

'''Dynamical model.''' The dynamics of the networks were simulated using a detailed kinetic model of gene regulation. Both independent and synergistic gene regulation occur in the networks. Both transcription and translation are modeled. However, the protein concentrations are not included in the provided datasets. As mentioned above, the datasets correspond to the mRNA concentration levels.

'''Noise.''' The simulations are based on stochastic differential equations (Langevin equations) to model internal noise in the dynamics of the networks. In addition, we add measurement noise to the generated gene expression datasets. We use an existing model of noise observed in microarrays, which is very similar to a mix of normal and lognormal noise.

'''Software.''' All networks and data were generated with version 2.0 of GeneNetWeaver (GNW). The previous version of GNW, which was used to generate the DREAM3 challenges, is available at [http://gnw.sourceforge.net gnw.sourceforge.net].

'''Additional information.''' Additional information, including a short description of the dynamical model, will be posted on our website: [http://lis.epfl.ch/research/projects/EvolutionOfAnalogNetworks/ReverseEngineeringGeneRegulatoryNetworks/DREAMChallenges.php DREAM4 ''In Silico'' Challenge additional information].

==Authors==

The challenge was provided by Daniel Marbach, Thomas Schaffter, and Dario Floreano from the [http://lis.epfl.ch/grn Laboratory of Intelligent Systems] of the Swiss Federal Institute of Technology in Lausanne. The challenge has been designed in collaboration with Robert Prill and Gustavo Stolovitzky from the IBM T.J. Watson Research Center in New York.

All data can be freely used. If you use this data in your publication, please cite the following papers:

* Marbach, D., Schaffter, T., Mattiussi, C. and Floreano, D. (2009) Generating Realistic ''in silico'' Gene Networks for Performance Assessment of Reverse Engineering Methods. Journal of Computational Biology, 16(2) pp. 229-239. [[http://infoscience.epfl.ch/record/128148 detailed record]] [[http://infoscience.epfl.ch/getfile.py?docid=20591&name=Marbach2008c-preprint&format=pdf&version=1 preprint]] [[http://infoscience.epfl.ch/export.py?recid=128148&fm=bibtex bibtex]]

* Stolovitzky G, Prill RJ, Califano A. "Lessons from the DREAM2 Challenges", in Stolovitzky G, Kahlem P, Califano A, Eds, Annals of the New York Academy of Sciences, 1158:159-95 (2009)

* Stolovitzky G, Monroe D, Califano A. "Dialogue on Reverse-Engineering Assessment and Methods: The DREAM of High-Throughput Pathway Inference", in Stolovitzky G and Califano A, Eds, Annals of the New York Academy of Sciences, 1115:11-22 (2007)

== Download ==

* [{{link}}/data/DREAM4/ Download Data]

Don't hesitate to post a question in the DREAM [{{link}}/discuss discussion board] if you need any clarification on this challenge.

D2c1full

Summary:

= BCL6 Transcriptional Target Prediction (DREAM2, Challenge 1) =

<div style="background-color:#FFFF99;width:100%">
This archival page describes the challenge exactly as it was presented to the participants. Go to the main [[D2c1|DREAM2 Challenge 1]] page to download data, view team rankings, cite this work, etc.
</div>

== Synopsis ==

A number of potential transcriptional targets of BCL6, a gene that encodes for a transcritpion factor active in B cells, have been identified with ChIP-on-chip data and functionally validated by perturbing the BCL6 pathway with CD40 and anti-IgM, and by over-expressing exogenous BCL6 in Ramos cell. We subselected a number of targets found in this way (the "gold standard posititve" set), and added a number decoys (genes that have no evidence of being BCL6 targets, named the "gold standard negative" set), compiling a list of 200 genes in total. Given this list of 200 genes, the challenge consists of identifying which ones are the true targets and which ones are the decoys, using an independent panel of gene expression data.

== Dataset ==

''File with Gene IDs:'' The file '''BCL6_targets_and_decoys.xls''' contains the Entrez GeneIDs (first column) along with the corresponding Affymetrix HGU95Av2 GeneChip probe sets corresponding to each gene (second column). When more than one probe set is associated with the same Entrez GeneID, the probe sets are separated by two slashes: //. Some of these genes are true transcriptional targets of BCL6. For the remaining genes, there is no evidence that they are BCL6 targets. To determine which of these 200 genes are BCL6 targets and which are not, you can use sequence information, gene ontology annotations, or any other tool you consider appropriate. You can also use the microarray data described below.

''Microarray Data:'' A panel of 336 Affymetrix HGU95Av2 GeneChip arrays probing B cells under different conditions can be accessed from the Gene Expression Omnibus database at http://www.ncbi.nlm.nih.gov/geo/ by querying for GEO accession GSE2350. Files with MAS5 normalized data in matrix format can be downloaded by clicking on the “Series Matrix File(s)” link near the bottom of the page. Note that there are two files ('''GSE2350_series_matrix-1.txt.gz''' with 255 chips and '''GSE2350_series_matrix-2.txt.gz''' with 81 chips) that must be joined together to utilize the entire dataset. In each file, the data matrix begins after a series of header lines which begin with the character ‘!’. In the first line, the entries after “ID_REF” are column headers listing the name of each sample. All succeeding lines (until the last one) contain a probe set ID followed by a series of numbers corresponding to the MAS5 normalized intensity values for this probe set and the corresponding sample listed on the column header. The file is terminated by the line “!series_matrix_table_end”. Alternatively, raw data in .cel format can be downloaded by clicking on the '''GSE2350_RAW.tar''' link. This data can then be normalized using the method of your choice.

''Useful Information:'' In the HGU95Av2 GeneChip, BCL6 (Entrez GeneID 604) is represented by three probe sets: 40091_at//978_at//979_g_at.

== Submission Information ==

Submit a ranked list of genes, ordered according to the confidence you assign to your prediction that a gene is a true BCL6 transcriptional target, from the most reliable (first row) to the least reliable (last row) prediction. Use a tab-separated 2 column format as in the example below:

:nnn \tab XYZ

where nnn is one of the Entrez GeneID identifiers in the file '''BCL6_targets_and_decoys.xls''', and XYZ is a score between 0 and 1 that indicates the confidence level you assign to the prediction that a gene is a true BCL6 transcriptional target. (E.g., XYZ=1 if the gene is deemed to be a target with highest confidence and XYZ = 0 if a gene is deemed not to to be a target.) All genes omitted from the list but that belong to the gold standard positive and gold standard negative sets will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save your prediction file as unformatted text, and name it:

:'''TeamName_BCL6targets.txt'''

where TeamName is the name of the team with which you registered for the challenge.

== Scoring Metrics ==

We will score the results using the area under the precision versus recall curve for the whole set of predictions. For the first ''k'' predictions (ranked by score, and for predictions with the same score, taken in the order they were submitted in the prediction file), precision is defined as the fraction of correct gold standard positive predictions to ''k'', and recall is the proportion of correct gold standard positive predictions out of all the possible gold standard positive targets. Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated.

D4c1

Summary:

__NOTOC__
= Peptide Recognition Domain (PRD) Specificity Prediction =
== DREAM4, Challenge 1 ==

== Synopsis ==

Many important protein-protein interactions are mediated by peptide recognition domains (PRD), which bind short linear sequence motifs in other proteins. For example, SH3 domains typically recognize proline-rich motifs, PDZ domains recognize hydrophobic C-terminal tails, and kinases recognize short sequence regions around a phosphorylatable residue [1].

Given the sequence of the domains, the challenge consists of predicting a position weight matrix (PWM) that describes the specificity profile of each of the given domains to their target peptides. Any publicly accessible peptide specificity information available for the domain may be used.

== Background ==

Ideally, PRD specificity could be predicted directly from the sequence of the domain itself. This will enable the prediction of protein-protein interaction networks directly from the genome sequence.

The specificity of selected human SH3, synthetic PDZ and kinase PRDs were experimentally mapped using phage display and combinatorial peptide libraries. The peptide libraries contain many short peptides with diverse sequences, around ten amino acids in length. The domain is used to select peptides from the library that bind to it. The set of peptides that bind to a domain defines a short, linear sequence pattern that the domain is expected to recognize. This pattern can be represented probabilistically as a position weight matrix (PWM). The PWM representation implicitly assumes independence of the motif positions. While in certain motifs interactions between some positions may exist, they are neglected for this challenge.

Publicly available information about the domain family that may be useful for prediction includes known ligands of members of the domain family from the literature or databases like [http://mint.bio.uniroma2.it/domino/search/searchWelcome.do DOMINO] [2] or [http://icb.med.cornell.edu/services/pdz/start PDZBase] [3] and structures from the [http://www.rcsb.org/pdb/home/home.do PDB] [4].

== The Challenge ==

Peptides bound by SH3, PDZ, and kinase PRDs were experimentally identified. These data constitute an unpublished "gold standard" for the binding specificity of the selected PRDs.

Given the sequence of the domains, the challenge consists of predicting a position weight matrix (PWM) that describes the specificity profile of each of the given domains to their target peptides. Any publicly accessible peptide specificity information available for the domain may be used.

== Data ==

* '''DREAM4_DomainSequences.txt''' contains 5 human SH3 domain sequences, 3 serine/threonine kinase sequences and 5 synthetic PDZ domain sequences modeled on Erbin (Erbb2 interacting protein).

== Submission ==

Using the provided tab delimited template file

* '''DREAM4_TeamName_PWM.txt'''

and keeping the formatting of this file, submit a ten-column PWM for each domain. An example PWM is illustrated below. Each row corresponds to an amino acid, each column corresponds to the probability that the given amino acid is found at that position. Each of the ten columns must sum to 1.0. (Note that the amino acids are ordered alphabetically by IUPAC single letter code. Please keep this template format.)

<center>
<pre>
A 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
C 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
D 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
E 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
F 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
G 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
H 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
I 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
K 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
L 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
M 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
N 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
P 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
Q 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
R 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
S 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
T 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
V 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
W 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
Y 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
</pre>
</center>

* If a column is not predicted, enter 0.05 for all rows in that column, signifying uniform background frequency.

* If a PWM is not predicted, leave 0.05 for all columns and all rows for that PWM.

* All PWM predictions must be placed in one text file according to the template, keeping the order of the template file as it is.

* A best performer will be identified for each of the three domain types (SH3, PDZ, and kinase). You must submit predictions for at least one of the domain types. All the instances of the PRD in a given domain type must be predicted in order for your submission to be scored in that domain type.

* Replace TeamName in the filename "DREAM4_TeamName_PRD.txt" with the name of your team before submitting.

== Scoring Metrics ==

The submitted PWM predictions will be judged exclusively by similarity to the experimentally mapped PWM using the distance induced by the Frobenius Norm (http://mathworld.wolfram.com/FrobeniusNorm.html).

Domain specific notes:

*Kinase: Column 6 in the PWM must correspond to the phosphorylatable S/T residue in the peptide that binds to the kinase.

* PDZ: Column 10 in the PWM must correspond to C-terminus of the peptide that binds to the PDZ domain.

* SH3: No anchor position in the PWM is defined. Every possible alignment of the predicted SH3 peptide specificity PWM with the experimentally mapped SH3 peptide specificity PWM of length >=5 will be tried and the final score will be equal to the highest similarity found.

== References ==

# Pawson T, Nash P (2003) Assembly of cell regulatory systems through protein interaction domains. Science 300: 445-452.
# Ceol A, Chatr-aryamontri A, Santonico E, Sacco R, Castagnoli L, et al. (2007) DOMINO: a database of domain-peptide interactions. Nucleic Acids Res 35: D557-560.
# Beuming T, Skrabanek L, Niv MY, Mukherjee P, Weinstein H (2005) PDZBase: a protein-protein interaction database for PDZ-domains. Bioinformatics 21: 827-828.
# Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat TN, et al. (2000) The Protein Data Bank. 28: 235-242.

== Authors ==

The challenge was provided by Gary Bader and Philip M. Kim, from the Terrence Donnelly Center for Cellular and Biomolecular Research, University of Toronto. Pre-publication data was provided generously by Sachdev Sidhu, Terrence Donnelly Center for Cellular and Biomolecular Research, University of Toronto and Ben Turk, Deparment of Pharmacology, Yale University. The challenge has been designed in collaboration with Robert Prill and Gustavo Stolovitzky from the IBM T.J. Watson Research Center in New York.

== Download ==

* [{{link}}/data/DREAM4/ Download Data]

Don't hesitate to post a question in the DREAM [{{link}}/discuss discussion board] if you need any clarification on this challenge.

D3c4full

Summary:

__NOTOC__
= DREAM3 In-Silico Network Challenge (DREAM3, Challenge 4) =

<div style="background-color:#FFFF99;width:100%">
This archival page describes the challenge exactly as it was presented to the participants. Go to the main [[D3c4|DREAM3 Challenge 4]] page to download data, view team rankings, cite this work, etc.
</div>

== Synopsis ==

The goal of the ''in silico'' challenges is the reverse engineering of gene networks from steady state and time series data. Participants are challenged to predict the directed unsigned network topology from the given ''in silico'' generated gene expression datasets.

== The Three Challenges ==

There are three ''in-silico'' challenges corresponding to gene networks with 10, 50, and 100 genes. Predictions are assessed independently for each challenge. Thus, teams may choose to submit predictions only for one or two of the challenges. However, we encourage teams to participate in all three challenges in order to compare how well different methods perform on different network sizes.

Each challenge consists of five gold standard networks. In order to participate in a challenge, predictions for all five networks of this challenge must be submitted. The rational is that in this way it will be possible to assess how consistently a method predicts the topology in five independent networks of the same type and size.

== The Datasets ==

For consistency, we provide the same type of data as in the [[The_In-Silico-Network_Challenges._Description | DREAM2 ''in-silico'' Challenge]]. For every network, the following experiments are simulated:

'''Heterozygous knock-down'''. The files ''*-heterozygous.tsv'' (the meaning of the wild card * will be explained lines below) contain the steady state levels for the wild-type and the heterozygous knock-down strains for each gene (+/-). Thus, for a network of size N there are N+1 experiments (wild-type plus knock-down of every gene).

'''Null-mutants'''. The files ''*-null-mutants.tsv'' contain the steady state levels for the wild-type and the null-mutant strains for each gene (-/-). Thus, for a network of size N there are N+1 experiments (wild-type plus knock-out of every gene).

'''Trajectories'''. The files ''*-trajectories.tsv'' contain time courses of the network recovering from several external perturbations. For the networks of size 50, the same number of time courses as in the DREAM2 in silico challenge are provided (23 different perturbations). For the networks of size 10 and 100, we give 4 and 46 perturbations respectively (each one with 21 time points).

The * in front of ''*–heterozygous.tsv'', ''*-null-mutants.tsv'' and ''*-trajectories.tsv'' can take the values *=InSilicoSizeN-OraganismK, where N=10, 50, or 100, Organism is Ecoli or Yeast and K = 1 or 2 if Organism is Ecoli, and K=1, 2, or 3 if Organism is Yeast.

Note that we call that data "Ecoli" because we are using a subetwork with a topology of connetions borrowed from the Ecoli Gene Regulatory network. As we wanted to keep a set of perturbations that was similar to those of DREAM2 In Silico Challenge, we abused notation and called the data heterozygous mutant (which should be read: transcription rate for that gene is half the wild type transcription rate) even to the networks with topology borrowed from Ecoli. This is the "freedom" given by the InSilico world but of course, Ecoli is haploid and the "heterozygous" data wouldn't make sense in real life for E. coli.

In all cases, the data corresponds to noisy measurements of mRNA levels, which have been normalized such that the maximum normalized gene expression value in a given dataset is one. These datasets can be downloaded from [http://wiki.c2b2.columbia.edu/dream/data/DREAM3 the DREAM3 data repository], after you have proceeded with the [http://wiki.c2b2.columbia.edu/dream/register registration to the challenge].

== Submission Information ==

The same submission format and scoring metrics as in the DREAM2 challenges are used. However, this year all predictions must be directed and unsigned. Important: there are no self-interactions (auto-regulatory loops) in the gold standard networks.

Submit a ranked list of regulatory link predictions ordered according to the confidence you assign to the predictions, from the most reliable (first row) to the least reliable (last row) prediction. Use a 3 tab-separated column format as in the example below:

A \tab B \tab XYZ

where A and B are two different genes (no self-interactions). Links are directed: the gene in the first column regulates the gene in the second column. (If both A regulates B and B regulates A, then both lines should be included.) XYZ is a score between 0 and 1 that indicates the confidence level you assign to the prediction. (E.g., XYZ = 1 if gene A is deemed to regulate gene B with highest confidence and XYZ = 0 if A is deemed not to directly regulate B. See [http://infoscience.epfl.ch/record/126163 Marbach et al. (2009)] for a discussion of how confidence levels could be derived from standard network predictions). All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

''TeamName_Challenge_Network.txt''

where TeamName is the name of the team with which you registered for the challenge, Challenge is either InSilicoSize10, InSilicoSize50, or InSilicoSize100, and Network is one of the five networks of the indicated challenge (Ecoli1, ..., Yeast3). As mentioned above, to participate in a challenge you need to submit predictions for all five networks of this challenge.

== Scoring Metrics ==

We will score the results using the area under the precision versus recall curve for the whole set of link predictions for a network. For the first k predictions (ranked by score, and for predictions with the same score, taken in the order they were submitted in the prediction files), precision is defined as the fraction of correct predictions to k, and recall is the proportion of correct predictions out of all the possible true connections. Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated.

Teams will be ranked according to their overall performance over the five networks of a challenge.

== How Were the ''in-silico'' Networks Generated? ==

Great care was taken to generate ''in-silico'' gene networks that are biologically plausible, both with respect to the network structure and the network dynamics. Network topologies were obtained by extracting sub-networks from the gene-to-gene interaction network of ''E.coli'' and ''S. cerevisiae''. Auto-regulatory interactions were removed, i.e., there are no self-interactions in the ''in-silico'' networks.

The dynamics of the networks were simulated using a detailed kinetic model based on one of several possible approaches for modeling gene regulation. Both independent and synergistic gene regulation occur in the networks.

Note that transcription and translation are modeled. However, the protein concentrations are not included in the provided datasets. As mentioned above, the datasets correspond to the mRNA concentration levels, as one would obtain from gene expression data.

D3c3full

Summary:

= Gene Expression Prediction (DREAM3, Challenge 3) =

<div style="background-color:#FFFF99;width:100%">
This archival page describes the challenge exactly as it was presented to the participants. Go to the main [[D3c3|DREAM3 Challenge 3]] page to download data, view team rankings, cite this work, etc.
</div>

== Synopsis ==

Gene expression time course data is provided for four different strains of yeast (''S. Cerevisiae''), after perturbation of the cells. The challenge is to predict the rank order of induction/repression of a small subset of genes (the "prediction targets") in one of the four strains, given complete data for three of the strains, and data for all genes except the prediction targets in the other strain. You are also allowed to use any information that is in the public domain and are expected to be forthcoming about what information was used.

== Background ==

GAT1, GCN4, and LEU3 are yeast transcription factors. Each of these transcription factors has something to do with controlling genes involved in nitrogen or amino acid metabolism. The genes are not essential because strains that have perfect deletions of any of these genes are viable. In this challenge, we provide gene expression data from four strains: (i) a strain that is wild-type for all three transcription factors (wt, or parental), (ii) a strain that is identical to the parental strain except that it has a deletion of the GAT1 gene (gat1Δ), (iii) a strain that is identical to the parental strain except that it has a deletion of the GCN4 gene (gcn4Δ), and (iv) a strain that is identical to the parental strain except that it has a deletion of the the LEU3 gene (leu3Δ).

Expression levels were assayed separately in all four strains following the addition of 3-aminotriazole (3AT). 3AT is an inhibitor of an enzyme in the histidine biosynthesis pathway and, in the appropriate media (which is the case in these experiments) inhibition of the histidine biosynthetic pathway has the effect of starving the cells for this essential amino acid.

Data from eight time points was obtained from 0 to 120 minutes. Time t=0 means the absence of 3AT.

== The Challenge ==

Predict, for a set of 50 genes, the expression levels in the gat1Δ strain in the absence of 3-aminotriazole (t=0) and at 7 time points ( t=10, 20, 30, 45, 60, 90 and 120 minutes) following the addition of 3AT. Absolute expression levels are not required or desired; instead, the fifty genes should be ranked according to relative induction or repression relative to the expression levels observed in the wild-type parental strain in the absence of 3AT.

== The Datasets ==

The files provided for this challenge are detailed below.

The file ''DREAM3_GeneExpressionChallenge_TargetList.txt'' is a tab-delimited file that lists the target genes whose relative induction/repression are to be predicted. The first column lists the Affymetrix probeset IDs. The second column lists the corresponding commonly-used gene names, as extracted from files obtained from Affymetrix. This file should also be used as a template for submission of predictions. Consequently, there are headings for eight additional columns (see section on Format of Predictions).

The file ''DREAM3_GeneExpressionChallenge_ExpressionData.txt'' is a tab-delimited file that provides the relevant expression data. Columns are labeled, and are summarized here as well. The first column gives the Affymetrix probeset ID. The second column lists the commonly used gene name if there is one for that probeset. The third column represents the absolute expression level (in arbitrary units) for the probeset in the parental strain at time t=0. The next set of 8 columns contains the time course data for the wild-type strain, the following set of 8 columns contains the time course data for the gat1Δ strain, the next set of 8 columns contains the time course data for the gcn4Δ strain, and final set of 8 columns contains the time course data for the leu3Δ strain. Within each set of columns, the time points are t=0, 10, 20, 30, 45, 60, 90 and 120 minutes. The values in all of these columns express transcript levels as the log (base 2) of the ratio of expression in the indicated strain and time point to the expression level in the parental strain at time t=0. Thus, positive values indicate higher levels of expression than is observed for that probeset in the parental strain at time t=0, and negative values indicate lower expression. Data is provided for all probesets and in all strains, and at all time points, except for the 50 probesets (genes) whose expression is to be predicted (''DREAM3_GeneExpressionChallenge_TargetList.txt''). For those genes, the text "PREDICT" was inserted in the corresponding entries in the columns that correspond to the gat1Δ data in the file ''DREAM3_GeneExpressionChallenge_ExpressionData.txt''.

''PLEASE NOTE''. The data that is being provided initially is derived from two technical replicates, using a single biological replicate. An additional biological replicate will be obtained soon, and a new version of the ''DREAM3_GeneExpressionChallenge_ExpressionData.txt'' file will be provided.

<div><font color="#ff0000">'''UPDATE NOTE''' (July 15, 2008)</font></div>

As noted in the original posting of this challenge, the data set that was provided initially ''DREAM3_GeneExpressionChallenge_ExpressionData.txt'', was based on a single biological replicate, with two technical replicates. We noted that the data file was going to be updated as additional data were obtained. Challenge participants are hereby notified that the original data file has now been superseded by the file

''DREAM3_GeneExpressionChallenge_ExpressionData_UPDATED.txt''.

The values in this file are based on the original data, plus a new biological replicate. All array data been reprocessed using the RMA algorithm within the commercial program GeneSpring. Probeset hybridization values were median normalized within arrays prior to the calculation of fold-change. This is the dataset that will be used in the evaluation of challenge predictions.

== Submission Information ==

Predictors should make a copy of the file ''DREAM3_GeneExpressionChallenge_TargetlLst.txt'', and rename it

''TeamName_ExpressionChallenge.txt'',

where TeamName is the name of the team with which you registered for the challenge. Next to the first two columns, which list the probeIDs and gene names of the prediction targets, are eight tab-separated columns labeled "rank time0", "rank time10" and so on. The genes should be ranked according to predicted fold-induction relative to the expression level for that gene in the wild-type strain at time 0. The gene predicted to have the highest fold-induction should be given the value "1", and the gene with the greatest fold-repression should be given the value "50". All other genes should be given rank values in between.

== Scoring Metrics ==

Predictions will be assessed based on rank order metrics such as Spearman’s rank correlation coefficient, and its corresponding p-value under the null hypothesis that the ranks are randomly distributed.

D3c2full

Summary:

= Signaling Response Prediction (DREAM3, Challenge 2) =

<div style="background-color:#FFFF99;width:100%">
This archival page describes the challenge exactly as it was presented to the participants. Go to the main [[D3c2|DREAM3 Challenge 2]] page to download data, view team rankings, cite this work, etc.
If you use this data, please cite the updated versions of these References:

*Alexopoulos LG, Saez-Rodriguez J, Cosgrove B, Lauffenburger DA, Sorger P. Comparative pathway maps of normal and transformed human hepatocytes reveal widespread differences in inflammatory and NF-*B signaling. ''submitted (2009)''
*Saez-Rodriguez J, Alexopoulos L, Epperlein J, Samaga R, Lauffenburger DA, Klamt, S and Sorger PK (2009) "Discrete logic modeling as a means to link protein signaling networks with functional analysis of mammalian signal transduction." Mol Syst Biol in press (2009).

</div>

== Synopsis ==

Approximately 10,000 intracellular measurements (fluorescence signals proportional to the concentrations of phosphorylated proteins) and extracellular measurements (concentrations of cytokines released in response to cell stimulation) were acquired in human normal hepatocytes and the hepatocellular carcinoma cell line HepG2 cells. The datasets consist of measurements of 17 phospho-proteins (at 0 min, 30 min, and 3 hrs) and 20 cytokines (at 0 min, 3 hrs, and 24 hrs) in two cell types (normal and cancer) after perturbations to the pathway induced by the combinatorial treatment of 7 stimuli and 7 selective inhibitors.

== The Two Challenges ==

The goal of this signaling response challenge is to predict the response to perturbations of a signaling pathway in normal and cancer human hepatocytes. We have implemented two sub-challenges:

'''The phospho-proteomics challenge'''. This challenge consists of predicting a subset of data points that have been measured but removed from the normal and cancer hepatocytes datasets. Specifically, we ask the participants to predict the concentration of the 17 phospho-proteins at two time points (30 minutes and 3 hours) in each one of 7 combinations of ligands and inhibitors for both the normal and cancer hepatocytes. As data, we provide the concentrations of all those 17 phospho-proteins for all the other combinations of ligands and inhibitors for both the normal and cancer hepatocytes. The t=0 time point does not need to be predicted as it corresponds to the unstimulated condition (no stimulus was applied; only inhibitor). Therefore, for each inhibitor, the un-stimulated t=0 value for each phospho-protein is the same across data panels corresponding to different stimuli.

'''The cytokine-release challenge'''. This challenge consists of predicting a subset of data points that have been measured but removed from the normal and cancer hepatocytes datasets. Specifically, we ask the participants to predict the concentration of the 20 cytokines at two time points (3 and 24 hours) in each one of 7 combinations of ligands and inhibitors for both the normal and cancer hepatocytes. As data, we provide the concentrations of all those 20 cytokines for all the other combinations of ligands and inhibitors.for both the normal and cancer hepatocytes. The t=0 time point does not need to be predicted as it corresponds to the unstimulated condition (no stimulus was applied ; only inhibitor) Therefore, for each inhibitor, the un-stimulated t=0 value for each cytokine is the same across data panels corresponding to different stimuli.

== The Datasets ==

Human normal and cancer hepatocytes (cell line HepG2s) were treated with 7 stimuli (Table 1a) that are relevant to hepatocyte physiology. For each applied stimulus, 7 selective inhibitors (Table 1b) that block the activity of specific molecules have been applied independently (i.e., only one inhibitor at a time). For each combination of stimulus-inhibitor, the concentration of 17 intracellular phospho-protein molecules (Table 1c) were measured at three time points (0, 30min, 3hours) after stimulation. Also for each combination of stimulus-inhibitor the extra-cellular concentration of 20 cytokines (Table 1d) released by the cells were measured at 3 time points (0, 3hrs, 24hrs) after stimulation. The experimental design is shown schematically in Figure 1, where the data for either a phospho-protein or a cytokine data is exemplified.

[[Image:Tables.jpg|center]]

[[Image:Figure.JPG|center]]

The data is contained in two spreadsheets, one for the phosphorylation data (''PhosphoproteinChallenge_DREAM3.csv'') and one for the cytokine release data (''CytokineChallenge_DREAM3.csv''). The data is structured according to the following format: in both files the first column contains the cell type (Normal or Cancer), the second column specifies the stimulus, the third column lists the inhibitor, and the fourth column contains the time of data acquisition in minutes. From column 5 to 21, the file ''PhosphoproteinChallenge_DREAM3.csv'' contains the abundance of the 17 phospho-proteins in arbitrary fluorescence units and in the order given in Table 1c. From column 5 to 24, the file ''PhosphoproteinChallenge_DREAM3.csv'' contains the abundance of the 20 measured extracellular cytokines in arbitrary fluorescence units and in the order given in Table 1d. The values that have to be predicted have been replaced in the data files by the text: “PREDICT”.

'''Useful Information regarding measurements'''

(a) ''Data integrity / linearity''. Significant effort was dedicated to data integrity. The data are reported as arbitrary (fluorescence) units in the range between 0 and ~29000. The upper limit (~29000) corresponds to the saturation limit of the detector. Experiments were performed in such a way that measurements are as much as possible within the linear range of the detector. In general, data can be considered linear but there are a few cases that measurements are closer to the upper detection limit of ~29000 (e.g. some cJUN and IL8 measurements) where linearity might have been lost.

(b) ''Detection limits/Repeatability''. The coefficient of variation for repeated measurements was found to be ~8% (mostly due to biological error). With our current experimental design the instrument detector can report data with accuracy as low as ~300. For example, changes from 55 fluorescence units (FU) to 110 FU cannot be considered “2 fold increase” because values lie within the noise error of the detector. On the contrary, data from 1000 to 2000 are significant.

(c) ''Inhibitor effects''. There are cases in which our inhibitors (i.e. MEKi, p38i, and JNKi) target molecules whose phosphorylation we measure (i.e. MEK12, p38, and JNK). In the case where the inhibitor is present, the phosphorylation state of the corresponding molecule (i.e. phospho-MEK, phospho-p38, and phospho-JNK) should be assumed "absent" and the phosphorylation value should not be used. This known inhibitor effect is more pronounced on the allosteric inhibitors (i.e. the effect of MEK inhibitor on the MEK phosphorylation). The effects of the inhibitors are indirectly corroborated from the phosphorylation state of their downstream targets (i.e. MEK -> ERK, p38 -> HSP27, JNK -> cJUN).

'''Additional data'''

Any additional prior data already present in the literature can be used. This could be especially useful if a model of the network is needed as part of a method to predict the excluded data.

== Submission Information ==

The participants to this challenge should submit predictions for either the phospho-protein concentration subchallenge or the cytokine release subchallenge, or both. The submission format of the predictions should be as follows:

For the phospho-proteomics sub-challenge, predictors should make a copy of the ''DREAM3_PhosphoproteinChallenge_Predictions.csv'' file, and rename it

''TeamName__PhosphoproteinChallenge_Predictions.csv'',

where TeamName is the name of the team with which you registered for the challenge. Fill in the boxes replacing the text “PREDICT” with the best prediction for the phospho-protein indicated in the header row, for the Simulus/Inhibitor/Time of data acquisition indicated in each row. If you do not add the predicted values for any stimulus-inhibitor-time-phosphoprotein combination, we will consider that your prediction was random. Save your file in the comma separated values (csv) format.

For the Cytokine release prediction challenge, predictors should make a copy of the ''DREAM3_CytokineChallenge_Predictions.csv'' file, and rename it

''TeamName_CytokineChallenge_Predictions.csv'',

where TeamName is the name of the team with which you registered for the challenge. Fill in the boxes replacing the text “PREDICT” with the best prediction for the cytokine indicated in the header row, for the Simulus/Inhibitor/Time of data acquisition indicated in each line. If you do not add the predicted values for any stimulus-inhibitor-time-phosphoprotein combination, we will consider that your prediction was random. Save your file in the comma separated values (csv) format.

== Scoring Metrics ==

For the N predictions to be made in each of the challenges, we will compute the score

[[Image:Formula.jpg|center]]

A p-value will be assigned to each of the submissions both in the phosphoprotein concentration and the cytokine release predictions.

D3c4

Summary:

__NOTOC__
= DREAM3 In-Silico Network Challenge (DREAM3, Challenge 4) =

== Synopsis ==

The goal of the ''in silico'' challenges is the reverse engineering of gene networks from steady state and time series data. Participants are challenged to predict the directed unsigned network topology from the given ''in silico'' generated gene expression datasets.

== About the Data ==

These challenges have been provided by [http://lis.epfl.ch/136698 Daniel Marbach] and his colleagues from the [http://lis.epfl.ch Laboratory of Intelligent Systems] of the Swiss Federal Institute of Technology in Lausanne. '''The data can be freely used.''' Please cite the DREAM project and the following paper in your publications:

Marbach, D., Schaffter, T., Mattiussi, C. and Floreano, D. (2009) Generating Realistic ''in silico'' Gene Networks for Performance Assessment of Reverse Engineering Methods. ''Journal of Computational Biology'', 16(2) pp. 229-239. [[http://infoscience.epfl.ch/record/128148 detailed record]] [[http://infoscience.epfl.ch/getfile.py?docid=20591&name=Marbach2008c-preprint&format=pdf&version=1 preprint]] [[http://infoscience.epfl.ch/export.py?recid=128148&fm=bibtex bibtex]]

== Best Performer ==

* bteam: Kevin Yip, Roger Alexander, Koon-Kiu Yan, and Mark Gerstein, Yale University

== Results & Additional Information ==

The challenge of size 10 had 29 participants, the one of size 50 had 27 participants, and the one of size 100 had 22 participants. This makes these challenges currently the most widely used gene network reverse engineering benchmark.

The challenges have been generated with ''GeneNetWeaver'' (GNW). GNW allows one to easily generate additional benchamarks of the same type as the DREAM3 ''in silico'' challenges. GNW is available open source at: [http://gnw.sourceforge.net gnw.sourceforge.net].

Additional information (the datasets without noise, the signed network structures, etc.) is available at: [http://lis.epfl.ch/?content=research/projects/EvolutionOfAnalogNetworks/ReverseEngineeringGeneRegulatoryNetworks/DREAMChallenges.php DREAM3 ''in silico'' challenge additional information].

== Take Action ==

* [[d3c4full|Full Description (archival)]]
* [{{link}}/results/DREAM3/?c=4_1 Team Rankings]
* [{{link}}/data/DREAM3/ Download Data]
* [{{link}}/data/gold-standards/DREAM3/ Download Gold Standard]
* [{{link}}/data/scripts/DREAM3 Download Evaluation Scripts]
* Download Team Predictions (anonymous)

D3c3

Summary:

__NOTOC__
= Gene Expression Prediction (DREAM3, Challenge 3) =

== Synopsis ==

Gene expression time course data is provided for four different strains of yeast (''S. Cerevisiae''), after perturbation of the cells. The challenge is to predict the rank order of induction/repression of a small subset of genes (the "prediction targets") in one of the four strains, given complete data for three of the strains, and data for all genes except the prediction targets in the other strain. You are also allowed to use any information that is in the public domain and are expected to be forthcoming about what information was used.

== About the Data ==

* Data generously provided by [http://www.gis.a-star.edu.sg/internet/site/investigators_custom.php?user_id=70 Neil Clarke], Genome Institute of Singapore
* '''Data may not be published without permission from the data provider.'''

== Best Performer ==

* GustafssonHornquistSweden: Mika Gustafsson and Michael Hornquist, Linkoping University
* dreamteam2008: Jianhua Ruan, University of Texas at San Antonio

== Take Action ==

* [[d3c3full|Full Description (archival)]]
* [{{link}}/results/DREAM3/?c=3_1 Team Rankings]
* [{{link}}/data/DREAM3/ Download Data]
* [{{link}}/data/gold-standards/DREAM3/ Download Gold Standard]
* [{{link}}/data/scripts/DREAM3 Download Evaluation Scripts]
* Download Team Predictions (anonymous)

D3c2

Summary:

__NOTOC__
= Signaling Response Prediction (DREAM3, Challenge 2) =

== Synopsis ==

Approximately 10,000 intracellular measurements (fluorescence signals proportional to the concentrations of phosphorylated proteins) and extracellular measurements (concentrations of cytokines released in response to cell stimulation) were acquired in human normal hepatocytes and the hepatocellular carcinoma cell line HepG2 cells. The datasets consist of measurements of 17 phospho-proteins (at 0 min, 30 min, and 3 hrs) and 20 cytokines (at 0 min, 3 hrs, and 24 hrs) in two cell types (normal and cancer) after perturbations to the pathway induced by the combinatorial treatment of 7 stimuli and 7 selective inhibitors.

== About the Data ==
Data generously provided by [http://sysbio.med.harvard.edu/faculty/sorger/ Peter Sorger], Harvard Medical School / MIT

'''References:'''
*Alexopoulos LG, Saez-Rodriguez J, Cosgrove B, Lauffenburger DA, Sorger P. Comparative pathway maps of normal and transformed human hepatocytes reveal widespread differences in inflammatory and NF-*B signaling. ''submitted''
*Saez-Rodriguez J, Alexopoulos L, Epperlein J, Samaga R, Lauffenburger DA, Klamt, S and Sorger PK (2009) "Discrete logic modeling as a means to link protein signaling networks with functional analysis of mammalian signal transduction." Mol Syst Biol in press.

== Best Performer ==

* GenomeSingapore (Phospho-protein and Cytokine sub-challenges): Guillaume Bourque and Neil D. Clarke, Genome Institute of Singapore
* Vital_SIB (Phospho-protein sub-challenge): Nicolas Guex and Ioannis Xenarios, Swiss Institute of Bioinformatics

== Take Action ==

* [[d3c2full|Full Description (archival)]]
* [{{link}}/results/DREAM3/?c=2 Team Rankings]
* [{{link}}/data/DREAM3/ Download Data]
* [{{link}}/data/gold-standards/DREAM3/ Download Gold Standard]
* [{{link}}/data/scripts/DREAM3 Download Evaluation Scripts]
* Download Team Predictions (anonymous)

Reverse Engineering

Summary:

= Reverse Engineering =

* Encyclopedia of Algorithms
* Data Seeking Algorithms
* Algorithms Seeking Data

Link

Summary:

http://wiki.c2b2.columbia.edu/dream09

Stubs

Summary:

Victor, can you list the stubs to the non-wiki pages here? Thanks

* discuss [[discuss]]
*
*

D3c1full

Summary:

__NOTOC__
= Signaling Cascade Identification (DREAM3 / Challenge 1) =

<div style="background-color:#FFFF99;width:100%">
This archival page describes the challenge exactly as it was presented to the participants. Go to the main [[D3c1|DREAM3 Challenge 1]] page to download data, view team rankings, cite this work, etc.
</div>

== Synopsis ==

The concentration of four intracellular proteins or phospho-proteins (X1, X2, X3 and X4) participating in a signaling cascade were measured in about 10<sup>4</sup>. cells by antibody staining and flow cytometry. The idea of this challenge is to explore what key aspects of the dynamics and topology of interactions of a signaling cascade can be inferred from incomplete flow cytometry data.

== Challenge ==

We measured the concentration of four components in a portion of a signal transduction cascade in primary cells of undisclosed nature as shown in the Figure.

[[Image:Signaling_Cascade_Model_and_Challenge_Figure.jpg|center]]
[[Image:FigureLegend.jpg|center]]

These cells were activated with ligands for one of their membrane-bound receptors. Two types of ligands: weak and strong (i.e., with different potency), and in different quantities, have been used. The data sets explained in the next section describe measurements on individual cells by antibody staining and flow cytometry for different proteins. The challenge is as follows: given the measurements of components X1, X2, X3 and X4 determine which of the following functions: kinase, protein, phosphorylated protein, phosphatase, activated phosphatase and phosphorylated complex in the signaling cascade of the figure correspond to the measured molecular species X1, …, X4. In other words: is X1 the activated phosphatase? Is X2 the kinase? And so on.

== Datasets ==

We obtained a series of measurements of the signal transduction in primary cells of undisclosed nature. The data sets offered to the participants compile measurements on individual cells by antibody staining and flow cytometry for 4 different proteins or phopshoproteins. Depending on the availability of reagents, each file contains measurements on 10<sup>4</sup> to 10<sup>5</sup> cells for the levels of protein Xi (i=1…4) at 5min of activation for one strong ligand and seven quantities and one weak ligand and five quantites. Each row represents an individual cell, and each entry indicates the fluorescence level in arbitrary units. The measurements were taken in the range in which fluorescence is linear with concentration.

The names and contents of the files are as follows.

''StrongLigand_j_Xi_X4.csv''

(where the extension csv stands for comma separated values) contain the values for the proteins Xi (first column) and X4 (second column), which were labeled in the same cell and measured simultaneously, but independently of the other proteins. It corresponds to the activation by 10<sup>7-j</sup> strong ligands (for 0<j<7) or no ligand (for j =7).

''WeakLigand_j_Xi_X4.csv''

(where the extension csv stands for comma separated values) contain the values for the proteins Xi (first column) and X4 (second column), which were labeled in the same cell and measured simultaneously, but independently of the other proteins. It corresponds to the activation by 10<sup>5-j</sup> strong ligands (for 0<j<5) or no ligand (for j=5).

== Submission of Predictions ==

Download the file ''DREAM3_SignalingCascadeChallenge.xls''. This file contains a header line and four rows corresponding to each of the four measured proteins X1, …, X4, as follows:

[[Image:Challenge1_Table.jpg|center]]

Complete the table by inserting a 1 in the box for the assignments considered correct and a 0 otherwise. For example, if Xi is deemed to be an activated phosphatase, then add a 1 in the corresponding box (intersection of row Xi and “activated phosphatase” column). Each row has to have exactly one 1 and six 0s. Each column can have either all zeros, or exactly one 1 and three 0s. After completing the table, save it in text (tab delimited) (*.txt) format as:

''TeamName_SignalingCascadeChallenge.txt'',

where ''TeamName'' is the name of the team with which you registered for the challenge. Unfilled boxes in the table will be considered to have a zero.

== Scoring Metrics ==

We will score the results based on the probability that a random assignment would result in the same number of correct assignments achieved in the actual prediction.

D3c1

Summary:

__NOTOC__
= Signaling Cascade Identification (DREAM3, Challenge 1) =

== Synopsis ==

The concentration of four intracellular proteins or phospho-proteins (X1, X2, X3 and X4) participating in a signaling cascade were measured in about 10<sup>4</sup> cells by antibody staining and flow cytometry. The idea of this challenge is to explore what key aspects of the dynamics and topology of interactions of a signaling cascade can be inferred from incomplete flow cytometry data.

== About the Data ==

* Data generously provided by [http://www.mskcc.org/mskcc/html/61284.cfm Gregoire Altan-Bonnet], Memorial Sloan-Kettering Cancer Center
* '''Reference:''' Feinerman O, Veiga J, Dorfman JR, Germain RN, Altan-Bonnet G. "Variability and robustness in T cell activation from regulated heterogeneity in protein levels." Science. 2008 Aug 22;321(5892):1081-4. [http://www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed&cmd=search&term=18719282 PubMed]

== Best Performer ==

* There was no best performer for this challenge.

== Take Action ==

* [[d3c1full|Full Challenge Description]] (archival)
* [{{link}}/results/DREAM3/?c=1 Team Rankings] (results)
* [{{link}}/data/DREAM3/ Download Training Data]
* [{{link}}/data/gold-standards/DREAM3/ Download Gold Standard]
* [{{link}}/data/scripts/DREAM3 Download Evaluation Scripts]
* Download Team Predictions (anonymous)

Re

Summary:

Home

Summary:

= Overview =
DREAM is a Dialogue for Reverse Engineering Assessments and Methods. The main objective is to catalyze the interaction between experiment and theory in the area of cellular network inference. The fundamental question for DREAM is simple: How can researchers assess how well they are describing the networks of interacting molecules that underlie biological systems? The answer is not so simple. Researchers have used a variety of algorithms to deduce the structure of very different biological and artificial networks, and evaluated their success using various metrics. What is still needed, and what DREAM aims to achieve, is a fair comparison of the strengths and weaknesses of the methods and a clear sense of the reliability of the network models they produce.

= Thrusts =
The DREAM project is composed of three interrelated thrusts.

* The organization of periodic [[Conferences]]
* The organization of Reverse-Engineering [[Challenges]]
* An online venue to [[Discuss]] reverse-engineering topics and curate [[Data]], [[Literature]], and Methods.

= Upcoming Conference and Challenges: DREAM4 (2009) =
The DREAM4 Conference is in the planning stages. The conference dates will be announced soon.

= Organizers =
* Gustavo Stolovitzky, IBM Computational Biology Center
* Andrea Califano, Columbia University

= Steering Committee =
Alexander Hartemink, Andre Levchenko, Benno Schwikowski, Diego Di Bernardo, Eran Segal, Fritz Roth, Hamid Bulouri, Harmen Bussemaker, Jim Collins, Joel Bader, John Moult, Marc Vidal, Mark Gerstein, Mike Snyder, Mike Yaffee, Pedro Mendes, Ron Shamir, Tim Gardner, Trey Ideker

= Sponsors =
* Columbia University Center for Multiscale Analysis Genomic and Cellular Networks (MAGNet)
* NIH Roadmap Initiative
* IBM Computational Biology Center
* The New York Academy of Sciences

Bobby

Summary:

* [[D4c1]]
* [[D4c2]]
* [[D4c3]]

* [[D4c1full]]
* [[D4c2full]]
* [[D4c3full]]

[[secret]]

== Experiments with links ==

* [[discuss]] "discuss" seems to resolve to the filesystem discuss/
* [[data]] "data" seems to resolve to the filesystem data/

* [{{link}}/data Data] uses a template to get to data/
* [{{link}}/data/results Results] uses a template to get to data/results/

* [{{link}}/data/results/dream3/?c=1 Results for DREAM3 Challenge 1] uses a templage to get to data/results/dream3/?c=1. Note that the URL for the results page script will probably change in the future because "results" are not really "data" and we want a standard URL for all results for all years. For example,

* results/?d=3&c=1 for DREAM3 Challenge1
* results/?d=4&c=2 for DREAM4 Challenge2

TODO:

* "you may / may not use this data set in a publication"
* journal article reference for each data set
* admin should always see what the team sees, plus more.

This is Bobby's page for website development and testing.

'''''For UNSIGNED submissions:'''''

:hello

::hello

:::hello

''hello''

'''hello'''

'''''hello'''''

-- [[stubs]]

== Notes ==

- remove "From Dream Initiative" on

every page

- Make title of page much bigger than

paragraph headings

add predictions to websi

* [[discuss]]

* [[home]]
* [[challenges]]

**DREAM3
*** [[d3c1]]
*** [[d3c2]]
*** [[d3c3]]
*** [[d3c4]]

**DREAM2
*** [[d2c1]]
*** [[d2c2]]
*** [[d2c3]]
*** [[d2c4]]

* [[conferences]]
* [[literature]]
* [[re]]

Conferences

Summary:

__NOTOC__
= Conferences =

== DREAM4 (2009) Conference and Challenges ==

* [[DREAM4conf| DREAM4 Conference Page]]

== DREAM3 Conference and Challenges, October 2008 ==

* [[DREAM3conf| DREAM3 Conference Page]] (archival)
* [http://www.nyas.org/recomb-dream DREAM3 Conference multimedia archive] (NYAS e-briefing)
* [http://compbio.mit.edu/recombsat RECOMB Regulatory Genomics / RECOMB Systems Biology / DREAM 2008 joint conference website]
* [http://compbio.mit.edu/recombsat/2008/Videos.html Streaming video of presentations]

== DREAM2 Conference and Challenges, December 2007 ==

* [[DREAM2conf| DREAM2 Conference Page]] (archival)
* [http://www.nyas.org/dream2007 DREAM2 Conference multimedia archive] (NYAS e-briefing)

== DREAM1 Conference, September 2006 ==

* [http://www.nyas.org/Publications/EBriefings/Detail.aspx?cid=40a1893d-6573-4ec9-90e9-39a04bbd9d40 DREAM1 Conference multimedia archive] (NYAS e-briefing)

== DREAM0 Initial Planning Meeting, March 2006 ==

* [http://www.nyas.org/dream DREAM Planning Meeting multimedia archive] (NYAS e-briefing)

Challenges

Summary:

__NOTOC__
= Challenges =

== DREAM4 (2009) (Submission window: September 15 - October 15, 2009)==

# [[d4c1|'''Peptide Recognition Domain (PRD) Specificity Prediction''']] - Predict protein-protein interactions at the level of binding domains and peptides
# [[d4c2|'''DREAM4 In Silico Network Challenge''']] - Infer simulated gene regulation networks and predict gene expression measurements
# [[D4c3|'''Predictive Signaling Network Modeling''']] - Predict phosphoprotein measurements using an interpretable, predictive network

: [{{link}}/data/DREAM4 Download DREAM4 Data] - ''website registration required''

== DREAM3 (2008) ==

# [[d3c1|Signaling Cascade Identification]] - Infer a signaling network from flow cytometery data
# [[d3c2|Signaling Response Prediction]] - Predict missing protein concentrations from a large corpus of measurements
# [[d3c3|Gene Expression Prediction]] - Predict missing gene expression measurements
# [[d3c4|DREAM3 In Silico Network Challenge]] - Infer simulated gene regulation networks

: [{{link}}/data/DREAM3 Download DREAM3 Data] - ''website registration required''

== DREAM2 (2007) ==

# [[d2c1|BCL6 Transcriptional Target Prediction]] - Predict the genes that a transcription factor binds to
# [[d2c2|Protein-Protein Interaction Network Inference]] - Predict a PPI network of 47 proteins
# [[d2c3|Synthetic Five-Gene Network Inference]] - Infer a gene regulation network from qPCR and microarray measurements
# [[d2c4|DREAM2 In Silico Network Challenge]] - Infer various network topologies from simulated "measurements"
# [[d2c5|Genome-Scale Network Inference]] - Reconstruct a genome scale regulatory network from a large collection of microarrays

: [{{link}}/data/DREAM2 Download DREAM2 Data] - ''website registration required''

OldHome

Summary:

{| style="width:100%; background:#fff; text-align:center; margin-top:+.9em; border:1px solid #ccc;"
|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">The DREAM3 Predictions and Network Inference Challenges</div>
<div style="font-size:150%; border:none; margin:1 1 0 1; padding:.5em .5em 0 .5em; color:#000;">[http://wiki.c2b2.columbia.edu/dream/data/results/dream3 View Results and scores of more than 400 predictions by 40 teams]</div>

|}

== Overview ==

DREAM is a Dialogue for Reverse Engineering Assessments and Methods. Its main objective is to catalyze the interaction between experiment and theory in the area of cellular network inference. The fundamental question for DREAM is simple: How can researchers assess how well they are describing the networks of interacting molecules that underlie biological systems? The answer is not so simple. Researchers have used a variety of algorithms to deduce the structure of very different biological and artificial networks, and evaluated their success using various metrics. What is still needed, and what DREAM aims to achieve, is a fair comparison of the strengths and weaknesses of the methods and a clear sense of the reliability of the network models they produce.

== DREAM thrusts ==

The DREAM project is composed of three interrelated thrusts.

* The organization of periodic [[Meetings | meetings]]
* The organization of [[DREAM Challenges|network inference challenges]]
* Provide a [http://wiki.c2b2.columbia.edu/dream/discuss discussion forum] for reverse engineering topics and the curation of a repository of
** [[The DREAM Project/Data | Data]]
** [[The DREAM Project/Literature | Literature]]
** [[Tools_and_Algorithms | Tools and Algorithms]]

== DREAM People ==
==== Organizers ====
* Gustavo Stolovitzky, IBM Computational Biology Center
* Andrea Califano, Columbia University
==== Steering Committee ====
Alexander Hartemink, Andre Levchenko, Benno Schwikowski, Diego Di Bernardo, Eran Segal, Fritz Roth, Hamid Bulouri, Harmen Bussemaker, Jim Collins, Joel Bader, John Moult, Marc Vidal, Mark Gerstein, Mike Snyder, Mike Yaffee, Pedro Mendes, Ron Shamir, Tim Gardner, Trey Ideker

== Add a comment in the new DREAM [http://wiki.c2b2.columbia.edu/dream/discuss Discussion Forum] ==

{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:10%; color:#000"| 
{| style="width:400px; border:solid 0px; background:none;"
|style="width:800px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:0.5em; color:#2f0;">DREAM [http://wiki.c2b2.columbia.edu/dream/discuss Discussion Forum] </div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="font-size:100%; padding:.3em;"> This forum is intended for discussion of anything related to Reverse Engineering in biological systems. Please feel free to comment on recent papers, interesting conferences or anything else related to pathway inference.
</div>
|}

== NEWS on the upcoming [[_The_3rd_DREAM_Conference | DREAM3 Conference]] ==

{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:10%; color:#000"|

{| style="width:600px; border:solid 0px; background:none;"
|style="width:600px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">The DREAM3 Predictions and Network Inference Challenges</div>
|}
|style="width:100%; padding:.1em; font-size:95%; text-align:center; color:#00f;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">[[The_DREAM3_Challenges|Download Challenges]]</div>
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;"> DREAM3 [http://wiki.c2b2.columbia.edu/dream/data/results/dream3/ Challenge Results] and [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold Standards]</div>
<small> You have to [http://wiki.c2b2.columbia.edu/dream/register/index.php?dream=3 register] to download the [[The_DREAM3_Challenges | DREAM3-challenges data]] and [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold Standards]</small>
|}

== DREAM Sponsors ==
* Columbia University Center for Multiscale Analysis Genomic and Cellular Networks (MAGNet)
* NIH Roadmap Initiative
* IBM Computational Biology Center
* The New York Academy of Sciences

== DREAM in the news ==
*In [http://www.bio-itworld.com/archive/silicobio/index_11132006.html BioIT World | Systems Biology ].

*In [http://www.bio-itworld.com/newsitems/2007/july/07-30-07-dream2 BioIT World.com ].

*In [http://biz.yahoo.com/iw/070815/0291204.html Yahoo! news]

*In [http://www.genengnews.com/news/bnitem.aspx?name=21826072 GEN news].

*In [http://online.wsj.com/public/article/PR-CO-20070815-903076-ZAWQmYvMXvTXp7X1V0_4b_t88j8_20080814.html?mod=crnews The Wall Street Journal online]

*[http://www.bio-itworld.com/issues/2008/july-august/russell-transcript.html <font color="ff0000">July 14, 2008</font>: In BioIT-World.com]

*[http://www.bio-itworld.com/full_newsletter.aspx?id=78714 <font color="ff0000">July 16, 2008</font>: In John Russell's Systems Biology newsletter]

*[http://www.genengnews.com/articles/chitem.aspx?aid=2630 <font color="ff0000">October 15, 2008</font>: "Deciphering Biological Networks", in Genetics Engineering and Biotechnology News]

*[http://www.bio-itworld.com/pb/2008/10/23/dream3-results.html <font color="ff0000">October 23, 2008</font>: "DREAM3 Predictions (and Their Grades) Are In", In Predictive Medicine newsletter]

*[http://www.bio-itworld.com/pb/2008/12/04/interpreting-dream3.html <font color="ff0000">December 4, 2008</font>: " Interpreting DREAM3", In Bio-ITWorld.com]

About DREAM

Summary:

== Overview ==

DREAM is a Dialogue for Reverse Engineering Assessments and Methods. Its main objective is to catalyze the interaction between experiment and theory in the area of cellular network inference. The fundamental question for DREAM is simple: How can researchers assess how well they are describing the networks of interacting molecules that underlie biological systems? The answer is not so simple. Researchers have used a variety of algorithms to deduce the structure of very different biological and artificial networks, and evaluated their success using various metrics. What is still needed, and what DREAM aims to achieve, is a fair comparison of the strengths and weaknesses of the methods and a clear sense of the reliability of the network models they produce.

== DREAM thrusts ==

The DREAM project is composed of three interrelated thrusts.

* The organization of periodic [[Meetings | meetings]]
* The organization of [[DREAM Challenges|network inference challenges]]
* Provide a [http://wiki.c2b2.columbia.edu/dream/discuss discussion forum] for reverse engineering topics and the curation of a repository of
** [[The DREAM Project/Data | Data]]
** [[The DREAM Project/Literature | Literature]]
** [[Tools_and_Algorithms | Tools and Algorithms]]

== DREAM People ==
==== Organizers ====
* Gustavo Stolovitzky, IBM Computational Biology Center
* Andrea Califano, Columbia University
==== Steering Committee ====
Alexander Hartemink, Andre Levchenko, Benno Schwikowski, Diego Di Bernardo, Eran Segal, Fritz Roth, Hamid Bulouri, Harmen Bussemaker, Jim Collins, Joel Bader, John Moult, Marc Vidal, Mark Gerstein, Mike Snyder, Mike Yaffee, Pedro Mendes, Ron Shamir, Tim Gardner, Trey Ideker

== Add a comment in the new DREAM [http://wiki.c2b2.columbia.edu/dream/discuss Discussion Forum] ==

{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:10%; color:#000"| 
{| style="width:400px; border:solid 0px; background:none;"
|style="width:800px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:0.5em; color:#2f0;">DREAM [http://wiki.c2b2.columbia.edu/dream/discuss Discussion Forum] </div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="font-size:100%; padding:.3em;"> This forum is intended for discussion of anything related to Reverse Engineering in biological systems. Please feel free to comment on recent papers, interesting conferences or anything else related to pathway inference.
</div>
|}

== NEWS on the upcoming [[_The_3rd_DREAM_Conference | DREAM3 Conference]] ==

{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:10%; color:#000"|

{| style="width:600px; border:solid 0px; background:none;"
|style="width:600px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">The DREAM3 Predictions and Network Inference Challenges</div>
|}
|style="width:100%; padding:.1em; font-size:95%; text-align:center; color:#00f;"|
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;">[[The_DREAM3_Challenges|Download Challenges]]</div>
<div style="font-size:150%; border:none; margin:1; padding:.5em; color:#000;"> DREAM3 [http://wiki.c2b2.columbia.edu/dream/data/results/dream3/ Challenge Results] and [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold Standards]</div>
<small> You have to [http://wiki.c2b2.columbia.edu/dream/register/index.php?dream=3 register] to download the [[The_DREAM3_Challenges | DREAM3-challenges data]] and [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold Standards]</small>
|}

== DREAM Sponsors ==
* Columbia University Center for Multiscale Analysis Genomic and Cellular Networks (MAGNet)
* NIH Roadmap Initiative
* IBM Computational Biology Center
* The New York Academy of Sciences

== DREAM in the news ==
*In [http://www.bio-itworld.com/archive/silicobio/index_11132006.html BioIT World | Systems Biology ].

*In [http://www.bio-itworld.com/newsitems/2007/july/07-30-07-dream2 BioIT World.com ].

*In [http://biz.yahoo.com/iw/070815/0291204.html Yahoo! news]

*In [http://www.genengnews.com/news/bnitem.aspx?name=21826072 GEN news].

*In [http://online.wsj.com/public/article/PR-CO-20070815-903076-ZAWQmYvMXvTXp7X1V0_4b_t88j8_20080814.html?mod=crnews The Wall Street Journal online]

*[http://www.bio-itworld.com/issues/2008/july-august/russell-transcript.html <font color="ff0000">July 14, 2008</font>: In BioIT-World.com]

*[http://www.bio-itworld.com/full_newsletter.aspx?id=78714 <font color="ff0000">July 16, 2008</font>: In John Russell's Systems Biology newsletter]

*[http://www.genengnews.com/articles/chitem.aspx?aid=2630 <font color="ff0000">October 15, 2008</font>: "Deciphering Biological Networks", in Genetics Engineering and Biotechnology News]

*[http://www.bio-itworld.com/pb/2008/10/23/dream3-results.html <font color="ff0000">October 23, 2008</font>: "DREAM3 Predictions (and Their Grades) Are In", In Predictive Medicine newsletter]

*[http://www.bio-itworld.com/pb/2008/12/04/interpreting-dream3.html <font color="ff0000">December 4, 2008</font>: " Interpreting DREAM3", In Bio-ITWorld.com]

NewHome

Summary:

== Results of the DREAM 2008 Conference and Challenges ==

The DREAM 2008 Conference has passed. At this time, you can:

* View the [http://wiki.c2b2.columbia.edu/dream/data/results/dream3/?c=1 team rankings in the 2008 Reverse Engineering Challenges]

* Read about the [http://wiki.c2b2.columbia.edu/dream/index.php/The_DREAM3_Challenges DREAM 2008 Challenges]

* Download the [http://wiki.c2b2.columbia.edu/dream/index.php/The_DREAM3_Challenges DREAM 2008 Challenges] to self-assess the accuracy of your algorithms

* Ask a question or contribute an answer on the [http://wiki.c2b2.columbia.edu/dream/discuss Discussion Board]

* Watch [http://compbio.mit.edu/recombsat/2008/Videos.html streaming video] of the 2008 conference presentations

* Visit the joint [http://compbio.mit.edu/recombsat/ RECOMB Regulatory Genomics / RECOMB Systems Biology / DREAM 2008] conference website

== What is DREAM? ==

'''DREAM is a Dialogue for Reverse Engineering Assessments and Methods'''. Its main objective is to catalyze the interaction between experiment and theory in the area of cellular network inference. The fundamental question for DREAM is simple: '''How can researchers assess how well they are describing the networks of interacting molecules that underlie biological systems?''' The answer is not so simple. Researchers have used a variety of algorithms to deduce the structure of very different biological and artificial networks, and evaluated their success using various metrics. What is still needed, and what DREAM aims to achieve, is a fair comparison of the strengths and weaknesses of the methods and a clear sense of the reliability of the network models they produce.

Read more [[About DREAM]].

== New Website ==

DREAM has outgrown it's humble website. '''We would like your suggestions''' for the purpose, content, and design of the new website. Potential features include:

* Encyclopedia of reverse engineering algorithms

* Team pages for describing methods and blatant self-promotion

* Discussion forum

* Repository of data sets

* Conference proceedings

Please leave a comment on the [http://wiki.c2b2.columbia.edu/dream/discuss Discussion Board] if you have suggestions.

== Previous DREAM Conferences ==

The New York Academy of Sciences (NYAS) produced an accompanying "e-briefing" for each of the previous meetings and conferences. The NYAS e-briefings include multimedia presentations of the presenters' talks.

*'''2007 Conference and Challenges'''

:* View the team rankings in the 2007 Reverse Engineering Challenges

:* Read about the 2007 Challenges

:* Visit the [http://www.nyas.org/ebriefreps/splash.asp?intEbriefID=705 NYAS e-briefing for the 2007 (2nd) DREAM Conference]

*'''2006 Conference'''

:*Visit the [http://www.nyas.org/ebriefreps/splash.asp?intEbriefID=596&PartnerCD=DREAM2&TrackCD=eB596 NYAS e-briefing for the 2006 (1st) DREAM Conference]

*'''2006 Planning Meeting'''

:*Visit the [http://www.nyas.org/ebriefreps/splash.asp?intEbriefID=534&PartnerCD=DREAMteam&TrackCD=eB534 NYAS e-briefing for the 2006 DREAM Planning Meeting]

== Organizers==
* Gustavo Stolovitzky, IBM Computational Biology Center
* Andrea Califano, Columbia University

== Steering Committee ==
Alexander Hartemink, Andre Levchenko, Benno Schwikowski, Diego Di Bernardo, Eran Segal, Fritz Roth, Hamid Bulouri, Harmen Bussemaker, Jim Collins, Joel Bader, John Moult, Marc Vidal, Mark Gerstein, Mike Snyder, Mike Yaffee, Pedro Mendes, Ron Shamir, Tim Gardner, Trey Ideker

== Sponsors ==

* Columbia University Center for Multiscale Analysis Genomic and Cellular Networks (MAGNet)

* NIH Roadmap Initiative

* IBM Computational Biology Center

* The New York Academy of Sciences

== In the News ==

*In [http://www.bio-itworld.com/archive/silicobio/index_11132006.html BioIT World | Systems Biology ].

*In [http://www.bio-itworld.com/newsitems/2007/july/07-30-07-dream2 BioIT World.com ].

*In [http://biz.yahoo.com/iw/070815/0291204.html Yahoo! news]

*In [http://www.genengnews.com/news/bnitem.aspx?name=21826072 GEN news].

*In [http://online.wsj.com/public/article/PR-CO-20070815-903076-ZAWQmYvMXvTXp7X1V0_4b_t88j8_20080814.html?mod=crnews The Wall Street Journal online]

*[http://www.bio-itworld.com/issues/2008/july-august/russell-transcript.html <font color="ff0000">July 14, 2008</font>: In BioIT-World.com]

*[http://www.bio-itworld.com/full_newsletter.aspx?id=78714 <font color="ff0000">July 16, 2008</font>: In John Russell's Systems Biology newsletter]

*[http://www.genengnews.com/articles/chitem.aspx?aid=2630 <font color="ff0000">October 15, 2008</font>: "Deciphering Biological Networks", in Genetics Engineering and Biotechnology News]

*[http://www.bio-itworld.com/pb/2008/10/23/dream3-results.html <font color="ff0000">October 23, 2008</font>: "DREAM3 Predictions (and Their Grades) Are In", In Predictive Medicine newsletter]

*[http://www.bio-itworld.com/pb/2008/12/04/interpreting-dream3.html <font color="ff0000">December 4, 2008</font>: " Interpreting DREAM3", In Bio-ITWorld.com]

Best Performers

Summary:

Vetria wants to '''edit'''

The DREAM3 In-Silico-Network Challenges. Description

Summary:

The goal of the ''in silico'' challenges is the reverse engineering of gene networks from steady state and time series data. Participants are challenged to predict the directed unsigned network topology from the given ''in silico'' generated gene expression datasets.

These challenges have been provided by [http://lis.epfl.ch/136698 Daniel Marbach] and his colleagues from the [http://lis.epfl.ch Laboratory of Intelligent Systems] of the Swiss Federal Institute of Technology in Lausanne. '''The data can be freely used.''' Please cite the DREAM project and the following paper in your publications:

Marbach, D., Schaffter, T., Mattiussi, C. and Floreano, D. (2009) Generating Realistic ''in silico'' Gene Networks for Performance Assessment of Reverse Engineering Methods. ''Journal of Computational Biology''. ''To appear''. [[http://infoscience.epfl.ch/record/128148 detailed record]] [[http://infoscience.epfl.ch/getfile.py?docid=20591&name=Marbach2008c-preprint&format=pdf&version=1 preprint]] [[http://infoscience.epfl.ch/export.py?recid=128148&fm=bibtex bibtex]]

==The Three Challenges ==

There are three ''in-silico'' challenges corresponding to gene networks with 10, 50, and 100 genes. Predictions are assessed independently for each challenge. Thus, teams may choose to submit predictions only for one or two of the challenges. However, we encourage teams to participate in all three challenges in order to compare how well different methods perform on different network sizes.

Each challenge consists of five gold standard networks. In order to participate in a challenge, predictions for all five networks of this challenge must be submitted. The rational is that in this way it will be possible to assess how consistently a method predicts the topology in five independent networks of the same type and size.

==The Datasets ==

For consistency, we provide the same type of data as in the [[The_In-Silico-Network_Challenges._Description | DREAM2 ''in-silico'' Challenge]]. For every network, the following experiments are simulated:

'''Heterozygous knock-down'''. The files ''*-heterozygous.tsv'' (the meaning of the wild card * will be explained lines below) contain the steady state levels for the wild-type and the heterozygous knock-down strains for each gene (+/-). Thus, for a network of size N there are N+1 experiments (wild-type plus knock-down of every gene).

'''Null-mutants'''. The files ''*-null-mutants.tsv'' contain the steady state levels for the wild-type and the null-mutant strains for each gene (-/-). Thus, for a network of size N there are N+1 experiments (wild-type plus knock-out of every gene).

'''Trajectories'''. The files ''*-trajectories.tsv'' contain time courses of the network recovering from several external perturbations. For the networks of size 50, the same number of time courses as in the DREAM2 in silico challenge are provided (23 different perturbations). For the networks of size 10 and 100, we give 4 and 46 perturbations respectively (each one with 21 time points).

The * in front of ''*–heterozygous.tsv'', ''*-null-mutants.tsv'' and ''*-trajectories.tsv'' can take the values *=InSilicoSizeN-OraganismK, where N=10, 50, or 100, Organism is Ecoli or Yeast and K = 1 or 2 if Organism is Ecoli, and K=1, 2, or 3 if Organism is Yeast.

Note that we call that data "Ecoli" because we are using a subetwork with a topology of connetions borrowed from the Ecoli Gene Regulatory network. As we wanted to keep a set of perturbations that was similar to those of DREAM2 In Silico Challenge, we abused notation and called the data heterozygous mutant (which should be read: transcription rate for that gene is half the wild type transcription rate) even to the networks with topology borrowed from Ecoli. This is the "freedom" given by the InSilico world but of course, Ecoli is haploid and the "heterozygous" data wouldn't make sense in real life for E. coli.

In all cases, the data corresponds to noisy measurements of mRNA levels, which have been normalized such that the maximum normalized gene expression value in a given dataset is one. These datasets can be downloaded from [http://wiki.c2b2.columbia.edu/dream/data/DREAM3 the DREAM3 data repository], after you have proceeded with the [http://wiki.c2b2.columbia.edu/dream/register registration to the challenge].

==Submission Information==

The same submission format and scoring metrics as in the DREAM2 challenges are used. However, this year all predictions must be directed and unsigned. Important: there are no self-interactions (auto-regulatory loops) in the gold standard networks.

Submit a ranked list of regulatory link predictions ordered according to the confidence you assign to the predictions, from the most reliable (first row) to the least reliable (last row) prediction. Use a 3 tab-separated column format as in the example below:

A \tab B \tab XYZ

where A and B are two different genes (no self-interactions). Links are directed: the gene in the first column regulates the gene in the second column. (If both A regulates B and B regulates A, then both lines should be included.) XYZ is a score between 0 and 1 that indicates the confidence level you assign to the prediction. (E.g., XYZ = 1 if gene A is deemed to regulate gene B with highest confidence and XYZ = 0 if A is deemed not to directly regulate B. See [http://infoscience.epfl.ch/record/126163 Marbach et al. (2008)] for a discussion of how confidence levels could be derived from standard network predictions). All pairs omitted from the list will be considered to appear randomly ordered at the end of the list with XYZ = 0. Save the file as text, and name it:

''TeamName_Challenge_Network.txt''

where TeamName is the name of the team with which you registered for the challenge, Challenge is either InSilicoSize10, InSilicoSize50, or InSilicoSize100, and Network is one of the five networks of the indicated challenge (Ecoli1, ..., Yeast3). As mentioned above, to participate in a challenge you need to submit predictions for all five networks of this challenge.

==Scoring Metrics==

We will score the results using the area under the precision versus recall curve for the whole set of link predictions for a network. For the first k predictions (ranked by score, and for predictions with the same score, taken in the order they were submitted in the prediction files), precision is defined as the fraction of correct predictions to k, and recall is the proportion of correct predictions out of all the possible true connections. Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated.

Teams will be ranked according to their overall performance over the five networks of a challenge.

==How Were the ''in-silico'' Networks Generated?==

Great care was taken to generate ''in-silico'' gene networks that are biologically plausible, both with respect to the network structure and the network dynamics. Network topologies were obtained by extracting sub-networks from the gene-to-gene interaction network of ''E.coli'' and ''S. cerevisiae''. Auto-regulatory interactions were removed, i.e., there are no self-interactions in the ''in-silico'' networks.

The dynamics of the networks were simulated using a detailed kinetic model based on one of several possible approaches for modeling gene regulation. Both independent and synergistic gene regulation occur in the networks.

Note that transcription and translation are modeled. However, the protein concentrations are not included in the provided datasets. As mentioned above, the datasets correspond to the mRNA concentration levels, as one would obtain from gene expression data.

==Results & Additional Information==

The challenge of size 10 had 29 participants, the one of size 50 had 27 participants, and the one of size 100 had 22 participants. This makes these challenges currently the most widely used gene network reverse engineering benchmark.

We would like to thank all participating teams and congratulate the team that achieved the best performance on all network sizes: [http://bioinfo.mbb.yale.edu/ Kevin Y. Yip, Roger P. Alexander, Koon-Kiu Yan, and Mark Gerstein from Yale University]. You can now view the [http://wiki.c2b2.columbia.edu/dream/data/results/dream3/ detailed results of all teams] and the [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ true structure of the networks].

The challenges have been generated with ''GeneNetWeaver'' (GNW). GNW allows one to easily generate additional benchamarks of the same type as the DREAM3 ''in silico'' challenges. GNW is available open source at: [http://gnw.sourceforge.net gnw.sourceforge.net].

Additional information (the datasets without noise, the signed network structures, etc.) is available at: [http://lis.epfl.ch/?content=research/projects/EvolutionOfAnalogNetworks/ReverseEngineeringGeneRegulatoryNetworks/DREAMChallenges.php DREAM3 ''in silico'' challenge additional information].

==Quick Links==

[http://wiki.c2b2.columbia.edu/dream/data/DREAM3 Data Download]

[http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold Standards]

[http://wiki.c2b2.columbia.edu/dream/data/results/dream3/ Results]

[http://lis.epfl.ch/?content=research/projects/EvolutionOfAnalogNetworks/ReverseEngineeringGeneRegulatoryNetworks/DREAMChallenges.php Additional information]

The Gene-Expression Prediction Challenge. Description

Summary:

Gene expression time course data is provided for four different strains of yeast (''S. Cerevisiae''), after perturbation of the cells. The challenge is to predict the rank order of induction/repression of a small subset of genes (the “prediction targets” in one of the four strains, given complete data for three of the strains, and data for all genes except the prediction targets in the other strain. Predictors are also allowed to use any information that is in the public domain but are expected to be forthcoming about what information was used.

==Background ==

GAT1, GCN4, and LEU3 are yeast transcription factors. Each of these transcription factors has something to do with controlling genes involved in nitrogen or amino acid metabolism. The genes are not essential because strains that have perfect deletions of any of these genes are viable. In this challenge, we provide gene expression data from four strains: (i) a strain that is wild-type for all three transcription factors (wt, or parental), (ii) a strain that is identical to the parental strain except that it has a deletion of the GAT1 gene (gat1Δ), (iii) a strain that is identical to the parental strain except that it has a deletion of the GCN4 gene (gcn4Δ), and (iv) a strain that is identical to the parental strain except that it has a deletion of the the LEU3 gene (leu3Δ).

Expression levels were assayed separately in all four strains following the addition of 3-aminotriazole (3AT). 3AT is an inhibitor of an enzyme in the histidine biosynthesis pathway and, in the appropriate media (which is the case in these experiments) inhibition of the histidine biosynthetic pathway has the effect of starving the cells for this essential amino acid.

Data from eight time points was obtained from 0 to 120 minutes. Time t=0 means the absence of 3AT.

==The Challenge==

Predict, for a set of 50 genes, the expression levels in the gat1Δ strain in the absence of 3-aminotriazole (t=0) and at 7 time points ( t=10, 20, 30, 45, 60, 90 and 120 minutes) following the addition of 3AT. Absolute expression levels are not required or desired; instead, the fifty genes should be ranked according to relative induction or repression relative to the expression levels observed in the wild-type parental strain in the absence of 3AT.

==The Datasets ==

The files provided for this challenge are detailed below.

The file ''DREAM3_GeneExpressionChallenge_TargetList.txt'' is a tab-delimited file that lists the target genes whose relative induction/repression are to be predicted. The first column lists the Affymetrix probeset IDs. The second column lists the corresponding commonly-used gene names, as extracted from files obtained from Affymetrix. This file should also be used as a template for submission of predictions. Consequently, there are headings for eight additional columns (see section on Format of Predictions).

The file ''DREAM3_GeneExpressionChallenge_ExpressionData.txt'' is a tab-delimited file that provides the relevant expression data. Columns are labeled, and are summarized here as well. The first column gives the Affymetrix probeset ID. The second column lists the commonly used gene name if there is one for that probeset. The third column represents the absolute expression level (in arbitrary units) for the probeset in the parental strain at time t=0. The next set of 8 columns contains the time course data for the wild-type strain, the following set of 8 columns contains the time course data for the gat1Δ strain, the next set of 8 columns contains the time course data for the gcn4Δ strain, and final set of 8 columns contains the time course data for the leu3Δ strain. Within each set of columns, the time points are t=0, 10, 20, 30, 45, 60, 90 and 120 minutes. The values in all of these columns express transcript levels as the log (base 2) of the ratio of expression in the indicated strain and time point to the expression level in the parental strain at time t=0. Thus, positive values indicate higher levels of expression than is observed for that probeset in the parental strain at time t=0, and negative values indicate lower expression. Data is provided for all probesets and in all strains, and at all time points, except for the 50 probesets (genes) whose expression is to be predicted (''DREAM3_GeneExpressionChallenge_TargetList.txt''). For those genes, the text “PREDICT” was inserted in the corresponding entries in the columns that correspond to the gat1Δ data in the file ''DREAM3_GeneExpressionChallenge_ExpressionData.txt''.

''PLEASE NOTE''. The data that is being provided initially is derived from two technical replicates, using a single biological replicate. An additional biological replicate will be obtained soon, and a new version of the ''DREAM3_GeneExpressionChallenge_ExpressionData.txt'' file will be provided.

<div><font color="#ff0000">'''UPDATE NOTE''' (July 15, 2008)</font></div>

As noted in the original posting of this challenge, the data set that was provided initially ''DREAM3_GeneExpressionChallenge_ExpressionData.txt'', was based on a single biological replicate, with two technical replicates. We noted that the data file was going to be updated as additional data were obtained. Challenge participants are hereby notified that the original data file has now been superseded by the file

''DREAM3_GeneExpressionChallenge_ExpressionData_UPDATED.txt''.

The values in this file are based on the original data, plus a new biological replicate. All array data been reprocessed using the RMA algorithm within the commercial program GeneSpring. Probeset hybridization values were median normalized within arrays prior to the calculation of fold-change. This is the dataset that will be used in the evaluation of challenge predictions.

==Submission Information==

Predictors should make a copy of the file ''DREAM3_GeneExpressionChallenge_TargetlLst.txt'', and rename it

''TeamName_ExpressionChallenge.txt'',

where TeamName is the name of the team with which you registered for the challenge. Next to the first two columns, which list the probeIDs and gene names of the prediction targets, are eight tab-separated columns labeled “rank time0”, “rank time10” and so on. The genes should be ranked according to predicted fold-induction relative to the expression level for that gene in the wild-type strain at time 0. The gene predicted to have the highest fold-induction should be given the value “1”, and the gene with the greatest fold-repression should be given the value “50”. All other genes should be given rank values in between.

==Scoring Metrics==

Predictions will be assessed based on rank order metrics such as Spearman’s rank correlation coefficient, and its corresponding p-value under the null hypothesis that the ranks are randomly distributed.

==Data Download==

{| style="width:100%; background:#fff; text-align:center; margin-top:+.9em; border:1px solid #ccc;"
|
<div style="font-size:150%; border:20px; margin:1; color:#000;">[http://wiki.c2b2.columbia.edu/dream/data/DREAM3 DREAM3 Data Download]</div>
|}

The Signaling-Response Prediction Challenge. Description

Summary:

Approximately 10,000 intracellular measurements (fluorescence signals proportional to the concentrations of phosphorylated proteins) and extracellular measurements (concentrations of cytokines released in response to cell stimulation) were acquired in human normal hepatocytes and the hepatocellular carcinoma cell line HepG2 cells. The datasets consist of measurements of 17 phospho-proteins (at 0, 30min, and 3 hrs) and 20 cytokines (at 0, 3hrs, and 24hrs) in two cell types (normal and cancer) after perturbations to the pathway induced by the combinatorial treatment of 7 stimuli and 7 selective inhibitors.

==The Two Challenges ==

The goal of this signaling response challenge is to predict the response to perturbations of a signaling pathway in normal and cancer human hepatocytes. We have implemented two sub-challenges:

'''The phospho-proteomics challenge'''. This challenge consists of predicting a subset of data points that have been measured but removed from the normal and cancer hepatocytes datasets. Specifically, we ask the participants to predict the concentration of the 17 phospho-proteins at two time points (30 minutes and 3 hours) in each one of 7 combinations of ligands and inhibitors for both the normal and cancer hepatocytes. As data, we provide the concentrations of all those 17 phospho-proteins for all the other combinations of ligands and inhibitors for both the normal and cancer hepatocytes. The t=0 time point does not need to be predicted as it corresponds to the unstimulated condition (no stimulus was applied; only inhibitor). Therefore, for each inhibitor, the un-stimulated t=0 value for each phospho-protein is the same across data panels corresponding to different stimuli.

'''The cytokine-release challenge'''. This challenge consists of predicting a subset of data points that have been measured but removed from the normal and cancer hepatocytes datasets. Specifically, we ask the participants to predict the concentration of the 20 cytokines at two time points (3 and 24 hours) in each one of 7 combinations of ligands and inhibitors for both the normal and cancer hepatocytes. As data, we provide the concentrations of all those 20 cytokines for all the other combinations of ligands and inhibitors.for both the normal and cancer hepatocytes. The t=0 time point does not need to be predicted as it corresponds to the unstimulated condition (no stimulus was applied ; only inhibitor) Therefore, for each inhibitor, the un-stimulated t=0 value for each cytokine is the same across data panels corresponding to different stimuli.

==The Datasets ==

Human normal and cancer hepatocytes (cell line HepG2s) were treated with 7 stimuli (Table 1a) that are relevant to hepatocyte physiology. For each applied stimulus, 7 selective inhibitors (Table 1b) that block the activity of specific molecules have been applied independently (i.e., only one inhibitor at a time). For each combination of stimulus-inhibitor, the concentration of 17 intracellular phospho-protein molecules (Table 1c) were measured at three time points (0, 30min, 3hours) after stimulation. Also for each combination of stimulus-inhibitor the extra-cellular concentration of 20 cytokines (Table 1d) released by the cells were measured at 3 time points (0, 3hrs, 24hrs) after stimulation. The experimental design is shown schematically in Figure 1, where the data for either a phospho-protein or a cytokine data is exemplified.

[[Image:Tables.jpg|center]]

[[Image:Figure.JPG|center]]

The data is contained in two spreadsheets, one for the phosphorylation data (''PhosphoproteinChallenge_DREAM3.csv'') and one for the cytokine release data (''CytokineChallenge_DREAM3.csv''). The data is structured according to the following format: in both files the first column contains the cell type (Normal or Cancer), the second column specifies the stimulus, the third column lists the inhibitor, and the fourth column contains the time of data acquisition in minutes. From column 5 to 21, the file ''PhosphoproteinChallenge_DREAM3.csv'' contains the abundance of the 17 phospho-proteins in arbitrary fluorescence units and in the order given in Table 1c. From column 5 to 24, the file ''PhosphoproteinChallenge_DREAM3.csv'' contains the abundance of the 20 measured extracellular cytokines in arbitrary fluorescence units and in the order given in Table 1d. The values that have to be predicted have been replaced in the data files by the text: “PREDICT”.

'''Useful Information regarding measurements'''

(a) ''Data integrity / linearity''. Significant effort was dedicated to data integrity. The data are reported as arbitrary (fluorescence) units in the range between 0 and ~29000. The upper limit (~29000) corresponds to the saturation limit of the detector. Experiments were performed in such a way that measurements are as much as possible within the linear range of the detector. In general, data can be considered linear but there are a few cases that measurements are closer to the upper detection limit of ~29000 (e.g. some cJUN and IL8 measurements) where linearity might have been lost.

(b) ''Detection limits/Repeatability''. The coefficient of variation for repeated measurements was found to be ~8% (mostly due to biological error). With our current experimental design the instrument detector can report data with accuracy as low as ~300. For example, changes from 55 fluorescence units (FU) to 110 FU cannot be considered “2 fold increase” because values lie within the noise error of the detector. On the contrary, data from 1000 to 2000 are significant.

(c) ''Inhibitor effects''. There are cases in which our inhibitors (i.e. MEKi, p38i, and JNKi) target molecules whose phosphorylation we measure (i.e. MEK12, p38, and JNK). In the case where the inhibitor is present, the phosphorylation state of the corresponding molecule (i.e. phospho-MEK, phospho-p38, and phospho-JNK) should be assumed "absent" and the phosphorylation value should not be used. This known inhibitor effect is more pronounced on the allosteric inhibitors (i.e. the effect of MEK inhibitor on the MEK phosphorylation). The effects of the inhibitors are indirectly corroborated from the phosphorylation state of their downstream targets (i.e. MEK -> ERK, p38 -> HSP27, JNK -> cJUN).

'''Additional data'''

Any additional prior data already present in the literature can be used. This could be especially useful if a model of the network is needed as part of a method to predict the excluded data.

==Submission Information==

The participants to this challenge should submit predictions for either the phospho-protein concentration subchallenge or the cytokine release subchallenge, or both. The submission format of the predictions should be as follows:

For the phospho-proteomics sub-challenge, predictors should make a copy of the ''DREAM3_PhosphoproteinChallenge_Predictions.csv'' file, and rename it

''TeamName__PhosphoproteinChallenge_Predictions.csv'',

where TeamName is the name of the team with which you registered for the challenge. Fill in the boxes replacing the text “PREDICT” with the best prediction for the phospho-protein indicated in the header row, for the Simulus/Inhibitor/Time of data acquisition indicated in each row. If you do not add the predicted values for any stimulus-inhibitor-time-phosphoprotein combination, we will consider that your prediction was random. Save your file in the comma separated values (csv) format.

For the Cytokine release prediction challenge, predictors should make a copy of the ''DREAM3_CytokineChallenge_Predictions.csv'' file, and rename it

''TeamName_CytokineChallenge_Predictions.csv'',

where TeamName is the name of the team with which you registered for the challenge. Fill in the boxes replacing the text “PREDICT” with the best prediction for the cytokine indicated in the header row, for the Simulus/Inhibitor/Time of data acquisition indicated in each line. If you do not add the predicted values for any stimulus-inhibitor-time-phosphoprotein combination, we will consider that your prediction was random. Save your file in the comma separated values (csv) format.

==Scoring Metrics==

For the N predictions to be made in each of the challenges, we will compute the score

[[Image:Formula.jpg|center]]

A p-value will be assigned to each of the submissions both in the phosphoprotein concentration and the cytokine release predictions.

==Data Download==

{| style="width:100%; background:#fff; text-align:center; margin-top:+.9em; border:1px solid #ccc;"
|
<div style="font-size:150%; border:20px; margin:1; color:#000;">[http://wiki.c2b2.columbia.edu/dream/data/DREAM3 DREAM3 Data Download]</div>
|}

The Signalling-Cascade Challenges. Description

Summary:

The concentration of four intracellular proteins or phospho-proteins (X1, X2, X3 and X4) participating in a signaling cascade were measured in about 10<sup>4,</sup>. cells by antibody staining and flow cytometry. The idea of this challenge is to explore what key aspects of the dynamics and topology of interactions of a signaling cascade can be inferred from incomplete flow cytometry data.

==The challenge==

We measured the concentration of four components in a portion of a signal transduction cascade in primary cells of undisclosed nature as shown in the Figure.

[[Image:Signaling_Cascade_Model_and_Challenge_Figure.jpg|center]]
[[Image:FigureLegend.jpg|center]]

These cells were activated with ligands for one of their membrane-bound receptors. Two types of ligands: weak and strong (i.e., with different potency), and in different quantities, have been used. The data sets explained in the next section describe measurements on individual cells by antibody staining and flow cytometry for different proteins. The challenge is as follows: given the measurements of components X1, X2, X3 and X4 determine which of the following functions: kinase, protein, phosphorylated protein, phosphatase, activated phosphatase and phosphorylated complex in the signaling cascade of the figure correspond to the measured molecular species X1, …, X4. In other words: is X1 the activated phosphatase? Is X2 the kinase? And so on.

==The datasets==

We obtained a series of measurements of the signal transduction in primary cells of undisclosed nature. The data sets offered to the participants compile measurements on individual cells by antibody staining and flow cytometry for 4 different proteins or phopshoproteins. Depending on the availability of reagents, each file contains measurements on 10<sup>4</sup> to 10<sup>5</sup> cells for the levels of protein Xi (i=1…4) at 5min of activation for one strong ligand and seven quantities and one weak ligand and five quantites. Each row represents an individual cell, and each entry indicates the fluorescence level in arbitrary units. The measurements were taken in the range in which fluorescence is linear with concentration.

The names and contents of the files are as follows.

The files

''StrongLigand_j_Xi_X4.csv''

(where the extension csv stands for comma separated values) contain the values for the proteins Xi (first column) and X4 (second column), which were labeled in the same cell and measured simultaneously, but independently of the other proteins. It corresponds to the activation by 10<sup>7-j</sup> strong ligands (for 0<j<7) or no ligand (for j =7).

The files

''WeakLigand_j_Xi_X4.csv''

(where the extension csv stands for comma separated values) contain the values for the proteins Xi (first column) and X4 (second column), which were labeled in the same cell and measured simultaneously, but independently of the other proteins. It corresponds to the activation by 10<sup>5-j</sup> strong ligands (for 0<j<5) or no ligand (for j=5).

==Submission of predictions==

Download the file ''DREAM3_SignalingCascadeChallenge.xls''. This file contains a header line and four rows corresponding to each of the four measured proteins X1, …, X4, as follows:

[[Image:Challenge1_Table.jpg|center]]

Complete the table by inserting a 1 in the box for the assignments considered correct and a 0 otherwise. For example, if Xi is deemed to be an activated phosphatase, then add a 1 in the corresponding box (intersection of row Xi and “activated phosphatase” column). Each row has to have exactly one 1 and six 0s. Each column can have either all zeros, or exactly one 1 and three 0s. After completing the table, save it in text (tab delimited) (*.txt) format as:

''TeamName_SignalingCascadeChallenge.txt'',

where ''TeamName'' is the name of the team with which you registered for the challenge. Unfilled boxes in the table will be considered to have a zero.

==Scoring metrics==

We will score the results based on the probability that a random assignment would result in the same number of correct assignments achieved in the actual prediction.

==Data Download==

{| style="width:100%; background:#fff; text-align:center; margin-top:+.9em; border:1px solid #ccc;"
|
<div style="font-size:150%; border:20px; margin:1; color:#000;">[http://wiki.c2b2.columbia.edu/dream/data/DREAM3 DREAM3 Data Download]</div>
|}

The DREAM3 Challenges

Summary:

The DREAM3 challenges consist of 4 datasets that were produced from biological or in-silico networks. The participants to the challenges will be able to make predictions about the result of measuremnts or of networks from which the data originated. We will disclose the results of the measurements and the networks and the researchers that produced the data during the DREAM3 conference.

== A few rules==
*Each participant or team of participants will choose a "Team Name".
*The Team Name cannot have blank spaces or special characters.
*Each team will be able to submit only one prediction per category.
*In order to download the data the participant has to [http://wiki.c2b2.columbia.edu/dream/register/index.php?dream=3 register for the challenge]. This registration is independent of the registration for the DREAM3 conference.

== Data Download ==

{| style="width:100%; background:#fcfcfc; margin-top:+.9em; border:1px solid #ccc;"
|style="width:40%; color:#000"| 
{| style="width:400px; border:solid 0px; background:none;"
|style="width:500px; text-align:center; white-space:nowrap; color:#f00;"|
<div style="font-size:250%; border:none; margin:1; padding:.5em; color:#2f0;">Important Notice</div>
|}
|style="width:100%; padding:.1em; font-size:95%; color:#00f;"|
<div style="padding:.3em;"> </div>
These datasets of Challenges 1-3 cannot be used for external publication without explicit permission from the data owners.
|}

Step 1. [http://wiki.c2b2.columbia.edu/dream/register/index.php?dream=3 Register your team for the challenge].
Once you have registered your Team for the challenge, a password will be sent to the e-mail address you provided in the registration.

Step 2. Download the datasets. You will need to use the password provided in Step 1 above.
* Challenge 1: The Signaling-Cascade Challenges. [[The Signalling-Cascade Challenges. Description | Data Description]]. [http://wiki.c2b2.columbia.edu/dream/data/DREAM3 Data Download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold-Standards].
* Challenge 2: The Signaling-Response Prediction Challenges. [[ The Signaling-Response Prediction Challenge. Description| Data Description]]. [http://wiki.c2b2.columbia.edu/dream/data/DREAM3/ Data download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold-Standards].
* Challenge 3: The Gene-Expression Prediction Challenge. [[The Gene-Expression Prediction Challenge. Description | Data Description]].[http://wiki.c2b2.columbia.edu/dream/data/DREAM3 Data Download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold-Standards].
* Challenge 4: The DREAM3 In-Silico-Network Challenges. [[The DREAM3 In-Silico-Network Challenges. Description | Data Description]]. [http://wiki.c2b2.columbia.edu/dream/data/DREAM3 Data Download]. [http://wiki.c2b2.columbia.edu/dream/data/gold-standards/dream3/ Gold-Standards].

The Protein-Protein Subnetwork Challenge. Description

Summary:

For many pairs of bait and prey genes, yeast protein-protein interactions were tested in an unbiased fashion using a high saturation, high-stringency variant of the yeast two-hybrid (Y2H) method. A high confidence subset of gene pairs that were found to interact in at least three repetitions of the experiment but that hadn’t been reported in the literature was extracted. There were 47 yeast genes involved in these pairs. Including self interactions, there are a total of 47*48/2 possible pairs of genes that can be formed with these 47 genes. As mentioned above some of these gene pairs were seen to consistently interact in at least three repetitions of the Y2H experiments: these gene pairs form the “gold standard positive” set. A second set among these gene pairs were seen never to interact in repeated experiments and were not reported as interacting in the literature; we call this the “gold standard negative” set. Finally in a third set of gene pairs, which we shall call the “undecided” set, genes were seen to interact only once or twice in repeated experiments, or were seen never to interact but were reported as interacting in the literature. The challenge consists of predicting which gene pairs belong to the gold standard positive set, and which gene pairs belong to the gold standard negative set.

----

==Dataset ==

''File with Gene IDs:'' The file '''Prot-Prot_Genes.xls''' contains a list of 47 yeast genes (identified with ORF IDs). The challenge consists of determining the set of true positive and the set of true negative protein-protein interactions among all the pairwise interactions between these 47 genes.

===Submission Information===

Submit a ranked list of gene pairs, ordered according to the confidence you assign to your prediction that a pair interacts, from the most reliable (first row) to the least reliable (last row) prediction. Use a tab-separated 3 column format as in the example below:

:YeastGeneA \tab YeastGeneB \tab XYZ

where YeastGeneA and YeastGeneB are genes in the file '''Prot-Prot_Genes.xls''', and XYZ is an interaction score between 0 and 1 that indicates the confidence level you assign to the prediction that a pair interacts. (E.g., XYZ = 1 if the pair is deemed to interact with highest confidence and XYZ = 0 if the pair is deemed not to interact.) All pairs omitted from the list but that belong to the gold standard positive or gold standard negative set will be considered to appear randomly ordered at the end of the list with XYZ = 0. Submitted pairs that belong to the undecided set will not be scored. Save the file as unformatted text, and name it:

:'''TeamName_ProtProtSubnet.txt'''

where TeamName is the name of the team with which you registered for the challenge.

===Scoring Metrics===

The submitted list will be judged exclusively on the gold standard positive and gold standard negative sets. Submitted pairs that belong to the undecided set will not be scored. We will score the results using the area under the precision versus recall curve for the whole set of predictions. For the first ''k'' predictions (ranked by score, and for predictions with the same score, taken in the order they were submitted in the list of gene pairs), precision is defined as the fraction of correct gold standard positive predictions to ''k'', and recall is the proportion of correct gold standard positive predictions out of all the possible gold standard positive interactions. Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated.

The Genome-Scale Network Challenge. Description

Summary:

A panel of single-channel microarrays was collected for a particular microorganism, including some already published and some in-print data. The data was appropriately normalized (to the logarithmic scale). The challenge consists of reconstructing a genome-scale transcriptional network for this organism. The accuracy of network inference will be judged using chromatin precipitation and otherwise experimentally verified Transcription Factor (TF)-target interactions.

----

===Dataset===

This challenge dataset consists of two files. One file, '''data.csv''' contains the experimental data, and the other, '''tfs.csv''', lists the transcription factors.

''File name:'' '''data.csv'''

''Description:'' This file contains a 3456 genes x 300 experiments dataset. The names of both genes and experiments have been withheld, and operon information is not provided. As described above, the experiments represent both published and not-yet-released data from a variety of sources. The 3456 genes include all known and putative transcription factors and all genes whose interactions will be used for testing, as well as a number of other recognized coding sequences. This file is comma-separated and is easily imported into Excel or any other program.

''File name:'' '''tfs.csv'''

''Description:'' This file contains the indices of rows belonging to transcription factors in the matrix from '''data.csv''', one per line.

===Submission Information===

Submit one network prediction in one or more of the following categories: DIRECTED-UNSIGNED, DIRECTED-SIGNED. Use the 3 tab-separated column format as in the example below:

:row#1 \tab row#2 \tab XYZ

where row#1 is the index of the row of a transcription factor and row#2 is the index of the row of one of its putative targets in the same order as the rows in the file '''data.csv'''. XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a the TF at row#1 regulates the gene at row#2. The value of XYZ will be different for UNSIGNED, SIGNED-EXCITATORY and SIGNED-INHIBITORY, and will be discussed below.

Note that row#1 has to be one of the values of the file '''tfs.cvs'''. Interactions for which row#1 does not correspond to a transcription factor will not be judged for scoring.

Only DIRECTED networks will be accepted. If the transcription factor at row#1 regulates the transcription factor at row#2, and also the transcription factor at row#2 regulates the transcription factor at row#1, then both lines should be included. Participants whose algorithms produce UNDIRECTED networks can submit their predicitons provided they directionalize their UNDIRECTED network into a DIRECTED one. To do that, simply assume that only transcription factors may be regulators. This will make all edges directed, except for TF1 --> TF2 edges. These edges must then be submitted twice as: TF1 --> TF2, and TF2 --> TF1, in effect as if predicting a feedback loop. Naturally, this strategy will produce no false positive if the feedback loop is correctly predicted, one false positive even when one edge is correctly predicted, and two false positives when it's not.

'''''For UNSIGNED submissions:'''''

:XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a TF regulates a target gene, regardless of the sign of the regulation. (E.g., XYZ = 1 if the pair TF-gene is deemed to be connected with highest confidence, and XYZ = 0 if the pair is deemed not to interact.) Order your predictions in decreasing order of XYZ values, i.e., from the most reliable prediciton (highest XYZ value) in the first row and the least reliable prediction (lowest XYZ value) in the last row. Save the file as text, and name it:

::'''TeamName_UNSIGNED_GenomeScale.txt'''

:where TeamName is the name of the team with which you registered for the challenge.

'''''For SIGNED submissions:'''''

:Submit one network predictions for excitatory connections and one for inhibitory connections.

:'''''For EXCITATORY connections:'''''

::XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a TF upregulates a target gene. (E.g., XYZ = 1 if the TF is deemed to upregulate the target gene with the highest confidence, and XYZ = 0 if the pair is deemed to be disconnected, or the TF is deemed to downregulate the target gene.) Order your predictions in decreasing order of XYZ values, i.e., from the most reliable prediction (highest XYZ value) in the first row, and the least reliable prediction (lowest XYZ value) in the last row. Save the file as text, and name it:

:::'''TeamName_SIGNED_EXCITATORY_GenomeScale.txt'''

::where TeamName is the name of the team with which you registered for the challenge

:'''''For INHIBITORY connections:'''''

::XYZ is a connectivity score between 0 and 1 that indicates the confidence level you assign to the prediction that a TF downregulates a target gene. (E.g., XYZ = 1 if the TF is deemed to downregulate the target gene with the highest confidence, and XYZ = 0 if the pair is deemed to be disconnected, or the TF is deemed to upregulate the target gene.) Order your predictions in decreasing order of XYZ values, i.e., from the most reliable prediction (highest XYZ value) in the first row and the least reliable prediction (lowest XYZ value) in the last row. Save the file as text, and name it:

:::'''TeamName_SIGNED_INHIBITORY_GenomeScale.txt'''

::where TeamName is the name of the team with which you registered for the challenge.

===Scoring metrics===
We will score the results using the area under the precision versus recall curve for the whole set of predicitons. No threshold need be applied to your predicitons, since even low precisions at increasing recall will contribute to the final score. All pairs omitted from the list in your prediction files will be considered to appear randomly ordered at the end of the list with XYZ = 0. For the first ''k'' predictions (ranked by connectivity score, and for predictions with the same score, taken in the order they were submitted in the prediction files), precision is defined as the fraction of correct predictions to ''k'', and recall is the proportion of correct predictions out of all the possible true connections (with the approperiate sign, if the category is SIGNED). Other metrics such as precision at 1%, 10%, 50%, and 80% recall, and the area under the ROC curve will also be evaluated.