Last edition:        
                    Jan 22nd, 2020        13h41 (GMT)

Please respect the rules of the Challenge. The organization reserves the right to not consider submissions that fail to abide to the rules presented in this page:


Registration

Both individual and team participations are welcome in the Challenge. This Challenge is part of the 16th European Congress on Digital Pathology (ECDP2020). For each team, at least one person must be registered in the congress to be eligible to the challenge's prizes. 

The contact information provided will not be used for any purposes other than those related to the congress or this Challenge.

Once the registration is completed and accepted, participants will receive instructions on how to download the dataset.

Please note that each person can only belong to one team.

Registration links: 


Participation

Participation in the Challenge is conditioned to the proper submission of:

  1. a short method description, and the code (or compiled version*) until December 31st, 2019
  2. the predictions until January 15th, 2020
  1. Short method description, the code (or compiled version*), and test set prediction until January 24th, 2020 until 9 a.m., January 28th, 2020 (Greenwich Mean Time - Portuguese time zone)


To be eligible for the prizes, it is also mandatory that at least one member of each team properly register and attend ECDP2020.

Results will be made publicly available on the Challenge website after the submission deadline.

Challenge participants grant the Challenge organization the permission to use the result of their methods. Nevertheless, participating teams maintain full ownership and rights to their method. The Challenge organization does not claim any ownership or rights to the developed works.

The best performing methods will be invited to collaborate on a journal paper describing and summarizing the different approaches used and the respective results achieved on the Challenge.

To ensure a fair comparison of the submitted methods, the usage of any private dataset during the development of the methods is not allowed.

Note that submission of a result in the Grand Challenge HEROHE does not warrant the presentation of a poster/oral presentation at ECDP2020. Each team is encouraged to submit their work as an abstract according to the abstract submission rules of ECDP2020.



* If the team decides to provide only a compiled version, it is their sole responsibility to ensure that it is ready to run on all major operating systems.


Data

The presented dataset contains 360 cases, 144 positives and 216 negatives. Each whole-slide image was saved in the  MIRAX format.

For more information please the Dataset page.




Attribution

The downloaded dataset, or any data derived from this dataset, cannot be given or redistributed under any circumstances to persons not belonging to the registered team.

This dataset was designed for being used exclusively in the context of the HEROHE Grand-Challenge and / or the submission of communications to the ECDP2020. The dataset, or any data derived from this dataset, cannot be used for any other purposes, including but not limited to military, industrial development/patenting, or academic use (including journal publications, conference papers, technical reports, presentations at conferences and meetings or teaching).

The organization anticipates that the results of this challenge will be published in an international peer-reviewed scientific journal. If and when this happens, the dataset will become available for research purposes under the Attribution-NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0). The corresponding publication should thus be cited. Updates on this issue will become available in the challenge web page.


Submission

Submission is to be performed in two stages. Until December 31st, 2019, each participant or team has to submit a short method description, and the code (see Participation Section). Until January 15, 2020 the predictions on the test dataset has to be submitted.

A zipped file must be submitted until January 24 (Portuguese time) at https://ecdp2020.grand-challenge.org/evaluation/submissions/create/. The submitted zipped file should contain three files/folders inside: one word file with the methods description, one csv file with the predictions, and a folder with the code and the corresponding README file.

This submission will be used to rank the methods for prize distribution.

Note that each team can only participate in the challenge once, i.e., each team can only submit an algorithm. If several attempts of submission are performed, only the last will be considered.

Details regarding the submission files:

The code (or compiled version*) must be released ready to use with minimal user interaction and should have a README file with a detailed explanation on how to run it. It should receive as input a folder with the test dataset (please refer possible file conversions needed previously) and should return a .csv file with the predictions.

The predictions file must be in the csv file format and should

  • be named after the team,
  • have one header row (pre-filled by us**),
  • have one row per sample, and
  • have three columns

 

The caseID should match the slide’s filename (without file extension)

The soft prediction should be a value, between 0 and 1 with the probability of that sample being positive.

The hard prediction should be an integer 0 (for negative slides) or 1 (for positive slides).

The short method description should include references to any source used during the work.


** A .csv file with the header and the caseID column already filled will be released with the test dataset


Prizes

The teams with the best results in the Challenge will be awarded with 1000 euros, 500 euros and 250 euros, for the first, second and third places, respectively. The representant of each of three best teams will also receive a free registration for the next year ECDP2021. This Challenge is sponsored by ROCHE.

The Challenge prizes will be awarded during the ECDP2020 conference, to be held in Porto, 13th-15th May 2020. Specifically,  ECDP2020  has allocated prizes to be distributed among the three teams with the best performance.

Eligibility to the Challenge's prizes is conditioned to the quality of the method and participation at the conference.



Evaluation

Submissions are scored based on the F1-score,  the harmonic mean of precision and recall: