HEROHE Banner
Join 
  Last edition:        
                      Aug 03, 2022

Attribution

This dataset is available for research purposes under the Attribution-NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0). 

Bellow, you can find a summary of (and not a substitute for) the license. (Disclaimer)

Under the license terms:

Attribution:  You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.

NonCommercial: You may not use the material for commercial purposes.

NoDerivatives: If you remix, transform, or build upon the material, you may not distribute the modified material.

No additional restrictions: You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.

If you used this dataset or part of it, the corresponding publication must be cited:

  1. Conde-Sousa et al., HEROHE Challenge: Predicting HER2 Status in Breast Cancer from Hematoxylin–Eosin Whole-Slide Imaging, 2022, J. Imaging, ; 8(8):213; DOI: 10.3390/jimaging8080213

Optionally, the following publication can also be cited:

  1. La Barbera, D., Polónia, A., Roitero, K., Conde-Sousa , E., Della Mea, V. Detection of HER2 from Haematoxylin-Eosin Slides Through a Cascade of Deep Learning Classifiers via Multi-Instance Learning (2020) J. Imaging, 6(9), 82; DOI: 10.3390/jimaging6090082

Registration

Both individual and team participations are welcome in the Challenge. This Challenge is part of the 16th European Congress on Digital Pathology (ECDP2020). For each team, at least one person must be registered in the congress to be eligible to the challenge's prizes. 

The contact information provided will not be used for any purposes other than those related to the congress or this Challenge.

Once the registration is completed and accepted, participants will receive instructions on how to download the dataset.

Please note that each person can only belong to one team.

Registration links: 

Participation

Participation in the Challenge is conditioned to the proper submission of:

  1. a short method description, and the code (or compiled version*) until December 31st, 2019
  2. the predictions until January 15th, 2020
  1. *Short method description, the code (or compiled version*), and test set prediction until January 24th, 2020 until 9 a.m., January 28th, 2020 (Greenwich Mean Time - Portuguese time zone)*

To be eligible for the prizes, it is also mandatory that at least one member of each team properly register and attend ECDP2020.

Results will be made publicly available on the Challenge website after the submission deadline.

Challenge participants grant the Challenge organization the permission to use the result of their methods. Nevertheless, participating teams maintain full ownership and rights to their method. The Challenge organization does not claim any ownership or rights to the developed works.

The best performing methods will be invited to collaborate on a journal paper describing and summarizing the different approaches used and the respective results achieved on the Challenge.

To ensure a fair comparison of the submitted methods, the usage of any private dataset during the development of the methods is not allowed.

*Note *that submission of a result in the Grand Challenge HEROHE does not warrant the presentation of a poster/oral presentation at ECDP2020. Each team is encouraged to submit their work as an abstract according to the abstract submission rules of ECDP2020.

* If the team decides to provide only a compiled version, it is their sole responsibility to ensure that it is ready to run on all major operating systems.

Data

The presented dataset contains 360 cases, 144 positives and 216 negatives. Each whole-slide image was saved in the  MIRAX format.

For more information please the Dataset page.

Submission

Submission is to be performed in two stages. Until December 31st, 2019, each participant or team has to submit a short method description, and the code (see Participation Section). Until January 15, 2020 the predictions on the test dataset has to be submitted.

A zipped file must be submitted until January 24 (Portuguese time) at https://ecdp2020.grand-challenge.org/evaluation/submissions/create/. The submitted zipped file should contain three files/folders inside: one word file with the methods description, one csv file with the predictions, and a folder with the code and the corresponding README file.

This submission will be used to rank the methods for prize distribution.

Note that each team can only participate in the challenge once, i.e., each team can only submit an algorithm. If several attempts of submission are performed, only the last will be considered.

Details regarding the submission files:

The code (or compiled version*) must be released ready to use with minimal user interaction and should have a README file with a detailed explanation on how to run it. It should receive as input a folder with the test dataset (please refer possible file conversions needed previously) and should return a .csv file with the predictions.

The predictions file must be in the csv file format and should

  • be named after the team,
  • have one header row (pre-filled by us**),**
  • have one row per sample, and
  • have three columns

*** ***

The caseID should match the slide’s filename (without file extension)

The soft prediction should be a value, between 0 and 1 with the probability of that sample being positive.

The hard prediction should be an integer 0 (for negative slides) or 1 (for positive slides).

The short method description should include references to any source used during the work.

** A .csv file with the header and the caseID column already filled will be released with the test dataset

Prizes

The teams with the best results in the Challenge will be awarded with 1000 euros, 500 euros and 250 euros, for the first, second and third places, respectively. The representant of each of three best teams will also receive a free registration for the next year ECDP2021. The Challenge prizes will be paid by IPATIMUP - Instituto de Patologia e Imunologia Molecular da Universidade do Porto, the entity that provides the organizational support of ECDP2020, through a sponsorship from ROCHE.

The Challenge prizes will be awarded during the ECDP2020 conference, to be held in Porto, 13th-15th May 2020. Specifically,  ECDP2020  has allocated prizes to be distributed among the three teams with the best performance***.

Eligibility to the Challenge's prizes is conditioned to the quality of the method and participation at the conference***.

Awarded:
Rank Team Team representative
1 Macaroon Ming Feng
2 MITEL Vincenzo Della Mea
3 Piaz Ehsan Montahaei

*** ECDP2020 was cancelled due to the coronavirus pandemic, so these points were not taken into account

Evaluation

Submissions are scored based on the F1-score,  the harmonic mean of precision and recall:

Evaluation panel: