ML/AI - Model Creation
Guidance needs to document best practices for defining trained model datasets. Perspective to be more prescriptive on model operative point metrics and performance based on the intended use. Model use cases should define a clear intended use, and design a blueprint for “package insert” (sample inclusion criteria, exclusion criteria, specific intended use, sensitivity, specificity, AUC/ROC, confidence interval/output prediction, etc). Models should potentially provide minimum confidence threshold or probability of output Robustness across variable datasets- try to define parameters that describe trained dataset and be able to reject samples that do not occur within the intended use case or variability of the training dataset. The goodness of fit of data relative to assumptions of model (and deployment site data) should be defined to better understand the parameters of trained model data set. Uncertainty metrics- models may be able to robustly reject an input (e.g. slide). Define acceptance criteria of a model, is this automated, or created by logic (LIS data). Reference ranges for training data, and capturing performance of new inputs to either accept or reject (i.e. appropriate/sufficient tissue, staining quality, etc). Understanding model generalizability and performance monitoring (over time) at a deployment site.
Key Elements, Next Steps, Timeline
Model performance metrics
Clinical intended use
Concerns & Problems
Deployment site performance
Ethical concerns
Value Proposition
Clear guidance on how AI algorithms are trained and evaluated
Implications & Efforts
Trained
Evaluated, validated, implemented
Their intended use (as established)
Their relevance for Clinical Guidelines (are presented)
Current Projects
News & Updates
Relevant Publications
Pan-cancer image-based detection of clinically actionable genetic alterations
Date: August 2020
Authors: Kather et al.
Link: Pubmed
Deep learning detects genetic alterations in cancer histology generated by adversarial networks
Date: February 2021
Authors: Krause et al.
Link: Pubmed
Deep Learning-Based Segmentation and Quantification in Experimental Kidney Histopathology
Date: January 2021
Authors: Bouteldja et al.
Link: Pubmed
Introduction to Artificial Intelligence and Machine Learning for Pathology
Date: January 2021
Authors: Harrison et al.
Link: Pubmed
The Ethics of Artificial Intelligence in Pathology and Laboratory Medicine: Principles and Practice
Date: January 2021
Authors: Jackson et al.
Link: NCBI
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images
Date: August 2019
Authors: Campanella et al.
Link: Pubmed
Validation of a digital pathology system including remote review during the COVID-19 pandemic
Date: November 2020
Authors: Hanna et al.
Link: Pubmed
Hidden Variables in Deep Learning Digital Pathology and Their Potential to Cause Batch Effects: Prediction Model Study
Date: February 2021
Authors: Schmitt et al.
Link: Pubmed
Synthesis of diagnostic quality cancer pathology images by generative adversarial networks
Date: July 2020
Authors: Levine et al.
Link: Pubmed
Accuracy and Efficiency of Deep-Learning–Based Automation of Dual Stain Cytology in Cervical Cancer Screening
Date: June 2020
Authors: Wentzensen et al.
Link: PDF
Group Leaders
Matthew Hanna, MD
Matthew G Hanna, MD is the Director of Digital Pathology Informatics at Memorial Sloan Kettering Cancer Center. He is an Assistant Attending of Breast pathology and Informatics at Memorial Sloan Kettering Cancer Center, where he completed his Oncologic Pathology fellowship training. Prior appointments include being a Clinical Instructor of Pathology Informatics at the University of Pittsburgh, where he also completed his Pathology Informatics fellowship. He completed his residency training at The Mount Sinai Hospital in New York. Dr Hanna serves as a member on the CAP Informatics Committee and served as a junior editor for the Journal of Pathology Informatics. He has strong interests in clinical and pathology informatics, computational pathology, and breast pathology.
Jochen Lennerz, MD, PhD
Dr. Lennerz is board certified by the American Board of Pathology and the American Board of Medical Genetics. He joined the Massachusetts General Hospital Department of Pathology and Center for Integrated Diagnostics as a staff pathologist in 2014, and is an assistant professor at Harvard Medical School. Dr. Lennerz trained as a pathologist assistant in Berlin, Germany in 1994 and studied both medicine and molecular medicine at the University of Erlangen, Germany where he also received his MD and PhD. He completed his residency training in anatomic pathology in 2008, and a fellowship in molecular genetic pathology in 2009 at Washington University in St. Louis, MO. After completing a two-year gastrointestinal and liver pathology fellowship at Massachusetts General Hospital in 2011, he led a research group on biomarkers in lymphoma at Ulm University, Germany. His interests are tissue-based biomarkers, and financial sustainability of molecular genetic diagnostics.
Steven Hart, PhD
Dr. Hart is an Associate Professor in the Department of Quantitative Health Sciences at Mayo Clinic. His research focuses on genetic predisposition to cancer and in correlating genomic signatures with histologic features from H&E slides to decrease unnecessary testing procedures and identify patients who would benefit from genetic predisposition testing.