National Institutes of Health/National Institute of General Medical Sciences (NIH/NIGMS)
P41GM103832
米国
National Institutes of Health/National Institute of General Medical Sciences (NIH/NIGMS)
R01GM079429
米国
National Institutes of Health/National Eye Institute (NIH/NEI)
PN2EY016525
米国
Ovarian Cancer Research Fund
5-258813
米国
Robert A. Welch Foundation
Q1242
米国
引用
ジャーナル: Microsc Microanal / 年: 2016 タイトル: Quantifying Variability of Manual Annotation in Cryo-Electron Tomograms. 著者: Corey W Hecksel / Michele C Darrow / Wei Dai / Jesús G Galaz-Montoya / Jessica A Chin / Patrick G Mitchell / Shurui Chen / Jemba Jakana / Michael F Schmid / Wah Chiu / 要旨: Although acknowledged to be variable and subjective, manual annotation of cryo-electron tomography data is commonly used to answer structural questions and to create a "ground truth" for evaluation ...Although acknowledged to be variable and subjective, manual annotation of cryo-electron tomography data is commonly used to answer structural questions and to create a "ground truth" for evaluation of automated segmentation algorithms. Validation of such annotation is lacking, but is critical for understanding the reproducibility of manual annotations. Here, we used voxel-based similarity scores for a variety of specimens, ranging in complexity and segmented by several annotators, to quantify the variation among their annotations. In addition, we have identified procedures for merging annotations to reduce variability, thereby increasing the reliability of manual annotation. Based on our analyses, we find that it is necessary to combine multiple manual annotations to increase the confidence level for answering structural questions. We also make recommendations to guide algorithm development for automated annotation of features of interest.