Date of Award
6-26-2026
Date Published
August 2026
Degree Type
Thesis
Degree Name
Master of Science (MS)
Department
Electrical Engineering and Computer Science
Advisor(s)
Michael Blatchley
Keywords
brightfield microscopy;fine-tuning;foundation models;image analysis;organoid segmentation;SAM 3
Subject Categories
Computer Sciences | Physical Sciences and Mathematics
Abstract
This thesis presents the first systematic application of SAM 3, a unified foundation model for promptable segmentation, to mouse small intestinal organoid brightfield microscopy image analysis. The work spans the complete pipeline from zero-shot baseline evaluation through domain-specific fine-tuning on a GPU cluster, and documents the full engineering process required to adapt a state-of-the-art foundation model to a novel biomedical imaging domain. A comprehensive literature review of over 20 papers spanning detection-based, classical segmentation, foundation model, and morphological analysis approaches identified a clear research gap that this thesis addresses. Five critical compatibility patches were developed to deploy SAM 3 on Mac Apple Silicon (M-series), replacing CUDA-specific operations with CPU-compatible alternatives. A systematic evaluation of 20 text prompts revealed that shape-based descriptors significantly outperform domain-specific biological terminology. The top four prompts — ‘cell’, ‘circular’, ‘round cell’, and ‘bubble’ — were combined into a multi-prompt pipeline with circularity, area, and IoU-based deduplication filters, achieving an 18% improvement in unique organoid detections per image over the initial zero-shot baseline (103 detections using the 'circular vesicle' prompt), while maintaining high precision through IoU-based deduplication across prompts. Baseline detection results were validated across multiple treatment conditions, with detection counts varying meaningfully between control and treated samples, reflecting genuine biological differences in organoid density and morphology. Three systematic failure modes were identified in the base model. To address them, a collaborative annotation effort was undertaken with PhD students from Professor Blatchley’s laboratory, and a data augmentation pipeline was implemented to expand the training corpus. The fine-tuning infrastructure was deployed on Syracuse University’s OrangeGrid HPC GPU cluster, requiring resolution of ten sequential engineering problems spanning HTCondor configuration, CUDA version compatibility, Hydra configuration management, model checkpoint integrity, decoder device placement, and a critical COCO annotation format incompatibility. This last issue caused SAM 3’s Hungarian assignment matcher to produce zero valid ground truth matches, a failure mode that was identified, root-caused, and corrected in this work. After resolving all issues, fine-tuning was completed over five epochs, with bbox regression loss decreasing 24% and detection counts improving across all prompts by +7% to +21% relative to the zero-shot baseline. Quantitative evaluation against Roboflow ground truth annotations at IoU@0.30 across 8 annotated images demonstrates that SAM 3 Baseline achieves Precision=0.906, Recall=0.544, F1=0.680 on spherical organoids and Precision=0.935, Recall=0.524, F1=0.671 on non-spherical organoids, outperforming OrganoID (F1=0.406 spherical, F1=0.304 non-spherical), OrgaExtractor (F1=0.042 spherical, F1=0.086 non-spherical), and TellU (F1=0.602 spherical, F1=0.615 non-spherical) without any domain-specific training. On spherical organoids, SAM 3 Baseline achieves F1=0.680 versus OrganoID's F1=0.406 — a 67% relative improvement. SAM 3 Fine-tuned achieves the highest precision on non-spherical crypt-containing organoids (P=0.935), the most challenging morphological class in the dataset. This work establishes a documented, reproducible foundation for automated organoid morphological analysis using SAM 3, with demonstrated accuracy improvements over existing tools and clear pathways for extending to video-based timelapse tracking.
Access
Open Access
Recommended Citation
Manani, Hiren, "From Detection to Segmentation: A Foundation Model Approach to Organoid Brightfield Image Analysis Using SAM 3" (2026). Theses - ALL. 1058.
https://surface.syr.edu/thesis/1058
