
To adapt the existing software to microscopy, the research team first evaluated it on a large set of open-source data, which showed the model’s potential for microscopy segmentation. To improve quality, the team retrained it on a large microscopy dataset. This dramatically improved the model’s performance for the segmentation of cells, nuclei and tiny structures in cells known as organelles.
The team then created their software, μSAM, which enables researchers and medical doctors to analyze images without the need to first manually paint structures or train a specific AI model. The software is already in wide use internationally, for example to analyze nerve cells in the ear as part of a project on hearing restoration, to segment artificial tumor cells for cancer research, or to analyze electron microscopy images of volcanic rocks.
“Analyzing cells or other structures is one of the most challenging tasks for researchers working in microscopy and is an important task for both basic research in biology and medical diagnostics,” says the author.
Identifying and delineating cell structures in microscopy images is crucial for understanding the complex processes of life. This task is called “segmentation” and it enables a range of applications, such as analysing the reaction of cells to drug treatments, or comparing cell structures in different genotypes. It was already possible to carry out automatic segmentation of those biological structures but the dedicated methods only worked in specific conditions and adapting them to new conditions was costly.
An international research team has now developed a method by retraining the existing AI-based software Segment Anything on over 17,000 microscopy images with over 2 million structures annotated by hand.
Their new model is called Segment Anything for Microscopy and it can precisely segment images of tissues, cells and similar structures in a wide range of settings. To make it available to researchers and medical doctors, they have also created μSAM, a user-friendly software to “segment anything” in microscopy images. Their work was published in Nature Methods.