Resources / Oncotopix Discovery: deep learning as a standard analysis method for digital pathology images
Discovery
Duration 15 min
David Mason and Fabian Schneider, Visiopharm
Oncotopix Discovery: deep learning as a standard analysis method for digital pathology images
Details
Duration 15 min
Transcript

Welcome to this Visiopharm presentation, introducing Oncotopix Discovery.

A quick disclaimer, all products mentioned in this presentation are for research use only.

Visiopharm does have a portfolio of clinically appropriate products. If you’re interested in learning more, please use the email address on the screen now.

Within the field of digital pathology and digital image analysis, there are many common questions that people have when approached with complex problems.

Here are some of them.

There are so many packages available, so many software, and so many approaches that many people are confused about which one is best or which one is appropriate for their particular system or model.

We also hear a lot of people who are domain experts in their biological area unsure how to convert their biological knowledge into analysis.

Lastly, many people who approach an image analysis workflow find that there’s too much time spent in development, training or modifying existing workflows.

This combined with many other questions leads us to the point that there’s a need for powerful tools, but that are easy to use and accessible to everybody.

Hopefully, during the course of this short talk, we will answer some of these questions and demonstrate how Visiopharm’s approach can be used to solve these and many other problems.

I’d like to start by addressing this first point.

Visiopharm has been in this digital pathology and digital image analysis space for over twenty years.

During this time, we’ve seen many different approaches, and we are of the opinion that deep learning has now got to a point where there is the optimal approach for the vast majority of questions within the digital pathology space. Complex problems require advanced tools, and this is where deep learning shines. You can see in this example, this is a tumor detection problem in breast cancer.

You can identify tumor areas by their darker hematoxylin or darker blue staining, such as the area identified here.

And you’ll notice that the contrast between these structures and stromal structures are very low.

It’s a really subtle difference in nuclear shape and also morphology.

This is also a particularly difficult problem because as you can see in this IHC image, some of the cells are positive for DAB, this brown stain, and some are negative.

And in this case, we can, of course, see a larger variance of the number of positive cells and their morphology when we look at different sections.

Deep learning allows you to segment these complex tissues without the need for secondary or complex stains, often using simple stains such as h and e or IHC DAB.

It handles subtle or low contrast structures very well.

And lastly, possibly the most important point is that deep learning allows you to build robust approaches that can handle stain and tissue variance, such as the ones that are commonly seen with different section thicknesses, different staining modalities, or different approaches.

These are key points as to why we feel deep learning is the best approach in the vast majority of biological questions.

But more than that, what this enables is to allow the software to take over and replace laborious manual tasks such as identifying areas of interest or excluding artifacts with end to end automation.

Now you don’t have to take our word for it. A simple search of the literature of papers published in peer reviewed journals shows that the field is definitely moving towards adoption of deep learning pathology, and we believe will continue to do so as these technologies are made more available to more people.

So what sort of problems can deep learning solve in the digital pathology space?

The answer is almost everything that we’ve come across.

From simple things such as nuclear detection in bright field and fluorescent images, tissue segmentation to identify areas of necrosis or metastasis, and even outside of oncology into toxicological pathology looking at things like kidney structure, glomerular detection, cell invasion, and cell classification.

Deep learning has almost infinite scope as to the sorts of projects, stains, and approaches that it can help with.

Having explained what deep learning can do, what I’d like to point out is that with Visiopharm’s release of the new Oncotopix Discovery platform, we have decided to take this technology and give it to everyone who accesses Visiopharm software.

We call this deep learning for everyone as we truly believe that this groundbreaking technology of deep learning can be used to solve a vast range of problems.

This means that anyone engaging with Visiopharm will then have access to deep learning tools in the standard package.

This means using these powerful tools across any modality, imaging mass cytometry, fluorescence, or bright field imaging, for example, and all of the major file formats.

Of course, the standard package comes with fully customizable outputs so you can create the metrics that you desire.

It’s accessible with no coding or scripting required, and it also has extensible functionality.

And what I mean by that is that for instance, if you use tissue microarrays, you can use our tissue arraying tool to easily deray and manage TMA cores for individual analysis.

Likewise, if your work involves high or ultraplex data as we often see in fluorescent imaging or imaging mass cytometry, you can use our specialized multiplex phenotyping and data exploration tools to help your studies using these data types.

If you ever do serial sections or serial staining approaches, regardless of the modality, brightfield, fluorescence, or a combination of both, We have tissue aligned tools that within the software register the serially sectioned or serially stained images.

These three packages are fully integrated with the Visiopharm software. So you can use them in combination with our deep learning tools that come as standard.

We also have a range of extended authoring tools that let users optimize deep learning settings and pick from a wider range of classifiers.

I’d like to now talk about how we can take these deep learning tools and use them to actually solve problems in a method that we call see it, train it, find it.

This really addresses the problem of taking your biological knowledge and converting it into analysis.

We’ve taken these tools that are highly powerful and basically made them easy to use without any computational knowledge whatsoever.

In the first step, the user simply looks at their images and identifies different areas that they’re interested in segmenting. In this case, you can see tumor and stroma highlighted.

The second step is simply to provide annotations, giving examples of tumor areas here in red and stromal areas here in green.

With a push of a button, the user can then train the software to show what these different areas look like across single or multiple images.

Once this is done, we can move on to step three, which is find it.

Taking a whole slide image and then simply running the app that has been produced, then provides you with any number of results in both regions or numbers that you’d like.

See it, train it, find this.

The last thing I’d like to mention is the pretrained knowledge that comes with Oncotopix Discovery.

As we’ve mentioned, it’s very easy for users to develop their own apps with the see it, train it, find it approach.

But in many cases, there’s even a quicker way to get to your answers.

Visiopharm can be used across any range of tissues, stains, and modalities.

And Oncotopix Discovery deep learning is versatile and robust.

So what this means is that not only can solutions be used to take into account the variability seen across different images, it can also handle different modalities such as fluorescence, imaging mass cytometry, and brightfield.

And this pre trained knowledge is available to end users with a push button simplicity.

A great example of this is with our nuclear detection apps available to users of Visiopharm that can just be plugged straight into your own data, and the majority of cases will work off the shelf.

This significantly speeds up development time and allows you to get to your results much faster than training these yourself.

I’d like to now give a quick example demonstration of some of the pre trained knowledge and how it looks in the Visiopharm software.

Here, we can see a whole face breast cancer section stained with IHC DAB for Ki sixty seven, a proliferation marker.

If we zoom into the image, we can see different areas of interest, and we have some tumor areas and some stroma areas as introduced earlier in the talk.

Visiopharm runs well across whole slide images, but can also be used on regions of interest within images.

It’s worth noting that regions of interest can be adapted, erased, or added to identify specific regions of interest.

Here, by selecting a region and then simply hitting run from the ribbon, we can start to use this pretrained knowledge to identify tumor areas within this region.

You’ll notice that in real time here, the tumor areas are identified and outlined in this blue dashed region of interest.

We also have access to customized metrics such as the tumor area in millimeter squared.

One of the really powerful things about this pretrained knowledge is that you can actually daisy chain together multiple apps here, not only to detect tumor, but for instance, to do a cell detection and classification, which can be specified to be within tumor regions, within stromal regions, or both.

This is particularly useful if you’ve done tissue segmentation, for instance, to exclude areas of artifacts.

These then are no longer analyzed as part of your whole sequence, saving you time and providing an end to end analysis solution.

As you can see, the cells are detected, classified, and then recorded.

Each of these cells is counted as either negative in blue or positive in red. The numbers are reported in the results table, and also a custom metric, the proliferation index, is calculated. This is simply the percentage of positive cells.

Here, the overlay’s opacity can be changed and customized to see the underlying data and confirm that these are in fact identifying and classifying cells correctly.

This is a great example of pre trained knowledge in action using off the shelf apps to do tumor detection and cell detection.

I’d like to highlight another example of where pre trained knowledge can be used and also the power of deep learning approaches.

Here, if we look at kidney images, where the aim of the project is to identify glomeruli, what we can see is that in all of these cases, the same app has been run to detect glomeruli regardless of their underlying staining.

On the left is c d sixty eight, in the middle is collagen, and on the right is nephron.

This highlights the power of deep learning where you can identify the same structures in three different staining regimes that are drastically different.

Simply by seeing it, training it, and finding it, users can create their own powerful deep learning apps to solve these sorts of problems.

I’d like to thank you for your attention for this presentation of Oncotopix Discovery, our approach of deep learning for everyone, see it, train it, find it, and the demonstration of pretrained knowledge.

If you have any questions after this presentation, you can use the email address shown on this slide.

About the webinar

Easy-to-use AI-based deep learning is now integrated as the standard analysis methodology in our best-in-class analysis software. Designed for anyone, it allows tissue-based researchers to tackle both simple and complex datasets across a wide variety of applications. In the past, only image analysis experts could analyse complex tissues.

Our new analysis packages include powerful pre-trained nuclei segmentation algorithms suitable for brightfield and fluorescence applications which can be further tuned for even more specificity, and deep-learning-based tissue segmentation to find tissues of interest, exclude artifacts, and enable scoring/counting within specific tissue compartments. Now, anyone with the understanding of tissue morphology can train an AI-based deep learning network to get accurate and reproducible data, making it easy to generate reliable quantitative results needed for breakthrough discoveries and publications.

Learning objectives
    • How the use of deep learning improves the analysis of digital pathology images
    • The use of deep learning to segment tissues, find artifacts, and localize scoring/counting to specific regions of tissue
    • Using pre-trained deep learning networks to segment nuclei across a range of tissue and staining types
Experts

David Mason, Senior Technical Specialist, Visiopharm

Dave Mason is a senior technical specialist in image analysis, supporting Visiopharm’s UK and European sales team. He has a background in cell biology and microbiology and has spent over a decade in academia, specializing in light microscopy and digital image analysis.

Dr Fabian Schneider, PhD, Product Manager Research, Visiopharm

Dr Fabian Schneider is part of Visiopharm’s R&D and Product Management team, responsible for phenotyping products as well as service projects for custom APP development. Fabian has over 10 years of international experience in cancer biology and immuno-oncology, working in academic research labs, clinical research teams and computational pathology groups in both academia and biopharma. Fabian received his Dr phil. nat. in Cell Biology in 2011 from the Johan Wolfgang Goethe University Frankfurt, Germany.

Questions? We’re here to help.
Contact us
Success
Your message has been successfully sent!