Thank you very much for the introduction and thank you very much for the invitation to this great event. It’s truly a great pleasure to be part of such an interesting conference.

I’m Tobias Dovmark. I’m a research scientist here at Novo Nordisk.

And today I’m going to talk to you about how you can use Fisher Farm software to automate quantitative nuclear fluorescence analysis in histological kidney and pancreas sections using convolutional neural networks that are inbuilt in the Visiopharm software.

So being a diabetes company for about one hundred years, we are very interested in type one diabetes and related diseases such as diabetic kidney disease.

And therefore the primary target organs for analysis that I will show you today will be rat pancreas and, and human kidney, sections.

So some of the usual problems scientists encounter when analyzing fluorescence is that you really, when you analyze different tissues where the nuclei appear different, the DAPI stain can appear different, you really need the ability to adapt your nuclear segmentation.

Also running a lot of experiments using different fluorescence markers.

You need to be able to adapt your analysis to different fluorescence wavelengths or different fluorescent targets.

And also you need to set a correct threshold, eliminating false positive background classifications.

So I’m going to talk to you about a systematic and adaptable approach to analyzing nuclear fluorescence that we’ve developed here at Novo Nordisk.

And I will show you how we analyze human kidneys in relation to human diabetic kidney disease. And I will also show you how we use these analysis for adapting to the analysis of red pancreas, both in relation to the nuclear segmentation, but also in relation to adapting it to different kinds of fluorescent stains.

So every part of our analysis begins with, begin with tissue detection. So we have this as a standard feature in our batch processes.

So here is an example of how we approach the tissue detection by outlining the tissue by adding the AF48 channel with the Dabi channel and using a standard deviation filter to get an easy and quick outline of the tissue we want to analyze.

Of course, you can also use other types of segmentations or manual segmentation if, if that’s preferable.

But really, a great challenge in every cellular analysis is the challenges of obtaining good nuclear segmentation.

So we’ve used the VisioForm software using the UNet’s Deep Learning module, as you can see here in the bottom left corner.

And the UNet is primarily designed to, for cell segmentation. Therefore, we chose this model and used in this case a magnification of 20x.

And as you can see here in the red labels, we’ve obtained a very good segmentation of each individual nuclei, especially also segmenting segmenting nuclei that are lying very close to one another, which you can see in this animation here with the red labels noting denoting every single nuclei.

So we’ve evaluated this nuclear segmentation that we’ve developed using the Visiopharm AI module.

And as you can see here in the top left corner, we have manually drawn ground truth examples of what we consider correctly labeled nuclei.

And the blue labels are the the the nuclear labels as predicted by the app.

And these, these labels we use to to calculate different metric coefficients to describe how well our nuclear segmentation is performing.

And as you can see in the bottom left corner, we have dice score sensitivity and precision about around zero point nine.

And at Jacquard index, which is also called the intersection over union, which measures the true positive area divided by the, the true positive, false positive, and false negative areas of the union of the two.

And we get a Jacquard index around point eight five.

So we have developed quite good nuclear segmentation.

And the next thing in, in analyzing nuclear fluorescence is to avoid inclusion of background fluorescence.

So to address this in a systematic manner, we have used this nuclear fluorescent segmentation algorithm that we’ve developed in the, in the Visiopharm software, and we have included this output variable, which measures the mean intensity of the four eighty eight channel in every nuclei per object. So we get a measurement of the green fluorescence in each nuclei using this algorithm.

And this makes us able to really get a nice overview of what fluorescence values we have in our sections and what we can consider as true positive staining and, what we can consider as background.

Here, is an example of how we have used these red nuclear labels and and run this algorithm across an entire tissue section.

And and if you plot this using a histogram here with, fluorescence values, these are sixteen bit images. So, these values go up to sixty five thousand five hundred.

And as you can see here, this is coupled with a rock plot where each line indicates a measurement. And as you can see, most of the values in this case, background values that are below ten thousand fluorescent values.

And we can use this.

This was an example of, of an entire tissue section. So using around half a million cells.

Here we have made a similar plot drawn from a smaller representative region.

And as you can see, the same distribution applies here. And we can use this to set a threshold of what we consider as the background and what we consider as true positive.

So it really gives us a very easy and nice way to adapt, fluorescence thresholds between experiments or within experiments.

So here you can see we can also use this to really inspect our data and inspect, our fluorescence values.

Here, zooming in on on the higher fluorescence values appearing in, in this. So here in red, we have the background and and these other measurements, what we have set as true positives.

So, you also often run into situations where you have different levels of background.

So here, as you can see in glomeruli, you have very little green background, whereas in the tubular cells you have a higher background of of green staining.

We can use this same principle here to, to segment all the cells and measure the green fluorescence in in all the cells indicated here in red. And, we can really see that we have these two populations of background.

So now that we have a tool to easily set the the fluorescence threshold, We are then ready to analyze, our tissue samples.

And as you can see here, this is analysis of KI67 fluorescence in the green channel, which measures proliferation, which is a routine, assay that we, we use. And as you can see here, this algorithm, correctly correctly labels the green fluorescence, fluorescent cells here indicated with orange label compared to the negative labels shown here in blue.

This is another example of a Ki-sixty seven fluorescence where you see this, this tubular background.

And as you can see here in the analysis in this animation, these orange labels are very good at at finding the true positive cells while leaving the cells with the background, as negative cells.

We have also used this approach to adapt this to, to a different stain. So this is an example of a tunnel stain measuring cell death in, in this case, in human kidney samples.

And as you can see here, as shown in the pink labels, are the cells that are positive for tunnel staining and the blue labels are cells that are negative for tunnel staining.

So here you can see that we have a very good cell segmentation and we have a very good classification of of positive cells.

This is another example also highlighting here the, how well the algorithm is at segmenting epithelial cells that are lying very close to one another and also, very good at at segment classifying true positive cells. There is also green staining, background staining here that are not associated with, ADAPI stain and the staining is therefore not included.

So taken together, I’ve shown you how we have used the VisioFarm software to develop an easy to use workflow that can use be used for adaptation to a wide range of, of, molecular targets within nuclei of cells, where we’ve used both tissue detection, a very good nuclear segmentation, global nuclear fluorescence evaluation to set a threshold, for an easy setup of a target analysis.

So sometimes a target analysis in an entire tissue section is not preferable. Sometimes it’s preferable to have to assess changes in particular compartments of tissue, for example, in in the glomerulus, in relation to diabetic kidney disease. So glomeruli are situated here in the renal cortex.

And as shown you here, they’re nicely visible in Pest stain that stains polysaccharides and tissue.

And we’ve used the Visioform AI algorithm, in this case, a deep lab, deep learning network on five x to segment out glomeruli in past stained tissue sections.

As you can see here, this gives us the ability to make specific regions around cells as shown here in in blue, make regions around the cells that are in the glomerular compartment.

And we can then transfer these regions to fluorescence images and compare, in this case, target expression in cells of aligned images. So these are fluorescence images that are aligned with PAS images, where regions around the glomeruli have been transferred from this AI algorithm to the to the fluorescence, fluorescent image.

And we can then use this to compare different disease states. So in this case, glomeruli, without diabetic kidney disease, with mild diabetic kidney disease, moderate diabetic kidney disease, and so on.

And, we can use this, to to evaluate target expression. So, in this case, we can measure how a target decreases with, cellular loss and diabetic kidney disease progression.

So now I’ve shown you how we use this systematic and adaptable approach to, to, to analyze targets in diabetic kidney disease.

And and now I’ll show you how we use this algorithm, to to really adapt it to nuclear, both to to adapt it to the analysis of the rat pancreas, both with respect to the nuclear segmentation and adaptation to the fluorescence analysis.

So here we have some sections of red pancreas that are stained with, emulate emulates in the red channel to stain the, the exocrine pancreas and Ki67 in the yellow channel, which stains proliferating cells. So in this case, we are interested in measuring proliferation only in the exocrine pancreas or the cells associated with red staining.

And we with relatively little retraining of the nuclear segmentation algorithm, we were able to obtain a very good nuclear segmentation for this analysis, where we were able to classify the cells according to to the red stain and the yellow stain. So as shown here, the white labels are cells not associated with with red fluorescence, where blue cells associated with red staining and green labels proliferating cells in the exocrine pancreas. So cells associated with both yellow nuclear fluorescence and red cytoplasmic fluorescence.

So how do we approach this? So we use the we used post processing in combination with this nuclear segmentation app that we’ve made, where we separated each individual cell using this yellow label.

Then we dilated each individual nuclei with an artificial cytoplasm.

We then use this cytoplasm to classify the cytoplasm according to either red fluorescence or lack of red fluorescence.

So, gray and white cytoplasm, respectively.

And then we classified the nuclei according to whether they were associated with the spread staining or not.

We, as I said before, we used the AI architect, which really gives you a lot of abilities to retrain your neural networks. So in this case, we used a free step of two layers.

We’ve also had very good results with freezing three layers and, and and doing retraining on a limited number of of, manual annotations using this AI Architect module.

We’ve also used rotation and flipping of the images and adjustments of brightness and contrast and so on, and really had have had great results using using this part of the Visiopharm software.

So here is just an example of the segmentation where you can see the segmentation of the cells, negative cells with the purple surrounding label and the positive cells with the red surrounding label, showing you how great this algorithm is at separating both the cells and classifying the cells correctly.

So, taken together, I’ve shown you this workflow that we’ve made for easy adaptation for a new target analysis.

I’ve also shown you how we use convolutional neural network to compartmentalize target analysis to really dig deeper into specific cellular expression levels, and also how we use the Visiopharm software to adapt our analysis to different organs so that we have the flexibility both to adapt the nuclear segmentation and the fluorescent segmentation.

At the end here, I would like to, to thank the organ donors and the families for donating the valuable kidney samples. And I will also like to thank Heather Ward, Claire Berman, Savantian Tapovan, and, Jonas Sain van Rohe, who have all contributed to this presentation.

So, thank you very much and, I’m happy to take questions.