Welcome, and thank you for joining this educational webinar on using deep learning to stratify mitotic events in complex cellular data.

Our speaker today is Dr Joe Daniele, an institute research scientist from MD Anderson Cancer Center and the translational research to advanced therapeutics and innovation in oncology, aka the Traction Group.

Before I turn the presentation over to Dr Daniele, I would like to provide a disclaimer that Dr Daniele and MD Anderson Cancer Center have not received any compensation and have no conflicts of interest with Visiopharm or Global Engage.

All of the information in science presented today is his own and will not serve as an endorsement for Visiopharm or our products.

So with the formalities out of the way, I’ll let you take it from here. Thanks for your time today, Dr Daniela.

Thank you very much for that introduction, and thank you to the organizers for the opportunity to present this lecture today. Title being using deep learning to stratify mitotic events in complex cellular data.

So jumping right in, there are three main things that I’d like to cover for organization. The first being rationale and motivation for why you would wanna quantify mitotic subtypes, and then two examples for how you would apply that method to specific things. One being characterizing drug response and tumor models, for instance, and the other being organoid characterization.

So not just the proliferative capability of certain organoids, but also using mitotic subtypes to find, morphological subtypes within those.

So starting with rationale and motivation, why would you want to quantify mitotic subtypes?

Well, there are three main things that I could think of. One being the characterization of drug response, another being the immune component, and the third being the opportunity to target inaccessible proteins.

Regarding characterization of drug response, normally, what’s, done is just the percent of tumor cells that are actively in mitosis. That’s usually by a marker like phosphohistin h three, and it’s in one number like a percentage.

But if you have images, there’s also in information in there as as well, which is what those cells are and where they are in that particular tumor or tissue, for instance.

And when you look at something like mitotic subtypes, you can start to get an idea of what and where those cells are in each phase.

That’s another type of information that’s also very interesting and useful, not only from drug development model or drug response model, but also disease modeling.

Now drilling down deeper, where those cells are and why they’re getting stuck is also important. So that more has to do with the biology inherent in what’s going on after this particular response.

Is DNA damage response getting turned on? Is mitotic collapse occurring? Is that is that the reason why the cells are getting stuck, or is apoptosis being induced?

That’s something where doing, essentially mitotic subtypes or, phosphoacin h three colocalized with some of these some of these markers like DDR apoptosis gives you a better handle on the biology of that particular situation.

Almost as a complement to the characterization of drug responses, the immune component.

And what I would first, mention is that phospho histone h three in particular, could be used to bolster key sixty seven, which is an early cell cycle marker to tell you which cells are actively in mitosis. That’s one thing you can use it for. But more importantly, you can start to identify which immune cells are actively dividing, and are they dividing more than tumor cells.

To that same extent, once you’ve identified which immune cells are dividing, where they’re actively dividing is also interesting.

And if there’s a drug treatment or this particular disease model, knowing if they’re stuck in, particular phases of mitosis could also be very interesting and and help you get an idea of how the immune component is being affected in whatever treatment is is occurring to that system.

Lastly, and this is a little bit more specialized, knowing which cells are essentially in certain phases of mitosis gives you the opportunity to target normally inaccessible proteins. And that has to do with the fact that nuclear envelope degrades between metaphase and telophase, and these proteins are now accessible. And so if your model or system, was characterized in that particular manner using mitotic subtypes, you could help it would help you decide which models or systems are most, amenable to that particular treatment and going after those inaccessible proteins.

So almost as a motivation slide, just to give you an idea of how mitotic subtypes, even in immunohistochemistry data, can help glean tighter data and more meaningful data from, essentially from microscope images. I wanted to show an image of a vehicle treated tumor where you have phosphohistin h three as your marker for proliferation, and you can see there’s quite a bit of staining here, about twenty seven percent positive Phospho Histin h three nuclei.

And then with certain treatment then, inhibits that proliferation, we see it go down to about nineteen point four. But looking at these particular images, this looks like a much more striking difference than, seven percent.

And the most obvious reasoning for that is potentially because you’re actually not getting a change in, more important cell types, which is, which the right here. We are getting a change in these, but it’s being masked by essentially less of a change in this cell type. So we have the speckled cell type here, and we have this stronger dab stained cell type here. Now if you were to segment all of those nuclei and then, ping the intensity of every single cell that was in your tissue and then plot the distribution of that, you would actually get a plot quite like this that shows a bimodality in the signal intensity.

And under the vehicles treated, you can see the highest proportion of cells are the strongly stained phosphohistin h three positive, cell types. And then you have a second peak down here at a lower dab intensity for this speckled cell type here, which you can determine that just by, essentially figuring out what the intensity of this is on that image and then correlating that back to the graph.

Now you can see that under the vehicle treated, there’s a higher amount of, higher relative frequency of cells in this stained actively mitosing cells and a lower amount here. But if you treat with something that slows down mitosis, what you get is this essentially is peak goes down and then this peak goes up, which is almost masking that difference that you’re normally seeing. And so if you were to graph that, if you combine all essentially cells, the speckled and the intensely stained ones, you do see a difference. You see some cells going down to about nineteen or so from twenty seven, but some of them don’t. Right? And there’s some, essentially obfuscation of that difference that would normally be there. However, if you eliminate the speckled cell types from your analysis, you can see there’s tighter clustering in the data and more significance even with, right here, just three mice in, this particular example.

So subdividing the nuclei, even in immunohistochemistry, in this particular case by signal intensity, is one way to get, a better handle on what exactly is going on.

So moving on to essentially the next topic, the characterizing drug response, that is almost a lead in into fluorescent characterization. So we looked at the different nuclei subtypes in phosphohisten h three and immunohistochemistry.

What happens when we look at it in fluorescence? So it turns out there are actually six different phases or six different morphologies morphologies that you can determine, from phosphohistin h three fluorescence microscopy data. You have interphase, prophase, prometaphase, metaphase, anaphase, and telophase. And in some particular models, you have a higher incidence of mitotic collapse. Now using incidence of mitotic collapse.

Now using deep learning, this is the, one method to enhance this type of marker classification and really find what percentage each of these morphologies is within a tissue.

Now just quickly, AI is, in this particular case, thought of as a computer system’s ability to perform tasks that normally require human intelligence. In this particular case, it’s visual perception or decision making.

So using a neural net, a deep neural net that’s, essentially many layers deep, you can begin to better characterize each of these areas of pixels so that essentially the confidence that some, region is interface, is essentially your interface category versus a metaphase care category changes.

So what I mean by that is if you were to look at a twenty x stitched image of a tumor that’s been stained for something like DAPI and phosphorescent h three, this is just a higher magnification of that. What you can do is find certain regions and then train a deep learning algorithm to identify which areas that are phosphoacin h three positive corresponds to the different cells, the different mitotic subtypes.

And after you’ve trained it appropriately, you can test that using heat maps. So the first one I’m gonna show you is essentially a heat map telling the AI confidence, the AI algorithms confidence in, finding background cells.

So what this is saying is in all the regions where it doesn’t see phosphohisten h three signal, it has very high confidence that that’s background. Those are non phosphohisten h three cells.

So that’s the the red indicates a high probability that the algorithm is confident.

In these areas where you do actually you did see phosphatidin h three signal, it’s not confident at all. It’s saying essentially that we should not consider this in any way background signal.

To that same extent, if you’ve appropriately trained your dataset, then you can do that for all all of those particular cell types. So in this particular case, you have interphase, and here is the highest probability of all the cell types that have that it believes are in interphase.

And that’s quite different from any of the other cell types here. You can see there are other cell types present, but the interphase ones are only being highlighted here. So essentially, you can test out your algorithm in this particular way, and visually confirm that training is going well.

You can then do that with all those different cell types. And if everything is looking good, then one marker effectively becomes seven with an appropriate training set. And, you can even, essentially, like, if you were to put a mask over it, it would look like this. And that allows you to identify particular cell types, not only their, prevalence, but also their location within a tissue. So if you’re really interested in metaphase or anaphase, essentially, you could identify all all of, the one particular color that pertains to that and then see if there’s an obvious pattern in, a tumor.

In this particular case, you also get numbers. And one way to graph that in this particular in in this example is in a, combination treatment. We’re looking at different drug combinations.

So this is a stacked histogram that gives you the proportion of each of these stages in my mitosis, essentially for each treatment. So interestingly, we don’t see much of a deviation in the variability of these things for vehicle treated treatment one or treatment two. But if you combine treatment one one and treatment two, you see an increased prevalence of interphase and prophase and then a decreased prevalence of telophase.

And if you plot this as a radar plot, then it becomes a little bit more apparent that potentially what’s happening is the cells are having difficulty entering into mitosis.

So the they’re essentially getting stuck at the interphase level and less and less of them are completing telophase.

Okay.

So moving on, that was just a tumor characterization and looking at drug response and diff different drug combinations.

Another system that I think this method could be quite useful for would be an organoid characterization.

And one nice thing about organoids is that they’re much more amenable to many testing many different combinations.

Essentially, each organoid is is an n, so you can have many n per well. And the timelines for these types of experiments are much, much shorter than what you might find in in mouse, experiments with PDXs, for instance.

So if you were to apply some of these methods to IF, stained sections of organoids, you can really get a very good handle on how drug combinations or different disease models are behaving before you would remove that to more in vivo settings.

Okay.

So in this particular case, what we did was we cultured organoids and Matrigel to a certain phase, a certain size, and then, basically dissolve the matrigel and, spun it down and essentially got it into histo gel and embedded into pathology blocks and stained it for different things. In this particular case, our main mitosis marker, phosphohistin h three, and DAPI.

Now what you can do with an image like that is you can segment it. You can segment each of those particular cells and then identify which cells are positive for phosphocinase three and which ones are negative.

And if you do this appropriately, you can also get the information of what the size of each organoid is.

So in this particular case, for organoid model one, I was able to plot the percent positivity of phosphohistin h three over the area of this organoid, and I saw a linear relationship between the two. You see the there’s more variability in the smaller ones. And then as they get bigger, they go down from one percent positivity.

Just a little bit, but they do.

Now using that same type of information and, an appropriate training set, you could perform your, deep learning algorithm for phosphorescent h three subtypes on the organoids.

And the first thing I did here was take the percent positivity for percent metaphase in each of these organoids or and plot it against the area of that organoid.

Interestingly, comparing any of these, the percent metaphase, tell you know, anaphase, telophase, etcetera, to the size, I didn’t see a linear relationship in any of these.

That’s not to say it doesn’t exist. In this particular model, I just didn’t see it. But what you could do is get what I almost call a fingerprint for the subtype frequency.

So if you were to look at the frequency of each particular phospho histone h three subtype or mitotic subtype in there for each organoid by organoid, you get a fingerprint for the prevalence of each of those within each model.

So for the most part, most organoids have really high metaphase and anaphase components with lower prometaphase and interphase.

And, that’s something that you could definitely compare once organoid is treated with this or that or you say, you know, essentially inactivate a certain protein is is this subtype frequency being affected?

One other thing that I’d like to bring up is another organoid model where I did not see a linear relationship between the positivity of phosphohistin h three and the organoid size.

Now looking at the actual images that are in there, I saw that after a certain size in the organoids, it looked like the positivity was staying around one percent. It looked like the prevalence of particular marker seemed to be about the same. Whereas in the smaller organoids, you saw a much higher prevalence of phosphohistinase three. And so what I did almost like in flow cytometry is I I took all of these and I binned them in one set of data, and I took all of these and I binned them in another set of data. And I saw that you actually do get that linear relationship back if you were to plot those different subtypes. In this this particular case, you have a high phospho histone h three subtype and a low phospho histone h three subtype.

To that same effect, you can then do your, fingerprinting for that. And interestingly, the high phospho histinase three subtypes looked more like, in terms of the fingerprint, the organoid one model by itself. Whereas the low phospho histinase three subtype had a different, essentially shape to this, with the biggest differences being in prometaphase and metaphase.

So this is one way you could track a response. You could look at the prevalence of this subtype to this subtype. In this particular case, we were able to do that using phospho histone h three positivity.

But this is essentially giving you an idea maybe of the biology in the system as well. And it could be very interesting to see how a drug affected this subtype more or this subtype more. And if there are certain mechanisms, for instance, of resistance inherent in that.

So two last slides just to wrap things up.

I wanted to cover a few practical tips for training a deep learning algorithm. The first being, avoid any unnecessary channels or inputs during the training. The one that I showed in the demonstration was only using DAPI and, phosphocinase three signal.

Nothing else. No autofluorescence or any other inputs. And that’s gonna ensure that the, the algorithm, the deep learning algorithm is not identifying anything unnecessary or anything that would mislead the, system. Okay. Second, monitor the first ten thousand iterations of that learning.

And then just like I showed, test the confidence for each of those features and re annotate and add annotations as necessary.

So I would do ten thousand and then test the confidence using using the heat maps as I demonstrated. And if things are tracking, then let another ten thousand go. Check again with the heat maps. And if they’re missing something or something is moving one mark in the wrong direction, then you re annotate and add annotations as necessary. Let it go another ten thousand. And if it’s starting to finally move in the right direction, then you wanna essentially let it run overnight and get to a point where it has very high probability, very high confidence that these features are being identified.

Once you’re confident in in all of that and you think the training is is enough, then you’re gonna get to essentially quantifying that particular data. So you wanna set all your probabilities to above to eighty percent or above, which will help avoid double labeling cells. And then in any post processing steps, you wanna remove small objects and fill holes that may have become vacant from that eighty probability.

So in terms of future directions, I think that there’s definitely interesting work to be done in the quantification and colocalization of phospho histone h three, as I mentioned before, understanding the biology of response, and looking at colocalization with apoptotic subtypes in some of these forms, and also DNA damage response with phospho gamma h two a x.

Now inherent in that, just looking at some images that we happen to have for, in this particular case, cleave caspase three, I could easily identify several subtypes of cleave caspase three and six subtypes of phospho gamma h two x with this particular one colocalizing with phosphohistin h three. So there’s probably a lot of information inherent in these particular markers and probably several other markers as well if you were interested in, for instance, the nature of DNA damage response or the nature of apoptosis. So of DNA damage response or the nature of apoptosis.

And, there’s definitely some interesting morphological features, for instance, between this, form of DDR or game h two x signal and this form of apoptosis.

Next, once you’ve identified any of these particular things, you can then drill down further in morphometric analysis. And the most prominent example of that would be in the nuclear cytosol ratio to see if there are, essentially tumor qualities to those particular cell types. But other things are definitely interesting as well in analysis, and that could tell you things about the nature of those cells. Lastly, going back to spatial information, what is dividing and where, the ability to identify these different subtypes and then correlate that to different distance measurements is definitely interesting as well.

And this is looking at differences in cells between tumor and stroma or stroma and vasculature, and not to forget the immune component in here as well. So understanding what’s dividing and where and how this relates to the different substructures and superstructures involved in tumors is is obviously very interesting in in a direction that, you know, I think is worth pursuing in biology.

So with that, I’d like to thank once again the organizers for the opportunity to present this work, and we’ll be doing any, question and answer via email. Thank you very much.

Thank you for an excellent presentation. And as Dr Daniela said, we’ll be happy to address any questions you might have at webinars at Visiopharm.com. I would like to thank Dr Daniele again for his time today, and thank you for your attention.