Chapters Transcript Video Artificial Intelligence in Cardiovascular Medicine Dr. Mo Elshazly explores what artificial intelligence is and why it is needed in clinical cardiovascular care. Hi, everyone. Pleasure to be here and thank you again for the wonderful uh invitation. Um So today I'm gonna be talking about artificial intelligence and cardiovascular care. Um My job today is to take you on a journey exploring A I some of its current applications and cardiovascular care specifically and then finally discuss some challenges on future directions. I'm going to review some technical aspects, but I'm also gonna make sure that I don't turn it into a technical deep engineering talk. So to begin, actually, this is a fun graphic. So this is a graphic that was completely created by a computer. This was not drawn by anyone. This is a platform called Dali Two, which comes from Salvador, Dali and Wally. Uh you know, the movie. Um and basically you prompted to create a futurist futuristic graphic of an anatomical heart um surrounded by a neural network and this is what it pops up. OK? And this is the future that we're jumping into is a generative A I, that's what it's called, it's gonna be huge. So I don't have relevant conflicts of interest to disclose um relevance to the stock. So these are quickly my objectives. What is A I, why do we need it? A I and cardiovascular care and then some challenges and how to move forward. So what is A I? There are so many definitions and this can actually get somewhat philosophical, but this is my favorite and simplest definition out of Merriam Webster. It's the study of computer systems designed to perform tasks that normally require human intelligence. And I'm gonna take a little bit of a deeper dive in, into, into this later. Now, you might think that A I is a new term, but it was actually coined back in 56 by John mccarthy. Uh And Alan Turing was even the one who kind of started talking about machine intelligence and then A I in general has gone through periods of uh let me see my mouse. No. All right. That's fine. Maybe I'll her here. OK. Yep. Should be. All right. Uh So A I in general has gone through periods of uh you know, uh downs and ups and in uh around this time in the early two thousands, that's when machine learning started kind of uh um picking up and then in 2012, that's when two guys, uh uh Jeffrey Hinton and uh Krissy came up with something called the re function. And with that came the explosive age of deep learning. So why do we need artificial intelligence in health care? And to answer this question, we need to quickly review the evolution of medicine over the last century. So since the beginning of mankind, practice of medicine has been largely reflexive, all right, it focuses on disease, diagnosis and treatment. And basically, we're tasked with observing brief snapshots of human health only when things go wrong. And that's when we get involved was largely parental where the doctor knows best. And while the concepts of things like prevention was introduced by uh people like William Osler, it was never really mainstream, but medicine has evolved to be much broader these days. Now, it focuses on prolonging life and improving its quality through prevention, disease diagnosis and treatment. We're even trying to reverse aging these days and it follows a very precise and holistic approach or that's what we're trying to do. Uh and the patient in this journey is your partner. So as you can see, we started looking at human health through a much bigger lens and using modern data collection tools. We now have unprecedented access to huge amounts of data. In fact, our ability to record health data points has grown much faster than our ability to make any clinical sense out of it. So that's where A I kind of starts coming in. It's trying to help us to make sense of all this data. Now, these are quick examples. I know they've uh Doctor Patel had given um uh a talk about this in general. But smart vas are an example of devices generating all this data that we really don't know what to do with. Um And uh and some of this data is clinical grade data that we can make sense of like ECG data and some of us are using our clinical practices. Genetics is another example of big data available to us. Uh It's widely available now. It's cheap through uh companies like 23 and me, for example, and Helix. And there's also all these national programs such such as all of us research program that's trying to collect uh millions and millions of uh genetic data. So it's gonna be up to A I and uh in addition to us to make sense of all of this. So I'm, I've given you a quick glimpse on the huge amount of data that's available to us. So how can we apply it in patient care and research? We need it in general for diagnostics. Uh And again, you can probably uh figure what I'm talking about here. Um So, you know, we're using clinical multi mix data for early detection, et cetera, et cetera imaging, uh uh data and deep learning for uh you know, expert level diagnosis. We need also for population health management. So more on the public health aspect therapeutics. Um So again, for pharmacogenomics for guiding drug therapy, et cetera and even for administration and regulation. OK. So looking at data of um you know, that's needed for big, big like management in big hospital centers and so on. So its applications are huge. Now I'm gonna start taking a little bit more of a technical dive. So how does it work? So as I mentioned earlier, A I is a big umbrella and refers to the study of computer systems normally designed to perform the tasks that require human intelligence. However, the recent popularity of A I has been propelled by the development of what we call machine learning. OK. And machine learning is the study of computer algorithms designed to learn without explicit programming. And that's here, that is the key. The idea here is that the machine learns itself or makes inferences without us explicitly asking it what to do or forcing it to do what we want it to do. Now, deep learning is another subset of machine learning which has really revolutionized the field since the early 20 tens. And it focuses on a subset of machine learning that utilizes what we call artificial neural networks. OK. And I'm gonna, this is gonna be uh the focus of my talk, deep learning. So there are two general categories uh uh in machine learning that you've probably heard of or um or might know about already. There is in general, we divide them into supervised and and supervised learning, supervised. Basically, you learn uh um the computer learns a function that maps a specific input to an output and it uses labeled or annotated data. That's the key here. We're labeling or annotating the data for the computer. So we're kind of supervising it or help, we're helping guide it. And this is actually the form that is most commonly used in cardiovascular image. OK? Because we gotta teach the data what this is. For example, what this echo view is, et cetera, et cetera. OK. So it can be used for classification and regression tasks and this is um um a gif just kind of showing how it's um kind of spits an output. If it spits out cat, we kind of help it relearn that it's a dog and it just this kind of process of uh learning back and forth. Now unsupervised learning, which is what people call the black box A I or what gets a little bit scary here in an unsupervised learning the computer or the algorithm learns a function that allows it to produce a certain output, but without using any specific labels. OK. And this can be used for things like clustering. So what is clustering basically you this, I think will make sense. You throw a large amount of data at the computer and you don't label the data whatsoever. And then the data, the computer basically creates an algorithm that is able to take this data and kind of organize it into what the computer thinks are different categories. So this is called classification. And we're using this to gain insights into certain things. So for example, um there are uh we'll go through some studies, but there was a study that used it to differentiate people with different lipid phenotypes because, you know, we have all these phenotypes and they're all like kind of inter intersect together and we don't know what's what and they're using these to kind of reinvent this whole thing. And then there are things called dimensionality reduction, which is reducing the amount of data that is, you know, you have, let's say a million different variables. How do you reduce that to 1000 that are meaningful because they all kind of communicate together and then generative A I, which is basically A I where, you know, if you've seen the videos of, you know, Tom Cruise saying stuff or Obama or whatever, this is all generative A I. And then some there are things called semi supervised that kind of interact kind of span both spectrum. OK. But again, this is a little bit on the technical aspect. So I'm gonna leave it out. Um Now this is a simplified schematic of what an artificial neural network is. And this is work that's done by one of my colleagues uh Ramsey. We um uh he's now at M US C. Uh He's a heart failure specialist but specializes in A I stuff. Um So the network uh really consists of what we call neurons. So each of these is a neuron or a note. OK. And there are a range in layers and there are multiple layers and every algorithm can get more complex. So there are. So for example, Z patch is uh if you guys uh you know know that ZZ uses the deep learning algorithm, it has 34 layers of uh neural networks. And basically each feature nodes value is calculated as the sum of the product of each corresponding input feature and weight. So basically what it does is it takes all these different clinical features and assigns different weights to them. OK. Um And it does that based on certain inferences and then basically each node then becomes the product of feature one and its weight kind of like that. OK. And then at the end, it ends up coming up with kind of one node or one neuron that then goes through something called an activation function and then eventually spits out a number for you here. This is a Sigma activation function. So it's basically a lock transformation. So you get anywhere from 0 to 1 uh Because you're trying to predict a heart failure in this instance. All right. Again, this can get very complicated, very quickly. But this is in general how uh uh these algorithms work now knowing the algorithms and how deep learning works. It's important to think about what types of medical data can be analyzed by ML. And this is where it gets starts getting more clinically relevant and interesting to uh hopefully all of you. So ta structured data is the data that we've worked with for years, right. It's what we use in regular statistics. So, you know, it's your Excel sheet with all these numbers, vital demographics, blah, blah imaging is a new thing that deep learning has allowed us to play with. Um And I'll, I'll show you how we do that through something called computer vision, then free text or what we call unstructured clinical data. So clinical notes imaging, basically your E hr how does it go through E hr and kind of make sense of it? And it does that through natural language processing and then uh wave forms which dive into the EP folks um like the Apple watches and all these remote monitors, they use something called time series analysis. So amplitude of your waves uh uh on one axis and time on your X axis. And this is just a quick schematic because you always, some people say that machine learning is statistics. Yes, to a certain extent it is they have, they intersect, especially whenever you have, you're trying to make inferences from a small number of variables, it ends up becoming really a glorified logistic regression um especially the supervised stuff where you're labeling it. But then once you start going to deep learning and unsupervised, there is no intersection with statistics whatsoever and it's a really, really novel field. Now also remember machine learning is not really a silver bullet. OK. And here this is uh a study from Ramsey where he looked at different studies that use machine learning. And they had compared, you know, performance of these were mostly supervised machine learning algorithms compared to a regular logistic regression. And you see they're better, but they're not significantly better all the time. OK. So, so the fact that someone wrote a paper and says that they've used, you know, machine learning to, you know, produce this results doesn't necessarily mean that they've uh you know, done something that is, you know, uh magical. OK. So make sure you um understand what's going on, what variables were used, what algorithms were used and everything is really very different. Right now, I'm gonna start going into different aspects uh of deep learning. And we're gonna go through uh imaging and then we're gonna go through natural language processing and then I'll do a little bit about remote monitor. OK. So compute uh uh uh computer vision is uh uh the field of artificial intelligence that basically trains computers to interpret and understand the visual world. OK. And um it does this uh through using basically converting digital images into numbers. And if you're interested in the technical aspect, this is basically how it does it. So these are black and white and great images. So these are eight bit encoded. So the computer comes up with two to the power of 8, 256 different numbers and assigns every scale of color a different number exactly like that. OK. And basically the computer starts seeing the image as numbers and that's how it starts kind of doing its uh algorithmic stuff and, and uh and mathematical uh functions, etcetera. OK. And then colored stuff gets a little bit more complex. You basically use different colors, are presenting uh different numbers are presenting three different colors which are blue, red and yellow. And uh and you go from there and then you can also increase the numbers with things like CT scan which have high resolution. Um So here are a few applications of uh uh computer vision uh in cardiovascular imaging. So uh one of them is image planning. OK. So uh what uh some algorithms are now able to do is able to look at MRI S and CTS and echoes and figure out kind of, you know, are you, this is four chamber view, three, chamber view, two, chamber view, et cetera, image processing and reconstruction, augmented interpretation diagnosis. So again, we're, we're learn, we're teaching it what Amyloid is and what hypertensive heart disease is. And it's able to predict a certain accuracy, risk prediction of hospitalization or of ischemia blah, blah. Um The phenotyping which is again, more of uh OK. So we'll take people with sarcoid, for example, can we apply an algorithm to them and actually find within sarcoid different phenotypes? There's the phenotype, for example, that is more with hard block and maybe there's a phenotype that's more astra with V and can an image kind of be able to differentiate both. And then also it helps in report generation, which kind of A I is being used right now by Ge and Phillips in, in uh in uh some report generation. But again, it's more of really basic form of A I. But this can be significantly enhanced and then automation of measurements and then even image acquisition and I'll show you an example here of two studies. This was a study done by um um uh it all published in nature in 2020. And most deep learning studies have actually used echo. They've actually looked at single frames. This was the first one to actually use a 3D kind of video. And they did that by training the algorithm, it's a convolution neural network. Um So they trained it on a publicly available kinetic human action video data set. So it's a data set that's available by Google that actually has human motion. And you take certain basics that the algorithms have learned from that and you apply it to other things. So it kind of pre-trains itself on this new uh um uh way of motion. And uh actually, it showed great accuracy for uh ejection fraction calculation even in patients with arrhythmia. So it was able to do it beat by beat. So it can give you an ejection fraction during an arrhythmia. So during a FIB or PV CS and an injection fraction during sinus rhythm. And this is kind of gonna be the future. It hasn't been uh clinically used yet with some of all these companies, but it's coming. And then this is an example of a clinical application of uh cardiovascular imaging in the eye that's currently being used commercially, this company called Caption Health. I don't know if you guys have heard of it, but uh Caption Health is an FD approved software that basically guides nurses who are not really trained in echo. So we're not taking talking about echo. So we're talking about uh just bedside nurses and it guides them to doing an echo. And what it does is it point, it kind of shows them how to move the ultrasound. I'm just gonna start from the beginning. It looks at the image figures out where they are and then kind of guides them on how to go from parasternal long to parasternal short for example, and, and so on. And then it captures the images that are of good quality and saves them. And they actually published the results here was published in uh published uh three years ago in Jama cardiology. And this is comparing a nurse examination to a sonographer examination. And basically these numbers reflect uh what is thought to be an adequate study. Uh And you can see here that the differences in percentage points are not big at all, like it gets maybe big with the IVC size but trained nurses who have never been trained in echo are doing a very good job of being guided by this. Uh And of course, the implications of that to expanding, you know, uh echo imaging to rural areas or underserved areas uh in the US or other parts of the world is, is, is huge. Now, these are like some applications I want to talk about in imaging. Now I'm gonna move and start talking a little bit about N LP and some of its applications. OK. Uh You guys have all heard about Chat GP T and open A I. So that's always like just want to talk about this. Um So chat GP T four says that N LP is a field of A I that focuses on the interaction between computers and humans through language. And again, its ultimate objective is to be able to read, decipher, understand and make sense of the human language in a valuable way, in a valuable way is very important and it's not supposed to take over the human race. All right. Uh So here's uh an N LP is also very complicated. I'm gonna try and uh uh uh simplify it as much as I can. OK. So uh this is an example of how a computer under understands words. Basically it converts words and in some instances, sentences into what we call tokens. OK? And then these tokens are basically converted into vectors which are numbers and they get placed in this big kind of, you know, uh a 3d space where um um uh words that have similar meaning or have similar context are kind of grouped together. OK? And each is assigned a certain vector, OK. So it's a 3d space, right? So they get an XYZ coordinate. Um And this is done by certain uh uh um algorithms or programs. Now, then what happens is uh after the words are converted into certain vectors or numbers that have some context, then the NOP algorithm, which one is Google's birth and the other one is opening eyes, chat GP T they use something called an attention mechanism and attention mechanism. Basically. What it does just to simplify it is that it gives weights to all these words in a sentence and based on that tries to predict the next word. And in, in Google's bird system, it does that through uh forward and backward reading. So it treats the sentence from the beginning uh and keeps going and then it treats it again uh uh backwards. And the idea is to try to increase the ability to predict uh the, the word. Now open A I is is so bird works better for actually um if you're looking at E hr an unstructured E hr uh it works better at making inferences and finding stuff because it's more accurate per word. But opening I um is much better at text generation. And the reason is that it doesn't do the backward thing, it just does it forward and it takes the sentence from the beginning to the end. It makes it produces weights for each word based on every single word that's preceding it in the sentence. And therefore its ability to produce language is fascinating. Again, this can get very, very, very complicated, but uh I just wanted to make it as simple as possible. Now, how do we use NOP in medicine? Uh So in general, I mean, there's so many applications um you know, you can think of, but I'm just gonna try and simplify the biggest ones that I think are gonna, we're gonna see a lot of in the next few years. Uh So for physicians summarizing clinical data, we get dumped with all these records and records and records and records and we can use chat GP T to summarize them. I'll show you some examples, automation of nodes, billing codes and orders. Imagine that you're writing a VT ablation note and all your billing codes get automatically bill maximally. Wonderful. Right? And actually there are some companies that do that. Um Imagine you're seeing a patient as an outpatient. Again, you're, you get automatic billing, you get the billing code, you get it linked to the diagnosis just from the algorithm's ability to read through your your notes. Medical education also show you some example of work uh we're doing right now and then research of course, scanning data data analytics. You can even use it as a coding statistics assistant. I'll show you some examples for patients augmenting and assisting interactions with medical professionals. There was a recent study published in Jama actually four days ago where it's not a great study because it was all done by physicians. But basically what they did was they took all uh some of patient incoming um inpatient bath. Uh the sorry, the uh inbasket messages that come through epic and they would process it through chat GP T versus uh physician response. And the chat GP T responses were much more personable and nicer because they were long and et cetera, et cetera. But of course, these are all rated by the physicians themselves. So it's not a great study that will put it there. But the potential of this in the future is, you know, you're having your own uh assistant that responds to your, uh a lot of your in basket message messages. It can be used as chat bots or diagnostic bots. And again, this is uh gonna be very interesting with the chat GP T because it's actual performance is excellent. It's performing on medical school exams, uh better than uh many physicians. Uh And then finally, personalized education instructions, interventions. I'll just to take you through quick examples. So I can show you the whole note. But this is one of those uh notes that was, I don't know, 20 pages long with all these different codes and all these pictures and images and, and, and all that stuff uh that's going on. And so one of those that takes you a while to kind of go through. OK. And basically all I did was I just copy, pasted the whole thing then read the single word into a chat GP T four and, and just prompted to summarize the cardiac history. OK? And this is what I got. Um and it's, it's, it's perfect. It's, it tells you everything about the afi flutter chat score. Um It even shows the images that are mostly relevant to uh to uh cardiovascular care. Um I mean, look at this even uh and then uh and then it even come, came up with, you know, summarize the assessment and plan from, from the snow and then I asked it to extract this in table format. OK? This is just very simple stuff, right? This is not nothing fancy. I'm just asking Chad GP T to do very simple stuff and it took this whole note and spit it out into um a table format. I asked it to put a gender diagnosis. It was able to recognize all these things and put them in these tables. So imagine you're gonna start doing research on some electronic health records and you know, you want to create an Excel sheet instead of, you know, finding uh 10 volunteer medical students. Now, Chad GP T can help you with a lot of this and then you can change this also into Excel format, et cetera. Uh Now I was talking about billing and ability to uh kind of uh automatically bill. And some companies like Amazon's Aws S, Amazon's cloud service are they're really developing their A I now to kind of be able to do things like that. Um So some hospitals, again, not the big institutions yet because I think they're still sorting through the technology a little bit, but they're able to take a lot of your notes and basically create automatic billing codes and snow, uh codes, blah, blah, blah, blah, uh and even medications, all that. So imagine something like this in the future. Again, the amount of time it's gonna save you. Now if you like to do research, uh this is an example uh of how you can use it as your um uh kind of coding assistant. Um So I use R for statistics. It's a coding language, uh kind of like Python, but basically I'm asking it here to generate the code for me rather than trying to have to find it or kind of develop it myself. And then like it gives you an error, let's say you put the code and give you an error. Then I tell chat, hey, I got this error and then chat tells me, oh, I'm so sorry. Uh Let me fix this for you and then it spits out the right code So if you're, if you're into statistics or you like to do these things, this can, this is huge and a lot of people are already using it. So again, as you can see, it's more of an assistant, right? It's not gonna take over human intelligence here. It's, it's more of an assistant that makes your life much easier. And I'll, I'll, I'll show you some examples. Now, this is uh uh um I'm showing you a proprietary video. So please do not distribute this. But this is actually a new thing we're working on with Chad GP T. Um And GP T has the ability to be prompted. So this is not available to the public yet. Um Prompting Chad GP T means you're asking it to kind of assume a certain role. So in this video that I'm gonna show you, we asked Chad GP T to assume the role of a 65 year old man with an an stemi who was a jokester. OK? And hard to extract serious answers from. OK. That's what we told Chad GP T. So Chad GP T created all of this through text, right? We then applied some other software to create this avatar. Uh But then we asked that Avatar, what brings you to the hospital today? OK. And what's fascinating here is that what you're gonna see? This is not a script that was written. OK. So everything's here. It's not your typical like, OK, we wrote a script and the avatar is saying it, this is all what Chad GP T generated when just we asked them to that, this is a 65 year old man with an in who's a jokester. And we're asking the patient, what brings you to the hospital today? Checked out? All right. So this is, and this is like conversation you can keep having, you can keep asking them certain questions and it, that, that patient in this instance, keeps answering questions in a certain kind of format. And they're always the jokes. So they're always making, you know, jokes around and, and it's hard to extract information from them. So imagine the implications of this in medical education. So now we're moving from a world where we're not giving students a certain set of standardized patients to go through. Now where the doctor Patel or doctor Keel can sit down and come up with their own scenario and they don't even have to write the script and the, the students can just talk to the, to uh this person and it's a conversation back and forth. And then after the end of the conversation, you can actually ask it to give you feedback and it actually does that and it's all automated. You don't like write any scripts and so on. So, uh this stuff can definitely change uh the way we do education in medicine and then imagine the implications of customer service and things like that. In the future. All right. So now I'm gonna transition to a different application of A I in, in cardiovascular uh medicine, which is remote monitoring. I'm not gonna go into the depths of this because I think uh Doctor Patel had covered some of it in his uh talk. But uh I'm gonna mention a couple of examples. So this is uh Zao uh deep learning algorithm. And uh this was a study that was published in 2019, basically showing that their deep learning algorithm was as good if not better than a committee consensus of cardiology. This is a confusion matrix and this is different 12 different arrhythmias. And in general, this line just shows you that there is so much agreement between the um cardiologist and the algorithm and this is 34 layer uh uh neural network and it's so good that it can even identify a few, a few bits of a FIB or SVT or whatever. And this has been the evolution of Zao algorithms over the years. So until 2015, it was expert rule, expert rule basically are algorithms that are made by experts. So it's like decision trees. OK. So we tell the algorithm if you see this, this means that et cetera and this was its performance compared to human expert level interpretation. And then we started doing some supervised machine learning which again got a little bit better. But then they use all this data that they've acquired over the years to do the first version of their deep learning algorithm and the performance became equal to human uh expert level interpretation. And then after 1 billion hours of data, they build their V two algorithm which now is even performing better than human level uh interpretation. And I know it took a while. But again, it's all about data, right? And this is, this is what Zao did here. That was great. They kept the data, they saved it in a certain way and they started doing stuff with it. Now, a screening again, this is uh more familiar to everyone. So the the algorithms in the Apple watch or the or the Fitbit. Um Again, these are all time series uh deep learning algorithms and depending on the algorithm that the company used their process predictive values differ fitbits was better. But also this is fitbit compared to Apple's 2018. Since then, Apple has really developed its algorithms and this is all PPG based algorithm. So this is when you get the notification uh that you have an irregular beat and that's based on photo uh uh uh Alem which is basically your heart rate. Um And now they have ECG. So you can add the ECG layer to confirm whether you have a fib or not. So the specific insensitivity now are very high. Now, this is a study that tried to look at 12 lead ecgs to predict LV dysfunction um So asymptomatic LV, dysfunction is estimated to be present almost 3 to 6% of the general population. These guys took a 45,000 data set and trained a deep learning uh algorithm to identify LVF less than 35% using a 12 lead ECG. Again, this is an algorithm that's looking at things in the 12 lead ECG that we don't even know what they are like if you like this is black box deep learning stuff. So you, the algorithm won't be able to tell you if it's the pr interval or the, you know, the, the P wave amplitude or the cure rests, duration, whatever it is. And then they applied it in a 53,000 validation data set and got relatively good sensitivity and specificity again, not great but good. And then they validated in eight international cohorts which is very important to do in machine learning and they got good results. What was super interesting in this study was that the ones who were false positive? So the ones where the algorithm said they do have uh an, an injection fresh less than 35%. But they did not, they had a four times odds of developing LV dysfunction in the future. So this is a little bit scary because uh he, he, this has not been extensively validated yet, but it, but it tells you that the algorithm is seeing something that is subclinical and we might be able to use this in the future for um uh screening. And then they took this algorithm did all these modifications to it and tried it on a single lead. Uh Apple watch to predict again, V dysfunction. They didn't get as good sensitive and specific specificities. But um it's, it's again an example of how you can build something through 12 lead and then try and apply through a single lead and maybe algorithms in the future will be able to spit out better results. Now, uh I've gone through examples. So I'm gonna quickly go through some of the challenges and how to move forward. OK. I'm gonna talk about these big themes here so quickly, data and security challenges. So I like to think about this in two big in two different big categories. So first of all, how do we facilitate data sharing for research and clinical purposes when it's desired by patients? So I'm a patient and I wanna, you know, I want you to use my data for research. OK. So we do that by building personal health clouds. So for example, Apple now asks you if you can share if you are willing to share your data with Apple all the time. So that's great, right? So it builds trust and also you're getting all your data on a personal health health cloud. We want to transition to opt in rather than opt out models for uh uh participation research. Um So when people come to the hospital. They want to opt in for research. Again, this helps build trust. We need to update hi P A privacy uh policies and make it, make it easy to manage private settings. Again, when Apple started making it easy to manage your privacy settings, people started feeling more trustworthy of Apple and that's why we need to start doing a lot more of now. How do you protect sensitive data from undesired breaches? Ok. So hacking basically, and we all know that regular cybersecurity stuff isn't enough now. So people are looking into certain things like Blockchain to be able to advance uh uh our ability to protect our data, et cetera, et cetera. And this is gonna be very important as we start applying interoperability and data transfer between different hospitals now regulations. So hardware regulations we all know about right. There's a device that goes to the FDA FDA goes through certain cycles of approval and some of it requires premarket approval, some of it don't, but software is like a bit of a little bit of a black box. So software has been designated as a medical device, especially A I algorithms in the last few years. And um realizing the challenges of regulating software because software gets updated very quickly. The FDA came up with um a certain uh kind of proposal of how to do this. This is currently being, this is what they're doing right now, but it's also being studied So it's not their final uh kind of uh uh uh approval strategy for software. But basically what it's doing is it's assigning software a certain level of risk and it's assigning companies a certain level of risk. And then they basically, if you're a high risk device, it always is a high risk software. So for example, a software that diagnoses M I, that's a high risk software, it can really impact uh uh outcomes. So this requires approval every time. But if it's a low to mid level uh mid risk kind of uh uh software and being developed by a trustworthy company, like again, uh I don't know fitbit App or any of these companies that developed many before and has worked with the FDA, then they can go to the market without having to go back to the FDA. So it's gonna become this kind of collaboration between companies and the FDA. And then when they go to the market with these algorithms, they need to collect post marketing data and that kind of feeds back to the FDA and it gets monitored regularly for any issues. But this is very important. I'll show you some examples why these are some examples of all these of algorithms including a life board here, which you might be familiar with that have gone through FDA approval over the past few years. So this is actually an active process and there's a lot of algorithms that are being approved now, clinical workflow integration is an important. Uh But I'm gonna mention it briefly. So I've talked about all these algorithms and what they can do, but sometimes what we fail to talk about is how do we integrate in our practices? Like how, how am I gonna use A I in my daily clinical practice? Um And this gets complicated. Let's say you want to use the Amazon stuff uh again, for automatic building. So you want to think about infrastructure, what E hr s you have, how it's gonna integrate with it? How's the interface gonna look like who's gonna build the interface? Because me trying to do this, it gets complicated very quickly because then you're, you have this great thing that you want to try. But then you have your institutional epic engineers who are or it people who in some institutions, there are not many and they have like a list of things to go through like building this order data set and doing this. And it takes it could take a year or two to even integrate something into it into Epic. Even though you have the algorithm personnel, who's gonna, let's say you're again entering an algorithm that is predicting heart failure, who's gonna monitor that and who's gonna react to it. So for example, I don't know what um you're using Pace Mate or pa art for all these different uh Cie Ds. But if you're using Pace Mate now they're tracking all this heart failure information from algorithms like heart logic that Boston has and it's giving you a number. Thank you. This guy's heart logic, you know, number has gone up over the past few months, which means that they're doing worse from heart fail perspective. But we just see this information, we just like no one does anything about it because we're eps and, and all that. So there's all this information that's there and it's been studied and it's been proven. But who is the person who's gonna respond to it and, and, and call the patient and maybe bring them in to check on them, blah, blah and then parameters one numbers below which we start raising red flags. Oh This guy is really that hard. He's gonna get hospitalized next week if I don't do anything about it, that kind of thing and then billing and blah blah blah. Now I'm gonna dive into some A I specific challenges here as the last part, remember these algorithms are so powerful. OK? So they can, they are so powerful that they can over fit the data over fit the data means that they're making the data perfect for the like. So they take the data and they develop the function basically is so perfect at predicting um whatever you ask it to predict. But then when you take it and you apply it in an external uh uh data set, it does horribly because it was just over fitted to the, to the data that you gave it. So you want something that's right in the middle, that's an ideal fit, that's not perfect. But then is more Generali able to other datasets in machine learning. Unlike traditional statistics, it's very important to have a uh an internal or external test set and a validation development uh set. So most statistics you'll develop an uh a model and then you'll kind of apply to another, you know, a different data set could be internal or external. Now, deep learning what it does is it takes, let's say it gave it 100,000 data points, it takes it and splits it into a training data set and a validation data set and then this becomes a dynamic process. So it labels part of it as training, part of it as validation and then that changes and it keeps doing this uh tuning or tuning hyper parameters and assessing performance and overfitting it just a cycle. So the validation data set really leaks into the training data set. That's why it's always important to take this and apply to an external data set. OK. And a lot of ML, like all the studies have been published, you take an ML model, you apply to external data set always performs worse. OK? Not necessarily much worse, but it always performs worse. And these are some examples here of uh studies where uh uh once you start applying it to an external data set, it starts performing a little bit worse now, uh poor general disability uh is also a problem due to overfitting of irrelevant features. So this is an example of uh a study where they were trying to use uh uh uh computer vision algorithm to detect pneumonia. All right. And they happened to uh do it at three different hospitals. However, one hospital is where most of the pneumonia was and, and that hospital which is Mount Sinai, they happen to uh uh label their, put these labels to determine left and right uh uh in a certain way versus other hospitals, which put it in an inverted way. So when you actually looked at the algorithm's performance externally, once you start external validating it, the A UC was horrible. OK? So A UC was great when you looked at every different hospital. But when you started using it across other hospitals, it was horrible because the algorithm was basically annotating to this. So, yeah, you have to be careful and you have to take a really deep dive into these things. Um All right. And this is another example of a study where they were trying to uh use a deep learning model to distinguish Tau from acute mac cardial infarction. Again, the distributions here were not exactly the same between the training cohort and the test cohort. So you can see, for example, they used uh more of the Phillips uh ie 33 model. So when with, with this model, there were 70 patients versus with the test cohort, it was mostly patients with acute myocardial infarction. So when you start looking at this, you start realizing that the computer was also an to this label there. OK. So you have to be very careful with these things. Of course, the the authors then took this and they uh build this segmentation network so they fix all of this. But again, this is why the initial data wasn't great. Now, um black box stuff uh so bias, racial bias in machine learning, this has been a big thing. And these are two examples. Uh uh One here is um this, if you guys are familiar with Optim, it's uh basically United Healthcare's uh kind of venture capital slash uh startup uh arm. And they developed this deep learning algorithm uh that they thought was predictive of uh uh um um basically how sick patients are and uh a group of academics took it and they found that uh uh blacks always had uh uh a higher score. And um uh and they, they, so basically here, the point is um that black patients were assigned the same level of risk, but they actually happened to be sicker. And the reason is they were using health care cost as a way to predict this risk score. So sicker black patients and, and less sick white patients were had the same health health care cost needs and they were getting the same score. But actually the black patients happened to be sicker man. This is another company from Canada that came up with uh a way to diagnose Alzheimer's from voice recognition. Now, the problem they face is it didn't work for non native Canadian English speakers. Ok? Because again, they trained it on a specific accent in Canada. Once they started applying it externally, it just didn't work. You can all imagine how that would look like. Um So, yeah, so, and again, this is a recurring theme. So it's always something that we have to be careful about. OK. And we have to make sure we have enough diversity. We have to really examine these algorithms look into their depth, do some studies like this trying to kind of figure out uh why the algorithm is failing or if it's failing, et cetera. And that's where the FDA is input is also going to be important. This is uh uh uh uh Jeff Hinton, which is again one of the fathers of uh deep learning. And uh he said that um he predicted that in five years, uh deep learning is gonna do better than radiologists and that was in 2016 and it hasn't yet. OK. So again, this is, comes to the whole thing of is it gonna replace uh us? Well, no, it will not, it will help. Uh And when people worry about the existential threat of A I Andrew and is one of the big A I scientists at uh at Stanford. He says, like worrying about overpopulation, Mars when we have not even set foot on the planet yet. Now, this is a great example of where A I can fail you. So this is a game from the second division of the English Premier League and they had this camera that is supposed to track the soccer ball. OK? And basically the idea here is that you would, you know, you would not need a camera man and people were watching on TV and just kept doing this and, and they started trying to figure out what was going on. And basically what I was doing was it was tracking the referee's head, think it was the ball. So what's the solution? Either no ball referees or don't use the algorithm? OK? But yeah, that's basically what it came through. Um and this has happened and this like this is the funny part of it, but this has been serious. So Ubers um um um uh self-driving A I uh last year, there was this big incident where it, uh you know, some driver had it on and was sleeping and there was um um a biker and it recognized it as uh another car or something like that or I can't, no, I can't remember exactly the details but it didn't do what it was supposed to do. And basically the, the car just slammed into the biker and, and they died. So it, it, it can get very serious and scary. So I think, uh I agree with e tool here that the great opportunity offered by A I is not really reducing. Um, I mean, it, it, yes, there's an opportunity there but what it does is the opportunity to restore the precious and time, honor, connection and trust. And again, we've talked about how NOP can be helpful and so on. So basically, it does the things that allow you to be to have the time to talk to your patients and you know, do a lot of the stuff that we've missed on because of the, all the paperwork and all this stuff. And then this is uh how I think the future of A I is gonna look like it's gonna be a collaboration here. There's an A, this is it a readers uh reading uh uh uh nuclear stress test with an algorithm. And you can see that with the algorithm, they perform better. And this is a study of uh uh uh uh also dermatologists reading uh pathology reports. And again, the collective uh A I plus the collective reading ability of dermatologists gives you the highest accuracy. And this is uh this is uh exactly what I think. Again, the future is gonna look like artificial intelligence will not replace cardiologists and sonographer. But cardiologists and sonographer who use A I will replace those who don't, so don't miss an opportunity to use A I uh whenever you get the chance to uh this is stuff that we're working on at N US C right now, we're trying to build uh um kind of a comprehensive uh data analytics, uh uh uh um a pipeline here trying to bring in data from CIE Ds and wearables and cardiac MRI and all that stuff. And of course, it's a lot of work, especially when the a lot of the infrastructure isn't there yet. But again, this is what Zao did in 2014, they were able to gather all their data in one place. And if we can do that, then um this can be huge. And remember, we always think that companies like Apple and Zao and so on are very powerful and they have all this data and they're the ones who can do these things. Uh What we don't realize that all these companies don't have patient data they like or or comprehensive patient data. So Zao will have ecgs but they have no clue what these patients. Um you know, history is or age or I mean, they know age but like what medications they're on, what their outcomes are and so on. And that's why they'll come to you and try to collaborate. So as an institution, if you build or invest in building these platforms, it also becomes a huge hub for industry to come in and wanna partner with you and want to build things and wanna uh build companies and then you can, of course, uh could develop things with them, share patents and so on. So this can be, once you invest in something like this, the potentials of it over 10 to 20 years can be huge. And with that, I'm, we have five minutes, I guess for questions. Published June 29, 2023 Created by Related Presenters Mohamed B. Elshazly, M.D., MBEE Medical University of South Carolina
