• If A.I. Can Diagnose Patients, What Are Doctors For? — The New Yorker

• A.I. Is Making Doctors Answer a Question: What Are They Really Good For? — The New York Times

Let’s set aside for now the surgeon’s technical performance in the OR; what I want to examine here is the relational work we do before the patient ever reaches the table. The growing public sentiment captured by these headlines is being exacerbated by those physicians who view AI as some sort of existential threat to our profession. They bristle when the patient arrives to their clinic appointment with chatbot logs or the patient’s spouse asks whether a particular test ChatGPT suggested might be helpful. At professional society meetings, they often point to the physician’s physical presence as something AI could never supplant: “sure the AI can give a diagnosis or recommend a course of action, but can it give the patient a hug, or hand their spouse a tissue?”

Meanwhile, multiple publications comparing the perceived empathy of chatbots vs. human clinicians have demonstrated that both patients and third-party evaluators often rate the chatbot as more empathetic and compassionate than the human. Whether or not the perceived empathy of these chatbots is being fueled, on some level, by sycophancy is a separate question — do people prefer the chatbot because it agrees with them, while the human will tell them they’re wrong? Are patients confusing validation with compassion? This distinction matters more than it might seem — a medical chatbot optimized for user approval may be doing something closer to the opposite of medicine, and the downstream consequences of that dynamic are only beginning to be understood.

Whatever the mechanism, patients don’t seem to care that the computer can’t give them a tissue; the human clinician is subject to constraints the chatbot never faces: finite patience, the inability to be in multiple places at once, human limits like hunger and exhaustion, etc.

So, with this shared context, allow me to answer the question at hand here by presenting an entirely different frame for the patient-physician relationship, borrowing a beautiful concept my own therapist shared with me from a senior psychiatrist’s book of advice to the next generation of therapists.

In his book The Gift of Therapy, Irvin D. Yalom presents an alternative view of the patient and therapist: not as the afflicted patient and the invulnerable healer, but instead as fellow travelers. The parable he uses to illustrate this is drawn from Hermann Hesse’s Magister Ludi. A young healer named Joseph, himself devastated and seeking spiritual guidance, goes on a pilgrimage to find the great elder healer Dion. Along the road he meets an old traveler, who, learning Joseph’s purpose, offers to guide him. As their long journey unfolds, the old traveler reveals his identity: he himself is Dion — the very man Joseph sought. And two decades later, on his deathbed, Dion finally confesses that on the very night they had met he was on a pilgrimage to a famous healer named Joseph, seeking salvation from the emptiness and spiritual void he could not cure within himself. They are not the afflicted and the healer, but instead fellow travelers seeking answers to what Yalom identifies as the four central concerns of human life: death, isolation, meaning, and freedom. Yalom goes on to lament the twenty years of Dion’s secrecy, positing that the real therapy occurs at the deathbed, when the façade of superiority finally gives way to honest and authentic humanity.

And do the surgeon and their patient not grapple with the same fundamental concerns as the therapist and his client? The possibility of death and disfigurement; the isolation of being the only one who will be sedated and naked in a sterile room of masked strangers; the question of whether recovery will restore meaning, form, and function; the terrifying freedom of agency and consent in the face of an inherently unknowable future. These fundamental concerns are not mere psychological constructs but instead components of the human condition that permeate every physician’s work, whether they are a psychiatrist or a surgeon.

Why then, do some surgeons and patients cling to the notion of the surgeon as an imperturbable oracle, all-knowing and decisive, dispensing answers to supplicants who must simply trust and comply? Perhaps remaining behind the curtain allows the surgeon to conceal their own humanity, to hide the agony and terror wrought by opening the flesh of another human being. Perhaps believing the surgeon is superhuman allows the patient to more peacefully submit their life to another’s control, allows them to be vulnerable by clinging to the fallacy of certainty.

Like Dion, however, we must shed the veil of performance before a genuine encounter can occur and true human connection can be found.

So, to answer those newspaper headlines, this is what I am good for: not for handing you a tissue or offering superhuman diagnostic capabilities, but instead to provide the accumulated, embodied, experiential knowledge of the wise fellow traveler who has dedicated a lifetime to walking these roads. Shared decision-making and informed consent are themselves invitations for co-navigation. As we sit in the clinic or in the emergency room and discuss your diagnosis and possible next steps, I am not an infallible oracle, but instead simply a wise traveler who has walked these paths before. I have seen what the road looks like around bends the patient hasn’t reached yet; I know which fears are well-founded and which are the mind’s night terrors. The surgeon and patient are fellow travelers — both human, both facing uncertainty — but the surgeon alone has stood at these crossroads with hundreds of other patients, each of them here for the first and only time, grappling with the exact same decision the patient now faces.

The physicians who feel threatened by the patient with the chatbot printout and the journalists asking what doctors are good for have both confused the performance of medicine for its substance. They see medicine as an information-delivery system, and measure us against tools that deliver information faster and more patiently than we can. Little of our inherent value as physicians is captured in a differential diagnosis or a recommended lab panel. It lives, instead, in the moment when a patient looks up from the bed and asks “what would you do if this were your mother?”

That is not a request for data or validation. It is Joseph at a crossroads, tired and vulnerable, looking for Dion.

Subscribe now

“The FDA’s traditional paradigm of medical device regulation was not designed for adaptive artificial intelligence and machine learning technologies.” — the FDA

After 21,000+ hours of training and over a thousand major operations, I recently completed chief residency and finally became a board-eligible general surgeon. In the final two years of my training, nearly every elective abdominal or thoracic operation was performed using Intuitive Surgical’s da Vinci xi surgical robot.

Many surgeons routinely record and archive operative video to allow for “game tape” review: identifying inefficiencies, optimizing technique, and analyzing technical errors when complications occur. Until recently, this data was local, living on the hospital’s servers or the surgeon’s own hardware.

Earlier this year, Intuitive Surgical released the da Vinci 5 across the US and European markets:

The da Vinci 5 comes with two major changes from the previous model:

• The Intuitive Hub now automatically uploads all that “game tape” video to Intuitive’s cloud servers after a period of local storage. This was marketed to hospitals as a way to offload data management burden while enabling quality analytics.

• The system now provides haptic feedback, from the robot’s arms to the surgeon console. This also enables the device to capture kinematic inputs, precisely recording exactly what manual instructions the surgeon is inputting at any given time.

It doesn’t take a crystal ball to imagine where the industry is hoping this leads: intra-operative video and kinematic inputs are the exact training data an AI model would need to learn how to autonomously perform surgery.

Ask any robotic surgeon, and they’ll tell you we’re less than 5 years away from the moment when these systems stop just assisting and start deciding.

Sitting back from the console, watching this four-armed machine tower over my patient, I kept returning to one question:

Level 3 surgical robots are already on the market

Let’s first establish some common terms. The language of “levels of autonomy” is borrowed from autonomous vehicles (AVs):

Adaptive Cruise Control is considered Level 1 autonomy, while Tesla’s Full Self Driving is considered Level 2. Waymo’s cars, where the user sits in the backseat and is not responsible for monitoring the car, is considered Level 4.

Consider, then, that three of the 49 surgical robots the FDA has already cleared have demonstrated Level 3 autonomy — capable of executing certain surgical tasks independently under specific conditions.

Right now, the tasks that are being executed autonomously are relatively minor; the three Level 3 automations already on the market are a bone milling machine, an automatic prostate biopsy machine, and a hair transplant machine:

The manufacturer of this follicular unit extraction machine has explicitly re-branded as an “AI-first company”, even going so far as to change their TLD to “Venus.ai”.

The FDA is stuck in 2015

The next step — Level 4-5 autonomy — will be equivalent to the surgeon “sitting in the backseat” while the robot “drives”. At this level of autonomy, with the surgeon “in the backseat”, these surgical robots will be, in a practical sense, “practicing medicine”. Yet the FDA — who is exclusively responsible for regulating medical devices in the United States — explicitly doesn’t regulate medical practice. The FDA continues to refer to this entire class of technology as “robotically-assisted surgical devices,” implying no autonomous decision-making capabilities and creating a jurisdictional void. Who is responsible when one of the already-deployed Level 3 systems malfunctions during autonomous operation? The surgeon? The hospital? The manufacturer? The robot itself?

These malfunctions are not a theoretical risk. Over ten thousand adverse events related to surgical robotic systems were reported to the FDA between 2000 and 2013, in an era when all surgical robots were still at Level 1; device and instrument malfunctions, such as the falling of burnt/broken pieces of instruments into the patient (14.7%), electrical arcing of instruments (10.5%), unintended operation of instruments (8.6%), system errors (5%), and video/imaging problems (2.6%), constituted a majority of the reports. Furthermore, the rate at which these events are being reported has increased 32-fold since 2006, dramatically outpacing the 10-fold increase in robotic surgery volume over the same period.

A consumer may reasonably assume that in order to get a device cleared by the FDA, its manufacturer would need to produce an indisputable body of clinical testing data. In reality, only 39% of these systems included clinical testing data in their FDA submissions. While the submissions for all three Level 3 robots did include such data, some ML capabilities went entirely undisclosed:

2 of the surgical robots were reported to have machine learning-enabled software features in their submissions to the FDA. However, 3 additional surgical robots were marketed to have machine learning-enabled capabilities on their product websites that were not in their FDA summary documents.

In a way, this could have been expected: RAND has proven adequate test-drive validation for AVs would require billions of miles of driving data. How could a manufacturer possibly produce adequate clinical testing data for an autonomous surgical robot? Cadavers and simulation can only get you so far, as any medical student will attest. AVs can be tested in computer simulations, on closed courses, and even on public roads with backup drivers. Would you volunteer to be a “test track” for an autonomous surgical robot?

Who is building the guardrails?

The honest answer right now is: nobody. Not comprehensively, anyway.

The FDA is still regulating these systems as “devices” while they’re rapidly evolving into medical practitioners, which will place them explicitly out of the FDA’s scope. And manufacturers — eager to capture their share of a market estimated to be worth between $9.3 and $38.4 billion within the next ten years — are moving faster than regulators can track, while sometimes developing and marketing AI capabilities that are never even disclosed to the FDA.

“Autonomous AI/ML applications that can cause serious injury or harms (i.e., morbidity or mortality) to patients if they malfunction would be considered high risk…The burden of responsibility may shift toward the stakeholders involved in AI/ML development and deployment.” — the FDA

If we start now, we can still build the regulatory framework we need.

Over the next several posts in this series, I’ll explore what we can learn from autonomous vehicle regulation, who needs to be at the table to build a better regulatory landscape, and what a multi-stakeholder consortium for proactive regulation could look like before Level 4-5 systems reach operating rooms.

Subscribe now

Semmelweis observed medical students conducting dissection on cadavers in the pathologic anatomy labs between childbirths, and hypothesized an association between some sort of contamination event occurring in the anatomy labs and the epidemic of maternal mortality. He suggested that the elimination of this contamination event through antisepsis would lead to a dramatic reduction of this mortality; and indeed the implementation of mandatory chlorinated handwashing eliminated the Puerperal fever epidemic that had plagued the Vienna hospital for nearly half a century.

Semmelweis's hypothesis — intra-operative contamination events give rise to post-operative infections — would become the first principle upon which the next century of research around surgical infection would be built.

20 years after Semmelweis, in 1867, came Lister and carbolic acid.

Surgeons were desperate for a way to avoid infections, and aseptic principles were rapidly adopted. Below are two paintings of surgeons at work in Philadelphia in the late 1800s.

In 1875, Samuel Gross and his assistants are wearing their street clothes. No one is wearing gloves. Surgical instruments are stored in what is essentially a toolbox.

Twelve years later, in 1889, David Agnew is wearing surgical whites. He and his resident are wearing gloves. The operative field is isolated with drapes. Instruments are in a sterilization tray.

(Some things never change: in both paintings, medical students are asleep in the audience.)

After clean OR and handwashing and sterile instruments came antibiotic prophylaxis and fancier and fancier sterilization methods and laminar airflow HEPA-filtered operating rooms and double-gloving and all the other accumulated effort of the surgical discipline over the past century and a half to eliminate intra-operative contamination events.

But what do we observe?

Surgical infections keep happening!

With the advent of modern murine models, perhaps we can try and experimentally produce surgical infections in the lab so that we can study them under necessary and sufficient conditions.

Let's consider an experimental model1 in which mice undergo a surgery consisting of an incision and instrumentation at 6 sites along the muscles of the back.

At each site, electrocautery is applied and a suture ligature is placed. Cultures are obtained at the time of surgery, to confirm the surgical tissues are sterile up until this point.

The wounds are then directly contaminated with bacterial cultures intra-operatively, and again and again for several days. On day 10 after surgery, the mice are sacrificed so the wounds can be assessed using this clinical wound infection score:

Scores range from 0 to 8 and characterize the type of discharge, the degree of dehiscence, the presence of an abscess, and culture growth.

What do we observe?

Despite direct contamination of the surgical wounds with bacterial cultures, clinically significant infections develop in only 20% of the mice. This sort of yield isn't nearly sufficient for experimental testing — 80% of our specimens would never even develop clinically significant infections to be experimented on.

It is only when we replace standard mouse chow (standard diet, SD) with a low-fiber, high-fat, high-carb western diet (WD) that we begin to reliably produce surgical site infections in this experimental model.

What if the bacteria causing these surgical infections are coming from the gut microbiome instead?

We can propose the following experiment, in which mice are not directly contaminated, but instead undergo a sterile surgery. There are four groups, western diet vs standard diet and peri-operative antibiotics versus none.

And what do we observe?

Rather than the introduction of antibiotics reducing infectious rates in the WD group back down to the rates for the SD group, we find that giving antibiotics worsens both the clinical wound infection score as well as increases the rate of polymicrobial infection in both the WD and SD groups.

What could possibly be going on here? We are sterilizing the animal with IV antibiotics at the time of surgery, and this is producing more infections? Where are these bacteria coming from?

Again, we can test this experimentally2, and with fluorescence imaging we can demonstrate that the causative bacteria originate at mucosal surfaces like the gums and the gut lining.

Image D on the left is a cecum of a mouse that did not receive antibiotics. Image C shows the cecum of a mouse that did receive antibiotics, which has now become colonized by fluorescent MRSA.

We can watch under FISH imaging as this MRSA is silently taken up by neutrophils at the mucosal border through phosphate-seeking chemotaxis behavior that is conserved by (effectively) all bacteria. These trojan-horse neutrophils can later travel to a remote site of inflammation — like the surgical wound in image B on the right — and release their infectious payload.

This is the Trojan horse hypothesis, in which the intestinal epithelium becomes progressively colonized by phosphate-seeking bacteria, which are silently transported to remote sites of inflammation within neutrophils, where they eventually release their infectious payload and produce surgical site infections far remote from the site of true "contamination" at the mucosal barrier.

What can we learn from this?

Well, at some point between Semmelweis and single use scrubs, the causal inference got out of hand.

What Semmelweis observed in 1847 was obviously real, and the advent of germ theory and antibiosis and aseptic surgery has saved countless lives.

At the same time, the discoveries that initially led to progress eventually became assumptions that limit our ability to see the error space and continue to produce useful improvements.

Subscribe now

FIRST

July 2009. My father and I drive home from the movies discussing plot holes. As we pull onto our exit ramp, an unfamiliar sight in such a familiar place quiets the discussion.

A white car, laying upside-down in the ditch beyond the road. A man strapped inside, white hair matted with blood. Huge glass shards strewn across the grass, acrid smoke blackening the sky. Half a dozen bystanders along the side of the road; a single woman squatting beside the deformed window, the only attendant at this unexpected wake brave enough to approach the metal coffin.

We approach the car and squat beside the window. My father offers the woman a bottle of water. "Do you think my wife will still love me?” the man gurgles from within. The woman pours lukewarm water through his hair, combing out clotted blood with her fingers.

Behind us, the wail of sirens. I count the patterns; one, two, three separate sirens, becoming deafening far faster than expected. Three gleaming red trucks arrive bearing men clad in armor, each retrieving the equipment needed for his role before descending into the ditch. Red Helmets brace the vehicle, remove the door, and cut the seatbelt. In a single smooth motion, they transfer the man onto a thick orange backboard and carry him to the waiting doors of an ambulance. As rapidly as they had arrived, the sirens fade away.

I think to myself,

“What a life it would be, to become one of them.”

SECOND

February 2011. I carry my turnout gear and Yellow Helmet through the engine bay, reviewing protocols and equipment under my breath. An unfamiliar sight catches my eye: beside the usual Red Helmets alongside the ambulance, a Blue Helmet labeled MEDIC. I enter the firehouse, and find a new face. With a warm smile, a woman with short hair and red cheeks introduces herself. I say I have never met a medic before, and ask if I can see her gear. She explains the contents of her med box, her intubation kit, her IV kit, her trauma bag. I ask if I can ride the ambulance today instead.

An alarm I have never heard before sounds overhead.

“MEDIC TO RESPOND”

A geriatric man, slumped on the floor against his toilet. Blood seeps through the man's pants and drips off the backboard as we extract him down the narrow stairwell. By the time the ambulance pulls away, the man is unconscious. I am very afraid. With calm, practiced hands, the medic places an IV and attaches her 12-lead monitor. I do not see fear, but instead focus and confidence. As the driver and medic wheel the patient past sliding double doors, I stay behind to clean and restock the ambulance for another dispatch.

Nine weeks later, a familiar alarm sounds overhead.

“MEDIC TO RESPOND, P1”

Now focused and confident, I ventilate the patient during transport. When we arrive, I continue ventilating into the trauma bay; it is my first time through the double doors. We arrive to find men and women clad in a different armor. Surrounded by chaos, a woman in a long white coat issues commands in an even, measured voice. Members of the orchestra close in around the bed. I watch until my crewmates come to retrieve me.

I think to myself,

“What a life it would be, to become one of them.”

THIRD

August 2016. A 63-year-old man is found to have a large, asymptomatic common hepatic artery aneurysm involving the bifurcation.

I listen as two attendings and the senior fellow discuss the particulars of the case; observe as they review imaging and plan their approach. I scrub, reviewing visceral anatomy under my breath. We apply bright blue drapes, and the Surgeons open the human machine. I feel wet stomach slide beneath my fingers as the calcified, golf-ball-sized aneurysm comes into view. As they control vessels and begin excision, bright red blood begins to pour into the field. Recoiling, I suction as they attempt to control the bleed. A minute passes, then another. "He's getting a little hypotensive, in the 80s now." With calm, practiced hands, the Surgeons localize a miniscule posterior rupture of the gastroduodenal just distal to the aneurysm. I do not see fear, but instead focus and confidence. I listen as the Surgeons collaborate in even, measured voices. They ligate the GDA; the scarlet waves cease, ebbing to the beat of the pulse oximeter. They complete the case. Each day, I wake my patient in the early morning and watch as he recovers uneventfully. Each day, I return to the OR and witness the work of calm, practiced hands.

I think to myself,

“What a life it would be, to become one of them.”

The Brain Enrichment Program was started by an MD PhD student one year above me at BUSM named Katelyn Blackburn. She and her classmates Pamela Huang and Alexis Clay converted a hobby project she had during her years before medical school into a club at BUSM that would forward the same mission.

What we all observed was that assisted living homes were filled with residents who were still enjoying capable lives. They were former working, thriving members of our society who had simply become feeble enough to require some daily assistance. With that aging had come illness; they and their friends and peers had begun to experience medical disease and care.

At the same time, we found healthcare professionals who were less-than-ideal at sharing information about disease and therapy with their patients. We, the medical students, by and large shied away from trying to explain complex disease to the elderly because we felt we were not adequately prepared to do so.

So, a solution emerges: we can go to our patients where they live, and practice talking about the pathology and therapy we are fascinated by to the population it affects the most. We began a regular schedule of short talks with a consistent agenda: a brain warm-up of a few puzzles and critical thinking exercises, followed by a 30-45 minute interactive talk about a specific topic relating to brain science. The BEP has covered an enormous range of topics in its 5 years, ranging from the psychology of music and meditation to the science around concussions in the NFL to deep brain stimulation implant surgery and Naegleria fowleri infections.

We took advice from our audience about what topics to cover and how to improve our presentations.

We implemented systems for regularly attending each other's talks and methods for providing each other with comprehensive feedback so we could improve together.

We expanded our presentations to a second site, doubled their frequency at both, and expanded our roster of medical students.

We began creating a package to help other medical schools found satellite clubs like ours.

And we provided a platform for fellow students to practice talking about scary things -- the foundation of the doctor-patient relationship and the core of what we are training to do.