Utah has quietly become the first US state to allow an artificial intelligence chatbot to renew prescriptions without the direct involvement of a licensed physician, according to an investigation by the Associated Press. The program, operated by a company called Doctronic, launched in January 2025 and has ignited a fierce medical debate that touches on patient safety, regulatory oversight, and the rapid deployment of AI in healthcare.
How the AI prescription renewal system works
Residents of Utah can now bypass the traditional doctor's office visit and refill their prescriptions entirely online through an AI chatbot. The system asks patients about their medication, medical history, and current symptoms. It then cross-references this information with a national pharmacy database. If the AI determines the prescription is safe to refill, it issues a renewal. If it identifies any potential issues or uncertainties, the case is escalated to a human doctor for review.
This process is far from the standard practice in the United States, where prescribing medications is traditionally restricted to licensed medical professionals, such as physicians, nurse practitioners, and physician assistants. Federal and state laws have long mandated that a human clinician be involved in prescribing decisions to ensure patient safety and appropriate use of medications.
The regulatory sandbox that made it possible
Doctronic's program was made possible only through a unique Utah state initiative known as a "regulatory sandbox." This framework allows state officials to temporarily waive certain licensing and regulatory laws for innovative AI applications that promise significant benefits. The sandbox is designed to encourage technological innovation by removing barriers that might otherwise stifle promising startups. However, critics argue that in this case, the sandbox has allowed a dangerous loophole that could put patients at risk.
The Utah Office of Administrative Rules approved the sandbox for Doctronic, effectively granting the AI chatbot the authority to perform tasks that would normally require a medical license. The company argues that its system is simply an enhancement of medical practice, not a replacement for a doctor, since human doctors still review every final decision in the current phase. However, the company has stated its intention to move toward fully automated prescription renewals in the near future.
The medical board caught off guard
Utah's medical licensing board, known as the Utah Medical Board, was blindsided by the launch. Board members reported learning about the program only after it made national news in January. In April 2025, eleven board members sent an official letter to state officials calling for an immediate halt to the pilot program. They cited serious risks associated with auto-renewing drugs that can have dangerous side effects or adverse interactions with other medications.
Dr. Alan Smith, a family physician who chairs the board (though he made clear he was speaking for himself and not the board), expressed frustration at being left out of the decision-making process. "We were essentially told: 'Yes this is going on. And no, you don’t have a say in it'," he said. The medical board's concerns were largely dismissed by the state, which declined to suspend the program, noting that during this first phase human doctors still review every refill. However, the board warned that the path to full automation is dangerously close.
The risks of automated prescription renewals
The American Medical Association (AMA) has publicly voiced concerns about the safety of AI-driven prescription services. In a statement, the AMA emphasized that "prescription renewals aren’t routine checkboxes" but complex medical decisions that require clinical judgment. Medications like blood thinners, for example, can become extremely dangerous if a patient develops internal bleeding. An AI chatbot may not be able to spot subtle signs of complications that a human doctor would catch during an examination.
Doctronic's current list of roughly 190 refillable medications includes such high-risk drugs. While the company claims its algorithm is rigorous, critics point to a lack of independent validation. The only published study by Doctronic was written by its own scientists and has not undergone peer review. The company plans to release peer-reviewed research later in 2025, but for now, the evidence supporting the system's safety remains proprietary.
Health safety researchers have long warned that medical chatbots can sound authoritative while dispensing dangerous advice. Studies have shown that large language models can generate confident but incorrect medical recommendations, especially when faced with ambiguous patient histories. Removing humans from the care loop altogether risks not only individual patient harm but also a broader erosion of trust in the healthcare system.
A regulatory vacuum by design
The Utah case exposes a significant regulatory vacuum that exists between federal and state oversight of medical AI. Medical devices, including software used in healthcare, are regulated by the U.S. Food and Drug Administration (FDA). Meanwhile, the practice of medicine, including prescribing, is regulated at the state level by medical boards. Doctronic has framed its AI as part of state-regulated medical practice, thus arguing it does not require FDA approval. However, some legal experts contend the system has crossed into FDA territory because it effectively makes prescribing decisions.
The FDA has reportedly told the Associated Press that it has not authorized any AI chatbot for prescription renewal purposes. At the same time, the agency has signaled a desire to encourage innovation, adopting a relatively hands-off posture that fits a broader loosening of oversight on AI health tools under the current administration. This regulatory ambiguity creates a gray area where companies can push forward with unproven technologies, leaving patients and practitioners uncertain about who is ultimately responsible for patient safety.
Historical parallels and warnings from experts
Dr. Eric Bressman of the University of Pennsylvania has drawn historical parallels between the current AI boom and the haphazard state of medicine in the early 20th century, before modern medical boards and rigorous licensing standards were established. "We have crossed a threshold in terms of giving something that is not human a medical license, whether or not we want to call it that," he said. He and other experts are not opposed to AI in prescribing per se, but they insist that any system must be held to standards as rigorous as those applied to human doctors.
The Cicero Institute, a pro-AI think tank founded by Palantir co-founder Joe Lonsdale, has been promoting a template for licensing AI medical services in other states. This think tank has advocated for the use of regulatory sandboxes similar to Utah's, arguing that they accelerate beneficial innovation. However, critics contend that the Cicero Institute's model prioritizes speed over safety and may encourage a race to the bottom among states competing for AI investment.
Meanwhile, major technology companies like Meta have been conducting their own safety evaluations of AI chatbots. Meta went so far as to pose as teenagers to test how competing chatbots handle sensitive topics, revealing that many models struggle to maintain safe boundaries. These findings underscore the general unreliability of current AI systems in high-stakes environments like healthcare.
The stakes in Utah are not abstract. If the Doctronic program proves successful, it could become a model for other states. Conversely, if a patient suffers a serious adverse event due to an AI-prescribed medication, the fallout could be catastrophic for public trust in both AI and state regulatory bodies. As one Utah law professor noted, companies risk letting the technology race beyond the evidence, potentially betraying public trust in the process.
For now, the Utah experiment continues, watched closely by healthcare professionals, regulators, and AI developers across the nation. The medical board remains on the sidelines, its concerns overridden by a state government eager to lead in AI innovation. Whether the pilot ultimately advances medicine or endangers patients will depend on the quality of the safeguards in place—and the ability of regulators to act before it is too late.