Australian Government Warns Doctors Over AI Scribing Tools as Privacy and Safety Concerns Grow
核心洞察
The proportion of Australian doctors using AI scribes nearly doubled from 22% in August 2024 to 40% by November 2025, according to an RACGP survey.
Government briefing papers describe AI scribing tools as having "little oversight," with some marketed as operating outside medical device regulations despite clinical use.
Approximately 20% of AI-generated clinical notes contain errors significant enough to impact patient diagnoses, raising concerns about patient safety and record accuracy.
The Australian government is urging healthcare professionals to exercise caution when using AI-powered medical scribing tools, as regulators examine whether stronger safeguards are needed around one of healthcare's fastest-growing technologies. According to briefing papers obtained by The Guardian Australia through freedom of information requests, Australia's federal health department has identified several risks associated with AI scribing tools, describing the technology as having "little oversight" and noting that some AI scribes are marketed as operating outside existing medical device regulations despite being used in clinical settings.
The concerns come as adoption continues to surge. An online survey conducted by the Royal Australian College of General Practitioners (搜索) (RACGP) found that the proportion of Australian doctors using AI scribes nearly doubled from 22 percent in August 2024 to 40 percent by November 2025. Technology providers also claim their platforms have processed hundreds of millions of consultations globally over the past 18 months.
How AI Scribes Function and What Can Go Wrong
AI scribes record, transcribe, and summarise doctor-patient conversations into clinical notes, feeding what they hear into large language models similar to ChatGPT or Claude. However, these systems do not simply transcribe text—they summarise and interpret conversations based on what the AI assumes a medical professional might say.
A critical concern is AI hallucination, where systems confidently state things that never happened. As one example cited in the Digital Rights Watch (搜索) report Off the Record, a doctor counselling a patient to quit smoking might find the AI confidently recorded that the patient was advised to avoid house fires. Early studies suggested up to 90% of AI-generated notes required correction, while more recent studies show that approximately 20% of AI clinical notes contain errors significant enough to impact patient diagnoses.
Catching these errors presents its own challenge. Humans struggle to identify others' mistakes under time pressure, and automation bias leads clinicians to accept AI output without scrutiny. As the Digital Rights Watch (搜索) report notes, "If clinicians must meticulously verify every note for potential AI-related mistakes, the promised productivity gains likely evaporate. If they don't, the health system will be flooded with unreliable data."
AI systems also frequently show biases relating to race, ethnicity, class, and gender. For patients from culturally and linguistically diverse backgrounds, or those for whom English is a second language, AI interpretations of speech can lead to medical records that "wildly diverge from reality."
Privacy and Data Sovereignty Risks
Privacy has emerged as another major concern. Officials found that some providers promote their products as privacy compliant while offering limited transparency about how patient information is processed. In some cases, healthcare providers may not even realise patient data is being transmitted to cloud servers outside Australia, potentially exposing sensitive medical information to additional risks.
The Digital Rights Watch (搜索) report highlights that AI scribes typically fail to disclose how data is handled, who has access to it, and what AI models are used. "Beyond vague assurances that data remains in Australia, health-care professionals simply cannot give patients concrete answers about who is accessing their data or what they are doing with it," the report states.
Medical professionals themselves are not well-equipped to provide guidance on data handling. Their professional bodies provide only high-level information, and clinicians are not AI experts capable of discussing these privacy issues with patients.
Patient Consent and Medicare Implications
Patient consent is another area drawing attention. The government found significant variation in how clinics obtain permission before recording consultations, arguing that meaningful informed consent requires patients to understand both the benefits and limitations of AI-assisted documentation. Consumer groups have reported instances where patients were told they would need to find another healthcare provider if they declined to have AI scribes used during appointments.
The health department also questioned marketing claims suggesting AI scribes can increase doctors' revenue by around 30 percent without extending working hours or seeing more patients. Officials noted that such claims could have broader implications for Australia's publicly funded Medicare system if higher billing becomes a primary incentive for adoption.
The Regulatory Landscape
Oversight is currently shared between the Therapeutic Goods Administration (搜索) (TGA), the Australian Health Practitioner Regulation Agency (搜索) (Ahpra), and the Office of the Australian Information Commissioner (搜索). The TGA is currently reviewing whether AI scribes should be formally classified as medical devices—a decision that could bring many platforms under stricter regulatory oversight. A report is expected in the coming months.
The TGA currently claims scribes are only medical devices if they provide diagnostic advice, a position that Digital Rights Watch (搜索) argues allows vendors to evade scrutiny. The advocacy group contends that the TGA's stance "ignores how generative AI is used in clinical practice, and that even transcriptions can contain hallucinations or biases that could affect diagnoses," leaving Australia's guidance out of step with nations such as the United Kingdom.
Digital Rights Watch (搜索) has called for the TGA to regulate AI scribes as it would any other medical device, subjecting them to the same safety testing requirements as any other piece of medical equipment. The organisation also urges the government to force AI service providers to clearly disclose what data they have access to, how they use it, and who can access it, with rigorous protections to ensure privacy and real penalties for breaches.
The debate reflects a broader challenge facing healthcare systems worldwide. AI tools are increasingly capable of reducing paperwork and giving doctors more time with patients, but governments are being forced to balance those efficiency gains against the need to protect patient privacy, ensure clinical accuracy, and establish clear accountability when artificial intelligence becomes part of medical decision-making.