AI has become part of almost every conversation in healthcare. You see it in clinical trials, patient engagement tools, and data analysis platforms. The expectations are high. Faster decisions. More efficient studies. Deeper patient insights. But beyond the excitement, there’s a more important question to answer: what does responsible use of AI actually look like in clinical and patient-focused research?

We explored this in a recent LifeSci Talks episode, where our COO and Co-Founder, RJ Kedziora, joined Mark Wade, Global Practice Leader at TransPerfect Life Sciences. The discussion stayed practical. It moved past the buzzwords and focused on how AI is really being used today, where it adds value, and where caution is still needed.

If you’re working in clinical research, digital health, or patient experience, this isn’t theoretical. AI can help you work more efficiently and uncover insights you might otherwise miss. At the same time, if it’s not used carefully, it can introduce real risks. Finding that balance is what responsible AI is all about.

What Does Responsible AI Look Like in Clinical and Patient-Focused Research?

• Keep a Human in the Loop

In clinical trials, digital health platforms, and medtech systems, decision-making is layered, regulated, and high-risk. AI can accelerate workflows, but it cannot replace accountability. That distinction is now widely reflected in how AI is being deployed in real-world healthcare environments.

Recent clinical AI research shows that the most reliable implementations are not fully autonomous. Instead, they use AI as a supporting layer alongside clinicians. A 2025 study on AI-assisted clinical trials found that using AI as a secondary reader, rather than a replacement, preserved trial accuracy and improved reliability even when models performed poorly. This reinforces a key principle. AI works best when it supports human judgment, not when it replaces it.

This is exactly how RJ explains that adoption has not reached full automation, noting that “we haven’t made that leap to the AI first round interview just yet. We’re still using humans for that.” In a clinical setting, that same logic applies to patient screening, eligibility checks, and endpoint validation.

He is equally clear about the limits. “Yes, a human still has to be in the mix. It’s an 80% solution.” That remaining 20 percent is not trivial. It includes clinical reasoning, ethical interpretation, and responsibility for outcomes.

For teams working in life sciences and digital health, this means redesigning workflows rather than removing people. AI should handle scale and speed. You remain responsible for decisions that affect patients.

• Address Bias in AI Systems

Bias is not an abstract concern in healthcare AI. It directly impacts patient inclusion, treatment pathways, and clinical outcomes. As AI adoption grows, so does the risk of reinforcing existing health disparities.

A 2025 review in healthcare AI highlights that bias can enter at every stage of the model lifecycle, from data collection to deployment, and can “exacerbate healthcare disparities” if not actively managed. This is particularly relevant in clinical trials, where underrepresentation of certain populations is already a known issue.

RJ acknowledges this reality without overstating it. As he put it, “Yes, it’s a challenge. It’s a problem. It’s something teams are actively dealing with, not something you can ignore or assume will fix itself.” That matters because it reflects how AI is actually being used in industry. They show up in real workflows, where outputs are helpful but still need to be checked, refined, and sometimes corrected.

The important shift comes in how the problem is addressed. RJ points out that “there are known methodologies to implement to help alleviate those.” It is about identifying it early, monitoring it continuously, and correcting it systematically.

There is also increasing pressure from both regulators and healthcare leaders to raise standards. RJ added, “We’re raising the bar of what it needs to be able to do for us to find and to actually use it.”

In practice, this means:

• Validating datasets before model training

• Testing outputs across diverse patient groups

• Monitoring models post-deployment

For medtech and life sciences teams, bias is no longer a side issue. It is already part of model validation and regulatory readiness.

• Control Hallucinations and Critical Errors

One of the most immediate risks in healthcare AI is hallucination. These are outputs that are incorrect, fabricated, or unsupported by evidence but presented with confidence.

Recent research confirms how serious this issue is. A 2025 study defines medical hallucinations as outputs that are “factually incorrect or unsupported by authoritative clinical evidence” and warns that they can directly influence clinical decisions and patient safety. Reports from 2026 also show that hallucinated content is already appearing in scientific literature, raising concerns about reliability and compliance in research environments.

RJ explains, “There are clearly situations where you have to be absolutely certain… where it’s life or death.” In those situations, the tolerance for error is zero. “You can’t have any hallucinations.”

This is where responsible AI becomes operational. It is not about whether AI is useful. It is about where it is safe to use.

RJ also acknowledges that failures happen. As he put it, “There are clear examples where the AI goes off the rails, and when it does, it’s not always obvious until you take a closer look.” That matters because it reflects the current state of AI deployment in healthcare. Performance is improving, but reliability is still uneven, especially in high-stakes scenarios where errors are harder to catch and carry real consequences.

For digital health and clinical research teams, the response is not avoidance. It is controlled usage:

• Use AI for data synthesis and pattern recognition

• Avoid relying on it for final clinical decisions without validation

• Build review layers into every critical workflow

Hallucinations are not a theoretical risk, but instead, they are a current limitation that must be managed through system design.

More: What Are the Benefits and Risks of AI in QA/RA Compliance?

• Apply Checks and Balances Like in Healthcare

Healthcare already operates within strict systems of validation, oversight, and accountability. Responsible AI does not require reinventing these systems. It requires extending them.

Frameworks developed in 2025 for clinical AI adoption emphasize the need for structured evaluation, including data quality checks, explainability, and human oversight throughout the lifecycle. Similarly, governance frameworks for AI-driven clinical research highlight the importance of review processes that address bias, transparency, and adaptive learning before deployment.

RJ explains, “Even with humans, you’re going to have multiple checks and balances.” That is already standard in trials, diagnostics, and regulatory processes.

The extension is straightforward. “Implement those same checks and balances with the AI.”

This is not theoretical guidance. It is operational:

• AI outputs should be auditable

• Decisions should require human validation

• Systems should include fail-safes and rollback mechanisms

RJ reinforces the expectation clearly. “If you’re using it for medical purposes, you are going to put those appropriate checks and balances in there.”

For medtech and life sciences organizations, this is where responsible AI becomes part of compliance, not innovation alone. It supports regulatory alignment, builds trust with stakeholders, and ensures that AI systems can withstand scrutiny.

• Use AI as Support, Not a Decision-Maker

The most effective AI deployments in healthcare today are not autonomous systems. They are collaborative systems that extend human capability.

Research continues to support this model. Studies on AI-assisted decision-making highlight ongoing concerns around accountability and transparency, particularly when AI is used without clear human oversight. These concerns are especially visible in regulated environments where responsibility for outcomes must be clearly defined.

RJ emphasizes that performance depends on how it is used. “Computers only do what you tell them to do. They don’t do what you want them to do.”

This is critical in clinical and patient-focused research. AI does not interpret context the way clinicians do. It executes based on input.

He illustrates this clearly. “It did what you told it.” That statement highlights a common misconception. AI does not independently reason. It is responding to instructions.

RJ added, “It’s an 80% solution… a really smart intern.”

For teams in digital health and life sciences, this has practical implications:

• AI can accelerate protocol design and data analysis

• It can simulate scenarios and identify patterns

• It can support early-stage research and hypothesis generation

But it should not:

• Replace clinical expertise

• Operate without supervision

• Be treated as a source of truth without validation

The value of AI is in amplification. It increases speed and scale. It does not replace responsibility.

Get Expert Guidance on Responsible AI for Your Clinical and Digital Health Programs

At Estenda, we help MedTech, life sciences, and digital health organizations turn AI into practical, real-world solutions. Our work covers strategy, custom software development, AI and machine learning, and advanced data analytics. We stay involved through implementation to ensure everything works in real clinical and operational settings.

With over 20 years of experience in healthcare technology, we bring both technical expertise and a strong understanding of regulatory requirements. We focus on building solutions that are not only innovative, but also safe, reliable, and aligned with how your teams actually operate.

If you are exploring AI for clinical trials, patient engagement, or data systems, this is the right time to get clarity.

Book your free 30-minute consultation today and speak directly with our team about your goals and challenges. Contact us at info@estenda.com to get started.