Session 4 • 20 minutes + Q&A

Vision Models for Medical Imaging

AI that can see: analyze images, interpret diagrams, and assist with visual documentation

Image Analysis
  • • Describe medical images in natural language
  • • Identify anatomical structures
  • • Compare before/after images
  • • Detect patterns and abnormalities
Diagram Interpretation
  • • Explain complex surgical diagrams
  • • Analyze flowcharts and decision trees
  • • Interpret data visualizations
  • • Extract info from figures in papers
Visual Q&A
  • • "What valve is shown in this echo?"
  • • "Describe the surgical approach"
  • • "What are the key findings in this CXR?"
  • • "Extract data from this graph"
Clinical Applications
Real-world uses for cardiothoracic surgeons

Imaging Documentation

Use Case: Generating structured reports from imaging studies

"Analyze this chest X-ray and provide a structured report including: cardiac silhouette size, lung fields, mediastinal contours, and any abnormalities. Use standard radiological terminology."
Faster preliminary readsConsistent reportingEducational tool

Surgical Planning

Use Case: Analyzing pre-operative imaging for surgical approach

"Based on this CT angiography, describe the coronary anatomy including: number of vessels, degree of stenosis, calcification burden, and suitability for bypass grafting vs. PCI."
Comprehensive pre-op assessmentTeam communicationPatient education

Research Figure Analysis

Use Case: Extracting data from published figures

"Extract the data points from this Kaplan-Meier survival curve and summarize the key findings including median survival times and p-values."
Rapid literature reviewData extraction for meta-analysisFigure interpretation
Vision AI Tools

GPT-4 Vision

Excellent general image understanding, detailed descriptions

ChatGPT Plus

Claude 3 with Vision

Careful, nuanced analysis, good at complex instructions

Claude Pro

Gemini 2.0 Flash

Fast multimodal processing, good for batch analysis

Google AI Studio
⚠️ Critical Limitations & Safety
!

Not FDA-approved for diagnosis: These are general-purpose AI tools, not medical devices

!

Variable accuracy: Performance depends on image quality and specific task

!

Hallucination risk: AI may confidently describe features that aren't present

!

Privacy critical: Remove all patient identifiers before uploading to public AI tools

✅ Appropriate Use Cases
  • ✓ Educational tool for trainees
  • ✓ Research figure analysis
  • ✓ Generating draft reports for review
  • ✓ Second opinion for complex cases
  • ✓ Patient education material creation
❌ Inappropriate Use Cases
  • ✗ Primary diagnostic tool
  • ✗ Replacing radiologist interpretation
  • ✗ Making treatment decisions without verification
  • ✗ Processing identifiable patient images on public tools
  • ✗ Automated clinical decision-making
Try It Yourself: Vision AI Playground
Upload de-identified medical images and see how vision models analyze visual content

Suggested Exercise:

  1. Find a de-identified chest X-ray or CT scan (or use a published image from a paper)
  2. Upload to the AI Playground and ask: "Describe the key findings in this image"
  3. Try a more specific prompt: "Analyze this image and identify: [specific structures/findings]"
  4. Compare responses from GPT-4 Vision, Claude, and Gemini
  5. Verify AI descriptions against your own clinical assessment

Privacy Reminder: Only upload de-identified images or images from published literature. Never upload identifiable patient data to public AI tools.

Key Takeaways
✓

Vision AI extends capabilities beyond text to images, diagrams, and visual data

✓

Multiple applications in clinical practice, research, and education

✓

Verification is essential—never rely solely on AI for clinical decisions

✓

Privacy matters—remove identifiers and follow institutional policies

Ready to Go Deeper?

This 90-minute workshop is just the beginning. Our comprehensive full-day program covers advanced applications, hands-on projects, and personalized consultation for your research and clinical workflows.