A SEMANTIC DIALOGUE SYSTEM FOR RADIOLOGISTS
GERMAN HIGH TECH AWARD 2011!
Motivation
With traditional user interfaces, users may browse or explore visualised patient data, but little to no help is given when it comes to the interpretation of what is being displayed. Semantic annotations should provide the necessary image information, and a semantic dialogue system should be used to ask questions about the image annotations while engaging the clinician in a natural speech dialogue. Our motivation in developing RadSpeech is the design and implementation of a multimodal dialogue system for the radiologist. Dialogue-based semantic image retrieval should provide the basis for help in clinical decision support and computer aided diagnosis.
Main Objectives
With RadSpeech, we aim to build the next generation of intelligent, scalable, and user-friendly semantic search interfaces for the medical imaging domain, based on semantic technologies. Ontology-based knowledge representation is used not only for the image contents, but also for the complex natural language understanding and dialogue management process. With the incorporation of higher level knowledge represented in ontologies, different semantic views of the same medical images (such as structural, functional, and disease aspects) can be explicitly stated and integrated.
A radiologist’s daily task in the SIEMENS patient image finding stations as installed at the University Hospital Erlangen defines the application scenario. We strive for more efficiency during the medical finding process and for more structured finding reports including semantic image annotations.
Project Plan
With Project Duration: June 2010 – June 2012
(As of June 2012, RadSpeech’ s development will be carried on in THESEUS Data Intelligence)
Principal Investigator: Dr. Daniel Sonntag, DFKI
Industrial Partners: Siemens Healthcare, Siemens Technology
Clinical Partner: University Hospital Erlangen
Associated researchers and developers:
Christian Reuschling (DFKI), Christian Husodo Schulz (DFKI), Michael Sintek (DFKI), Daniel Porta (DFKI), Dr. Manuel Möller (DFKI), Maha Ahmed Baker (DFKI), Jochen Setz (DFKI), Milos Kalab (DFKI), Colette Weihrauch (DFKI), Luis Galárraga (DFKI), Martin Kramer (Siemens)
Industrial and clinical advisors:
Dr. Sascha Seifert (Siemens), Prof. Dr. Alexander Cavallaro (University Hospital Erlangen), Dr. Matthias Hammon (University Hospital Erlangen)
News
(NOVEMBER 2012)
Follow-up project ERmed, Kick-off!
(June 2012)
Interview with Daniel Sonntag in eHealth.com Journal (Issue June 2012, DE)
(June 2012)
(May 2012)
We exhibit We exhibit at Deutscher Röntgenkongress 2012 at the eHealth Conference 2012!
BMWi press release of the MEDICO event (DE)
(MARCH 2012)
We We exhibit at CeBIT 2012! DFKI Newletter, 1/2012 (EN DE)
- (1) iPad / iPhone Collaborative Speech Dialogue Interaction!
- (2) Interactive Paper and Smartphones for Mammography!
Please visit the Cebit THESEUS / EIT talks on Friday, March 9
(February 2012)
(January 2012)
(December 2011)
We organise a THESEUS medical imaging event (DE)
(NOVEMBER 2011)
Radspeech wins the German High Tech Champions (GHTC) Award! (EN)
(SEPTEMBER 2011)
Deutsches Ärzteblatt (German Medical Journal) 2011; 108(38) (DE)
“Radiologiearbeitsplatz am iPad” / “Radiology workstation for the iPad”
(JUNE 2011)
(March 2011)
(JANUARY 2011)
We extended the speech dialogue interaction scenario by a new input modality: interactive paper. In order to improve reporting practices, we allow the radiologist to write structured
reports with a special pen on normal paper. “Interactive Paper for Radiology Findings” was presented at IUI 2011 (the premier conference for reporting on the study of user interfaces with intelligent devices). The following picture shows (Left) Daniel Sonntag, DFKI; (Right) Daniel Bobrow, PARC at the IUI demo event. (Flyer)
(SEPTEMBER 2010)
After the last scenario update of innovative radiology applications (cf. the IUI 2010 demo), we plan to extend the desktop-based dialogue interaction to mobile speech interaction with medical image contents. The THESEUS CTC-WP4 dialogue shell should provide architecture to include mobile speech-based interfaces into the complex medical finding workflow in order to implement “RadSpeech on Mobile Devices.”
(SEPTEMBER 2010)
Some Introductory Publications
– RadSpeech, a mobile dialogue system for radiologists, IUI 2012
– Theseus Usecase MEDICO (in German, pp. 97), acatech 2011
– Mobile Facetted Browsing LODD Applications for Supporting Clinicians, OM 2011
– A Discourse and Dialogue Infrastructure for Industrial Dissemination, IWSDS 2010
– Design and Implementation of a Semantic Dialogue System for Radiologists, Nova Science 2010
