With traditional user interfaces, users may browse or explore visualised patient data, but little to no help is given when it comes to the interpretation of what is being displayed. Semantic annotations should provide the necessary image information, and a semantic dialogue system should be used to ask questions about the image annotations while engaging the clinician in a natural speech dialogue. Our motivation in developing RadSpeech is the design and implementation of a multimodal dialogue system for the radiologist. Dialogue-based semantic image retrieval should provide the basis for help in clinical decision support and computer aided diagnosis.

Images with courtesy of Siemens AG / University Hospital Erlangen.
Main Objectives

With RadSpeech, we aim to build the next generation of intelligent, scalable, and user-friendly semantic search interfaces for the medical imaging domain, based on semantic technologies. Ontology-based knowledge representation is used not only for the image contents, but also for the complex natural language understanding and dialogue management process. With the incorporation of higher level knowledge represented in ontologies, different semantic views of the same medical images (such as structural, functional, and disease aspects) can be explicitly stated and integrated.

A radiologist’s daily task in the SIEMENS patient image finding stations as installed at the University Hospital Erlangen defines the application scenario. We strive for more efficiency during the medical finding process and for more structured finding reports including semantic image annotations.

Project Plan

With Project Duration: June 2010 – June 2012
(As of June 2012, RadSpeech’ s development will be carried on in THESEUS Data Intelligence)

Principal Investigator: Dr. Daniel Sonntag, DFKI
Industrial Partners: Siemens Healthcare, Siemens Technology
Clinical Partner: University Hospital Erlangen

Associated researchers and developers:
Christian Reuschling (DFKI), Christian Husodo Schulz (DFKI), Michael Sintek (DFKI), Daniel Porta (DFKI), Dr. Manuel Möller (DFKI), Maha Ahmed Baker (DFKI), Jochen Setz (DFKI), Milos Kalab (DFKI), Colette Weihrauch (DFKI), Luis Galárraga (DFKI), Martin Kramer (Siemens)

Industrial and clinical advisors:
Dr. Sascha Seifert (Siemens),  Prof. Dr. Alexander Cavallaro (University Hospital Erlangen), Dr. Matthias Hammon (University Hospital Erlangen)

(June 2012)

Interview with Daniel Sonntag in Journal (Issue June 2012, DE)

(June 2012)

We exhibit at the eHealth Conference 2012!

  • Collaborative iPad scenario
  • Press articles: THESEUS News (DE), (DE)  
(May  2012)

We exhibit We exhibit at Deutscher Röntgenkongress 2012 at the eHealth Conference 2012!

BMWi press release of the MEDICO event (DE)

(MARCH 2012)

We We exhibit at CeBIT 2012!  DFKI Newletter, 1/2012 (EN DE)

  • (1) iPad / iPhone Collaborative Speech Dialogue Interaction!
  • (2) Interactive Paper and Smartphones for Mammography!

Please visit the Cebit THESEUS / EIT talks  on Friday, March 9

(February 2012)

We have been selected to (re)present DFKI’s  IT solutions at the THESEUS final congress to Dr. Philipp Rösler, German Minister for Economic Affairs (DEFR)

(January 2012)

We exhibit at the eHealth Conference 2012!

  • Radspeech Article in HealthTechWire (DE)
  • Radspeech added to Nuance’s medical speech documentation news (DE)
(December 2011)

We organise a THESEUS medical imaging event (DE)


Radspeech wins the German High Tech Champions (GHTC) Award! (EN)


Deutsches Ärzteblatt (German Medical Journal) 2011; 108(38) (DE)
“Radiologiearbeitsplatz am iPad” / “Radiology workstation for the iPad”

(JUNE 2011)

THESEUS Newsletter: Radiology workstations of the future / “Radiologenarbeitsplatz der Zukunft” (DE)

We exhibit at Deutscher Röntgenkongress 2011!  

(THESEUS Newsletter DE)

  • (1) iPad Multitouch Interaction and Semantic Search!
  • (2) iPad Speech Dialogue Interaction!
(May 2011)

We exhibit at “Night of Science” in Berlin 2011! 

(THESEUS Newsletter DE DE)

(March 2011)

We exhibit at “We exhibit at CeBIT 2011!  BMWi Announcement (DE)

(DFKI Newsletter DE EN

  • (1) iPad Multitouch Interaction and Semantic Search!
  • (2) iPad Speech Dialogue Interaction!
  • (3) Interactive Paper!
(JANUARY 2011)

We extended the speech dialogue interaction scenario by a new input modality: interactive paper. In order to improve reporting practices, we allow the radiologist to write structured
reports with a special pen on normal paper. “Interactive Paper for Radiology Findings” was presented at IUI 2011 (the premier conference for reporting on the study of user interfaces with intelligent devices). The following picture shows (Left) Daniel Sonntag, DFKI; (Right) Daniel Bobrow, PARC at the IUI demo event. (Flyer)


After the last scenario update of innovative radiology applications (cf. the IUI 2010 demo), we plan to extend the desktop-based dialogue interaction to mobile speech interaction with medical image contents. The THESEUS CTC-WP4 dialogue shell should provide architecture to include mobile speech-based interfaces into the complex medical finding workflow in order to implement “RadSpeech on Mobile Devices.”

Dr. Daniel Sonntag at the THESEUS Innovation Center Berlin with the Radspeech and THESEUS-Medico prototypes.