Research and Advances
Architecture and Hardware Medical image modeling tools and applications


Medical image analysis and modeling are essential to many fields of medicine, including radiology, surgery, and medical education. Traditionally, doctors have based their practice on 2D images, such as those produced from scanners and microscopes. This limits them from accurately seeing and understanding the 3D organ structure and function that is often crucial for the timely diagnosis and appropriate treatment of disease. The significant increase in computing power in recent years has allowed the development of powerful new 3D image analysis and visualization algorithms that promise to change the way medicine is practiced. The ability to visualize a body's internal structures and functions in 3D will allow doctors to improve their current practice and may lead to the discovery of new knowledge.
  1. Article
  2. Author
  3. Figures


Computer-based analysis tools are also becoming increasingly important due to the emergence of improved scanners, such as 3T magnetic resonance imaging (MRI) and 64-slice computed tomography (CT), as well as new imaging methods (for example, optical imaging). This new generation of imaging devices provides dramatically increased data complexity and resolution. Therefore, novel computer algorithms and software tools must be developed to aid the robust 3D analysis of the body’s structure and function. In the coming years, the computer-based tools utilized in the practice of medicine must evolve from being fundamentally two-dimensional to become fully three-dimensional.

These increased computational demands create new and exciting possibilities for practitioners in this interdisciplinary area to collaborate with doctors to develop robust and reliable methods and software for computer-based analysis of the body’s structures and functions. The challenges stem from the complexity of the data in terms of size (often several gigabytes), geometrical complexity, motion, organ structure, and imaging effects such as nonlinearities and noise. For example, new tools for image analysis must be developed that can handle functions such as organ segmentation, reconstruction of 3D organ geometry and motion, disease diagnosis, surgical planning, and education. These new tools will require novel research and collaborative efforts among computer scientists, electrical engineers, mathematicians, statisticians, biologists, and physicians.

The synergistic and interdisciplinary nature of research in this area and the need for software tool development for medical applications is reflected in several major initiatives from the National Institutes of Health (NIH) and the National Science Foundation (NSF). As part of the NIH Roadmap for Medicine launched in 2002 (see a series of new technology development initiatives has been announced, including the creation of new centers for biomedical computing whose mission will be to develop new infrastructure for biomedical computing. This is the first time the NIH has put such emphasis on computational tool development for medical applications and clearly indicates the importance of this area for the future advancement of medicine. Tool development is also supported at the NSF in the newly formed Science and Engineering Information Integration and Informatics (SEIII) program. This interdisciplinary program funds a broad array of computer science research within science and engineering contexts. This includes research in medical and healthcare informatics as well as supporting the life sciences through bioinformatics research.

These increased computational demands create new and exciting possibilities for practitioners in this interdisciplinary area to collaborate with doctors to develop robust and reliable methods and software for computer-based analysis of the body’s structures and functions.

The articles in this special section investigate recent efforts in medical image analysis and modeling from medical data that significantly improve the quality of medical practice and education. The article by Delingette and Ayache describes the development of a surgical simulator for minimally invasive surgery to improve doctor training prior to operating on actual patients. In these operations, the surgeon must acquire specific skills and advanced hand-eye coordination. Typically, surgeons train on dedicated mechanical systems, known as endotrainers, or on animals. Digital surgery simulation offers a number of advantages compared to these methods, the most important of which are greater realism and enhanced flexibility, allowing the simulation of various scenarios ranging from standard pathologies to extremely rare cases. The development of medical imagery and 3D reconstruction techniques offers the potential to create a different model for each patient, thereby customizing the surgical procedure to each individual.

The article by Kaufmann et al. presents a computer-graphics alternative to optical colonoscopy, which is currently being used clinically, and is expensive, uncomfortable, and ultimately provides an incomplete examination. Research in this area is motivated by the fact that colorectal cancer currently ranks as the third-most common human malignancy and the second leading cause of cancer-related deaths in the U.S. Using this new method, a computer-based extraction of the colon from CT scans is used to reconstruct a 3D model of the colon. The visualization software allows the physician to interactively navigate through the colon using volume rendering. An intuitive user interface with customized tools supports virtual biopsy to inspect any suspicious regions the physician may encounter.

Park et al. discuss 3D modeling and analysis of heart motion from MRI-tagged data. Heart disease is the leading cause of death in the Western world and consequently the study of normal and pathological heart behavior has become a topic of rigorous research. In particular, the study of the shape and motion of the heart is important because many heart diseases are believed to be strongly correlated to the resulting changes in the shape and motion of the heart. The article demonstrates that this type of modeling is now possible; its intention is to be used both clinically for the improved timely diagnosis of heart disease and for the first detailed quantitation of the normal and abnormal heartbeat.

The article by Imielinska and Molholt describes efforts to develop computer-based methods for teaching anatomy and for clinical training at Columbia University. This project is currently based on data from the Visible Human Project, an initiative sponsored by the NIH and the National Library of Medicine (NLM) that includes whole anatomical male and female data (see These efforts will allow the future development of 3D software that will complement the current practices used to teach anatomy in medical schools.

The final article, by Yoo and Ackerman, explores the recent efforts by the NLM developing publicly available segmentation and registration software: a public resource in high-dimension data processing tools. The effort has led to the creation of Insight, a project for open source software development in image-processing tools along with the Insight Software Consortium, which includes 11 participating universities and commercial institutions (see This publicly available software promises to benefit both the research and the clinical medical communities, as well as companies specializing in related software tools.

The articles here exemplify the revolution currently occurring in medical image analysis and modeling for clinical and educational applications. While these articles focus on software tools related to anatomy, many more tools for coupling organ anatomy and physiology, molecular modeling, bioinformatics, genomics, instrumentation, and other exciting application areas are currently being developed. New computer-based methods and initiatives will provide better insight into the structure and function of the human body, generating future findings that will lead to improvements in clinical practice. New advances that will enhance computer-aided diagnosis and computer-aided surgery are promised by the dynamic integration of measurements and modeling (see These directions are the new multidisciplinary grand challenges in applications, algorithms, measurements, and computer systems software, and are a significant departure from past research, which typically dealt with relatively simple and low-dimensional data.

This is an exciting multidisciplinary area that will play a major role in the development of new and powerful algorithms and improved measurement methods to deal with the complex, dynamic, and multidimensional nature of medical data.

Back to Top

Back to Top


UF1 Figure. Model-based automated segmentation of the heart ventricles from end diastole (beginning of contraction) to end systole (end of contraction) in tagged MRI sequences. Top row: Coronal views: Each image shows the blood-filled left ventricle in red, and right ventricle in pink. Bottom row: Axial views corresponding to the top row.   Albert Montillo, 2004, Automated volumetric model construction and dynamic segmentation of the heart ventricles in tagged MRI. Ph.D. Thesis, University of Pennsylvania, PA,

Back to top

Join the Discussion (0)

Become a Member or Sign In to Post a Comment

The Latest from CACM

Shape the Future of Computing

ACM encourages its members to take a direct hand in shaping the future of the association. There are more ways than ever to get involved.

Get Involved

Communications of the ACM (CACM) is now a fully Open Access publication.

By opening CACM to the world, we hope to increase engagement among the broader computer science community and encourage non-members to discover the rich resources ACM has to offer.

Learn More