Artificial intelligence (AI) technology developed in Japan has successfully found features in pathology images from human cancer patients, without annotation, that could be understood by human doctors.1 Further, the AI identified features relevant to cancer prognosis that were not previously noted by pathologists, enabling a greater accuracy in predicting prostate cancer recurrence compared to pathologist-based diagnosis. Combining the predictions made by the AI with those by human pathologists led to an even greater accuracy.

“This technology could contribute to personalized medicine by making highly accurate prediction of cancer recurrence possible by acquiring new knowledge from images,” says Yoichiro Yamamoto, MD, PhD, of the Riken Center for Advanced Intelligence Project (AIP) in Tokyo. “It could also contribute to understanding how AI can be used safely in medicine by helping to resolve the issue of AI being seen as a ‘black box.’”

A research group led by Yamamoto and Go Kimura, MD, PhD, of Nippon Medical School in Tokyo, in collaboration with a number of university hospitals in Japan, adopted an approach called “unsupervised learning.” When humans teach the AI, it is not possible for the system to acquire knowledge beyond what is currently known. But rather than being “taught” medical knowledge, the Riken AI was asked to learn using unsupervised deep neural networks, known as autoencoders, without being given any medical knowledge. The researchers developed a method for translating the features found by the AI—only numbers initially—into high-resolution images that can be understood by humans.

To work toward this goal, the research group acquired 13,188 whole-mount pathology slide images of the prostate from Nippon Medical School Hospital (NMSH). The amount of data was enormous, equivalent to approximately 86 billion image patches (subimages divided for deep neural networks), and the computation was performed on AIP’s powerful RAIDEN supercomputer.

The AI learned using pathology images without diagnostic annotation from 11 million image patches. Features found by AI included cancer diagnostic criteria that have been used worldwide, on the Gleason score, but the system also identified features involving the stroma—connective tissues supporting an organ—in noncancer areas that experts had not been aware of. In order to evaluate these AI-found features, the research group verified the performance of recurrence prediction using the remaining cases from NMSH (internal validation).

The group found that the features discovered by the AI were more accurate at predicting recurrences than predictions made based on the human-established cancer criteria developed by pathologists, the Gleason score. Furthermore, combining both AI-found features and the human-established criteria predicted the recurrence more accurately than using either method. The group confirmed the results using another dataset including 2,276 whole-mount pathology images—using 10 billion image patches—from St. Marianna University Hospital and Aichi Medical University Hospital.

“I was very happy to discover that the AI was able to identify cancer on its own from unannotated pathology images,” says Yamamoto, “I was extremely surprised to see that AI found features that can be used to predict recurrence that pathologists had not identified.

“We have shown that AI can automatically acquire human-understandable knowledge from diagnostic annotation-free histopathology images,” he adds. “This ‘newborn’ knowledge could be useful for patients by allowing highly accurate predictions of cancer recurrence. What is very nice is that we found that combining the AI’s predictions with those of a pathologist increased the accuracy even further, showing that AI can be used hand-in-hand with doctors to improve medical care. In addition, the AI can be used as a tool to discover characteristics of diseases that have not been noted so far, and since it does not require human knowledge, it could be used in other fields outside medicine.”


  1. Yamamoto Y, Tsuzuki T, Akatsuka J, et al. Automated acquisition of explainable knowledge from unannotated histopathology images. Nat Commun.  2019;10(1):5642. doi:10.1038/s41467-019-13647-8.

Featured image: The AIP’s RAIDEN AI supercomputer. Photo courtesy, RIKEN