The MultiCaRe Dataset

Developing reliable AI systems for healthcare depends on access to large, diverse, and high-quality datasets. However, obtaining clinical data remains one of the biggest challenges in medical AI research due to privacy regulations, limited public availability, and the complexity of collecting multimodal clinical information.

To help address this challenge, we developed MultiCaRe, an open-source multimodal dataset built from PubMed Central open-access case reports. The dataset contains more than 98,000 de-identified clinical cases, 139,000 medical images, and a hierarchical taxonomy of over 140 image classes.

By preserving the relationship between clinical narratives and their corresponding images, MultiCaRe provides a flexible resource for machine learning, medical natural language processing, computer vision, multimodal AI, and medical education.

Why We Built MultiCaRe

Publicly available medical datasets have played a fundamental role in advancing healthcare AI. However, most existing resources are designed around a single task, imaging modality, disease, or medical specialty.

This specialization is useful for targeted benchmarking, but it can make it difficult to develop systems that need broader clinical context or must reason across multiple types of information.

MultiCaRe was designed to address this limitation by bringing together clinical narratives, medical images, image captions, structured metadata, and hierarchical image labels within a single open-source resource.

Rather than focusing on a single benchmark, MultiCaRe supports the creation of many different datasets and research workflows. Researchers can work with the complete collection or generate specialized subsets tailored to a particular population, modality, anatomical region, specialty, or research question.

What’s Included

MultiCaRe combines several types of clinical information into a single resource, providing both the scale required for machine learning and the contextual information needed for multimodal research.

98,000+ Clinical Cases

De-identified clinical cases extracted from more than 72,000 PubMed Central open-access case reports.

139,000+ Medical Images

Medical images spanning numerous imaging modalities, anatomical regions, and clinical specialties.

140+ Image Classes

A hierarchical taxonomy organized through logical relationships such as mutual exclusivity and subsumption.

Multimodal Data

Clinical narratives, medical images, image captions, structured metadata, and image labels.

Broad Clinical Coverage

Cases from oncology, cardiology, pathology, surgery, radiology, dermatology, ophthalmology, and many other fields.

Open-Source Under CC0

Unrestricted use for research, education, dataset development, and commercial applications.

This combination of scale, diversity, and multimodal information allows researchers to work with the complete dataset or create highly specialized subsets designed around their own research questions.

Create Custom Datasets in Minutes

One of MultiCaRe’s main advantages is that researchers do not need to work with the entire dataset. Using the accompanying multiversity Python library, cases can be filtered and exported according to the requirements of a specific project.

from multiversity.multicare_dataset import MedicalDatasetCreator

mdc = MedicalDatasetCreator(
    directory='medical_datasets'
)

filters = [
    {
        'field': 'gender',
        'string_list': ['Male']
    },
    {
        'field': 'label',
        'string_list': ['mri', 'head']
    }
]

mdc.create_dataset(
    dataset_name='brain_mri_dataset',
    filter_list=filters,
    dataset_type='multimodal'
)

Applications

Thanks to its combination of clinical narratives, medical images, structured metadata, and hierarchical labels, MultiCaRe can support projects across research, education, and healthcare innovation.

Medical Imaging & Computer Vision

Develop image-classification models, create benchmark datasets, evaluate computer-vision algorithms, or assemble specialized collections for particular modalities, anatomical regions, or diseases.

Medical NLP

Extract clinical information, classify medical text, build information-retrieval systems, and develop natural-language-processing datasets using real-world clinical narratives.

Multimodal AI

Train and evaluate vision-language models, develop retrieval-augmented generation systems, explore multimodal clinical reasoning, or build assistants that combine textual and visual information.

Medical Education

Create curated collections of clinical cases for teaching, case-based learning, self-assessment, educational demonstrations, and training across multiple specialties.

Healthcare AI Research

Rapidly assemble reproducible datasets for benchmarking, feasibility studies, model evaluation, proof-of-concept projects, and exploratory healthcare AI research.

Getting Started

MultiCaRe is freely available under the CC0 license. Researchers, developers, educators, and organizations can use, modify, redistribute, and build upon the dataset for research, educational, and commercial purposes.

A Flexible Foundation for Clinical AI

Whether the objective is to train a medical imaging model, explore multimodal AI, evaluate an information-retrieval pipeline, create educational resources, or conduct healthcare research, MultiCaRe provides a practical foundation for assembling the data required.

Its value comes not only from its scale, but from the relationship between cases, narratives, images, captions, metadata, and hierarchical labels. This structure makes it possible to move beyond isolated image or text datasets and investigate systems that work with more complete clinical information.

At Argenticare, we develop and share open-source datasets, research resources, and educational tools that help make healthcare AI research more accessible, reproducible, and useful.