MIMIC Clinical Decision Making

This leaderboard shows current scores of models on the MIMIC Clinical Decision Making (MIMIC-CDM) and MIMIC Clinical Decision Making Full Information (MIMIC-CDM-FI) datasets. The dataset can be found here. The code used to run the models can be found here.


deepseek-ai/DeepSeek-R1-Distill-Llama-70B	70.48	89.9	65.7	66.1	60.2	70	8192	16


google/gemma-3-27b-it	70.48	89.9	65.7	66.1	60.2	70	8192	16
google/medgemma-27b-text-it	69.1	89.1	58	61.1	68.2	70	8192	16
google/gemma-2-27b	68.65	85.9	70.2	58.8	59.7	70	8192	8
google/gemma-3-12b-it	66.2	86.1	59.4	61.1	58.2	70	8192	16
meta-llama/Meta-Llama-3-70B-Instruct	63.32	84.6	59	54.5	55.2	70	8192	4
meta-llama/Llama-3.3-70B-Instruct	62.42	84.2	56.8	58	50.7	70	8192	4
meta-llama/Meta-Llama-3.1-70B-Instruct	60.45	84.7	51.7	55.6	49.8	70	8192	4
TheBloke/Llama2-70B-OASST-SFT-v10-GPTQ	54.9	82	48	45.5	44.1	70	4096	4
TheBloke/WizardLM-70B-V1.0-GPTQ	53.92	78.4	47.4	44.4	45.5	70	4096	4
google/medgemma-4b-it	53.05	67.7	61.6	37	45.9	70	8192	16
deepseek-ai/DeepSeek-R1-Distill-Llama-70B	51.6	78.8	41.2	46.3	40.1	70	8192	4
google/gemma-3-4b-it	50.8	64.3	55.7	36.2	47	70	8192	16
TheBloke/Llama-2-70B-Chat-GPTQ	45.45	74.6	33.8	34.2	39.2	70	4096	4
aaditya/OpenBioLLM-Llama3-70B	35.52	60.7	29.5	17.1	34.8	70	8192	4
google/gemma-3-1b-it	21.45	45.2	15.9	12.8	11.9	70	8192	16