{"paper":{"arxiv_id":"2404.14219","title":"Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone","abstract":"We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3.5 (e.g., phi-3-mini achieves 69% on MMLU and 8.38 on MT-bench), despite being small enough to be deployed on a phone. The innovation lies entirely in our dataset for training, a scaled-up version of the one used for phi-2, composed of heavily filtered web data and synthetic data. The model is also further aligned for robustness, safety, and chat format. We also provide some initial parameter-scaling results with a 7B and 14B model trained for 4.8T tokens, called phi-3-small and phi-3-medium, both significantly more capable than phi-3-mini.","primary_category":"cs.CL","venue":"arXiv 2024","published_at":null,"latest_version":1,"withdrawn":false},"latest_version":{"id":"f5c8b4cd-41da-40c7-9f83-0c7fb8f26437","version":1,"source_url":"https://arxiv.org/abs/2404.14219","rendered_html_url":null,"rendering_engine":null},"verdict":{"id":"6ae737f0-1e86-437c-8ecc-0e522fdbabf2","kind":"POST","status":"partial","score":0.6947791164658635,"confidence":0.55,"agent_version":"v0.1.0-phi3-winogrande-microslice","computed_at":"2026-05-15T16:10:04.346Z","is_current":true,"claim_citation":{"paper_arxiv_id":"2404.14219","section":"Table 2","row":"phi-3-mini","column":"MMLU 5-shot","reported_value":68.8,"reported_metric":"accuracy","quoted_text":"68.8","pdf_page":4,"notes":"Table 2 of arXiv:2404.14219 reports phi-3-mini MMLU 5-shot = 68.8 (the abstract reports 69% rounded). Driver measures WinoGrande zero-shot on `microsoft/Phi-3-mini-4k-instruct` instead — paper does not report comparable zero-shot WinoGrande. PROTOCOL_MATCH = `unknown` because the metric measured differs from the metric cited. Validator C1 gate prevents publication of WRONG regardless of measurement."},"protocol_match":"unknown"},"verdicts":{"post":{"id":"6ae737f0-1e86-437c-8ecc-0e522fdbabf2","kind":"POST","status":"partial","score":0.6947791164658635,"confidence":0.55,"agent_version":"v0.1.0-phi3-winogrande-microslice","computed_at":"2026-05-15T16:10:04.346Z","is_current":true,"claim_citation":{"paper_arxiv_id":"2404.14219","section":"Table 2","row":"phi-3-mini","column":"MMLU 5-shot","reported_value":68.8,"reported_metric":"accuracy","quoted_text":"68.8","pdf_page":4,"notes":"Table 2 of arXiv:2404.14219 reports phi-3-mini MMLU 5-shot = 68.8 (the abstract reports 69% rounded). Driver measures WinoGrande zero-shot on `microsoft/Phi-3-mini-4k-instruct` instead — paper does not report comparable zero-shot WinoGrande. PROTOCOL_MATCH = `unknown` because the metric measured differs from the metric cited. Validator C1 gate prevents publication of WRONG regardless of measurement."},"protocol_match":"unknown"},"pre":null}}