GSM8K

Every result

• Mistral-7B-v0.152.2%Reported by the model's maker (Mistral AI / Jiang et al.), Oct 10, 2023

  Mistral reported 52.2% on GSM8K for Mistral 7B v0.1 base in its October 2023 paper.

  Tasks: Exact dataset split/revision unspecified in the inspected paper

  Limits: Mistral reported these results in its own common evaluation pipeline; we did not reproduce the runs. The table identifies the pretrained base variant, without a checkpoint hash. No Chat/Instruct or served API model is implied. Missing dataset/run metadata remain unknown. Compare only this paper’s paired rows for the same benchmark; no cross-paper or overall model ranking is established. maj@8 is an aggregation condition, not pass@1 or an equal inference-cost claim.

  Jiang et al. / Mistral AI; hosted by arXiv

• Llama 2 7B pretrained (July 2023 release)16.0%Reported by the model's maker (Mistral AI / Jiang et al.), Oct 10, 2023

  Mistral reported 16.0% on GSM8K for Llama 2 7B pretrained in its October 2023 paper.

  Tasks: Exact dataset split/revision unspecified in the inspected paper

  Limits: Mistral reported these results in its own common evaluation pipeline; we did not reproduce the runs. The table identifies the pretrained base variant, without a checkpoint hash. No Chat/Instruct or served API model is implied. Missing dataset/run metadata remain unknown. Compare only this paper’s paired rows for the same benchmark; no cross-paper or overall model ranking is established. Mistral is a competing model provider evaluating Meta’s model; this is a provider-run comparison, not an independent external laboratory test. maj@8 is an aggregation condition, not pass@1 or an equal inference-cost claim.

  Jiang et al. / Mistral AI; hosted by arXiv