Home › Datasets › Mathematical Reasoning

Reasoning & Math Pack

How to Fine-Tune an LLM for Mathematical Reasoning

Step-by-step mathematical problem solving with verified, correct reasoning — arithmetic through applied word problems.

Fine-tune a model on this dataset Preview Dataset

$29 for the dataset. Training is priced separately by model, starting at $47.

What can this kind of fine-tuning teach an LLM?

A general-purpose model may already know some of this subject. Fine-tuning serves a different purpose: it repeatedly demonstrates the kinds of requests, response patterns, terminology, decisions and workflows you want the model to handle more consistently.

For Mathematical Reasoning, the dataset focuses on step-by-step mathematical problem solving with verified, correct reasoning — arithmetic through applied word problems. Fine-tuning shapes recurring model behavior; it does not turn the model into a live database or guarantee correctness on every new case.

What training data should cover

Useful supervised fine-tuning data should resemble the work you expect the model to perform. This Rebiha dataset's domain definition is organized around 17 coverage areas:

How Rebiha prepared the training material

This page contains 824 instruction-response pairs from the existing Rebiha dataset. Its domain definition was designed around the coverage areas above so the training material spans multiple realistic tasks rather than one narrow prompt pattern.

Rebiha's broader dataset workflow uses structured quality review to help identify problems such as contradictions, unsupported claims, unsupported assumptions, calculation errors and other weak training examples. You can inspect real examples on this page before deciding whether the dataset fits your model and use case.

What is in this dataset

824 instruction-response pairs covering step-by-step mathematical problem solving with verified, correct reasoning — arithmetic through applied word problems. Each pair is a realistic instruction with a full written response, in the JSONL format used for supervised fine-tuning. No synthetic filler, no duplicated rows.

Example pairs

Real rows from the dataset, shown in full.

A train leaves Station A at 9:00 AM traveling at 60 mph toward Station B, which is 240 miles away. At the same time, a second train leaves Station B traveling at 80 mph toward Station A. At what time will the two trains pass each other, and how far from Station A will they be when they meet?

Step 1: Understand the problem. The two trains are moving toward each other, so their relative speed is the sum of their individual speeds: 60 + 80 = 140 mph. Step 2: The total distance between them initially is 240 miles. Since they are closing the gap at 140 mph, the time until they meet is distance / relative speed = 240 / 140 = 12/7 hours = 1 hour and 5/7 of an hour. 5/7 of an hour = (5/7)*60 = 300/7 ≈ 42.857 minutes, so the meeting time is about 1 hour 42.857 minutes after 9:00 AM. That is 10:42:51 AM (since 0.857 minutes = 51.43 seconds). Step 3: Distance from Station A when they meet = speed of train A * time = 60 * (12/7) = 720/7 = 102.857 miles. Step 4: Verify: Train B travels 80 * (12/7) = 960/7 ≈ 137.143 miles. Sum = 720/7 + 960/7 = 1680/7 = 240 miles, which matches. Final answer: They meet at approximately 10:42:51 AM, about 102.857 miles (exactly 720/7 miles) from Station A.

Solve the system of equations: 2x + 3y = 7 and 5x - 2y = 8. Show all steps and verify your solution.

Step 1: We have: (1) 2x + 3y = 7, (2) 5x - 2y = 8. I'll use elimination. Multiply (1) by 2: 4x + 6y = 14. Multiply (2) by 3: 15x - 6y = 24. Step 2: Add the two new equations: (4x + 6y) + (15x - 6y) = 14 + 24 → 19x = 38 → x = 2. Step 3: Substitute x = 2 into (1): 2(2) + 3y = 7 → 4 + 3y = 7 → 3y = 3 → y = 1. Step 4: Verify in (2): 5(2) - 2(1) = 10 - 2 = 8, which matches. Final solution: x = 2, y = 1.

A store increases the price of a jacket by 20%, then during a sale reduces the new price by 25%. If the final sale price is $72, what was the original price before the increase?

Step 1: Let the original price be P. After a 20% increase, the price becomes P * 1.20 = 1.2P. Step 2: Then a 25% reduction means the price is multiplied by (1 - 0.25) = 0.75, so the final price is (1.2P) * 0.75 = 0.9P. Step 3: We know the final price is $72, so 0.9P = 72. Step 4: Divide both sides by 0.9: P = 72 / 0.9 = 80. Step 5: Verify: Original $80, increase 20% gives $96, then 25% off $96 gives $96 * 0.75 = $72. Correct. Final answer: $80.

Who this is for

Builders who want their fine-tuned model to reliably perform this specific capability, as part of a broader assistant or agent.

Which models this works with

The dataset is plain JSONL, so it works with any instruction-tuned open model. On Rebiha you can train it directly on:

Training uses LoRA, or QLoRA on a 4-bit base for larger models. Your base model's weights are never modified — training produces a separate adapter.

How it works

  1. Pick this dataset and a base model.
  2. Rebiha runs the job on a managed GPU. You set up nothing.
  3. Download your model: a ready-to-run GGUF, plus developer assets (adapter, tokenizer, configs).

Fine-tuning or RAG?

Use RAG when the model needs facts that change — your prices, your documents, today's inventory. Use fine-tuning when you want consistent behavior and structure without prompting for it every time. This dataset teaches the shape of this capability, not facts to look up.

How managed fine-tuning works

  1. Inspect the dataset examples and choose a compatible base model.
  2. Rebiha runs the training job on managed GPU infrastructure using LoRA or QLoRA as appropriate.
  3. Download portable model assets for your own workflow, including the available ready-to-run and developer outputs for the selected training path.

Fine-tuning or RAG?

Use fine-tuning when the objective is recurring behavior: response structure, domain patterns, terminology, workflow, or how the model approaches a class of requests. Use RAG when answers depend on changing or proprietary facts that should be retrieved at request time. The two approaches can also work together.

Know what goes in. Own what comes out.

You can inspect real examples before training, choose the model you want to adapt, and keep portable model assets when training is complete.

Related datasets

Start fine-tuning