Home › Datasets › Code Review Judgment

Coding & Engineering Pack

How to Fine-Tune an LLM for Code Review & Engineering Judgment

Giving structured, prioritized code review judgment — what must change, what's a nice-to-have, and why — distinct from simply explaining what code does.

Fine-tune a model on this dataset Preview Dataset

$39 for the dataset. Training is priced separately by model, starting at $47.

What can this kind of fine-tuning teach an LLM?

A general-purpose model may already know some of this subject. Fine-tuning serves a different purpose: it repeatedly demonstrates the kinds of requests, response patterns, terminology, decisions and workflows you want the model to handle more consistently.

For Code Review Judgment, the dataset focuses on giving structured, prioritized code review judgment — what must change, what's a nice-to-have, and why — distinct from simply explaining what code does. Fine-tuning shapes recurring model behavior; it does not turn the model into a live database or guarantee correctness on every new case.

What training data should cover

Useful supervised fine-tuning data should resemble the work you expect the model to perform. This Rebiha dataset's domain definition is organized around 12 coverage areas:

How Rebiha prepared the training material

This page contains 653 instruction-response pairs from the existing Rebiha dataset. Its domain definition was designed around the coverage areas above so the training material spans multiple realistic tasks rather than one narrow prompt pattern.

Rebiha's broader dataset workflow uses structured quality review to help identify problems such as contradictions, unsupported claims, unsupported assumptions, calculation errors and other weak training examples. You can inspect real examples on this page before deciding whether the dataset fits your model and use case.

What is in this dataset

653 instruction-response pairs covering giving structured, prioritized code review judgment — what must change, what's a nice-to-have, and why — distinct from simply explaining what code does. Each pair is a realistic instruction with a full written response, in the JSONL format used for supervised fine-tuning. No synthetic filler, no duplicated rows.

Example pairs

Real rows from the dataset, shown in full.

Review this PR that adds a new endpoint to an existing Go service. The author uses a global in-memory map for rate limiting instead of the existing Redis-based limiter to avoid adding a dependency. The code is correct and tests pass. What's your judgment?

Blocking: You must not ship this. The global map approach breaks horizontal scaling — once you run more than one instance, each has its own counter, so rate limits are effectively per-instance, not per-user. That's a correctness issue, not a style issue. Also, the map is accessed from multiple goroutines without a mutex; even if the current load is low, it's a data race that will surface under concurrency. Use the existing Redis limiter — that's what it's for. Nice-to-have: add a test that simulates two instances sharing the same Redis to prove the limit is global. Nitpick: the map's unbounded growth would leak memory if you kept it, but that's moot once you switch.

I'm reviewing a Python PR where the author replaced a synchronous file read with a thread pool to speed up processing. The logic is correct, but I'm worried about the trade-off. How should I frame my feedback?

The trade-off here is real: thread pools add concurrency but not parallelism for CPU-bound work due to the GIL. If the file reads are I/O-bound, threads help; if they're CPU-bound, you're just adding overhead. Blocking: check whether the file is being read from a local disk or network — if it's network, threads are fine, but if it's a local SSD, the bottleneck is likely parsing, not I/O. Also, you're creating a new ThreadPoolExecutor per call — that's a thread-creation overhead spike. Move it to a module-level pool. Nice-to-have: consider asyncio with aiofiles instead, since it's a lighter concurrency model and matches the rest of the codebase's async style. But don't block on that; the thread pool is acceptable if I/O-bound. The real question to ask the author: what's the measured speedup? If it's less than 2x, the added complexity isn't worth it.

Here's a JS function submitted for review. It computes a discount by chaining a bunch of ternary operators and uses a switch on a string to determine the customer tier. The logic is correct per the spec. What's your review?

Blocking: The logic being correct isn't enough — this is a maintainability landmine. The nested ternaries are unreadable; a future dev will need to trace parentheses to understand the precedence. Extract each tier's discount into a named function or a lookup table keyed by tier. That's not a style nit — it's a correctness risk because when the business rule changes (and it will), someone will edit the wrong branch. Also, the switch falls through to a default that throws, but you're throwing inside a string comparison that's case-sensitive — 'premium' vs 'Premium' will throw. That's a bug, not a nitpick: normalize the input before the switch. Nice-to-have: add a comment explaining the discount formula's origin so future devs don't 'fix' it incorrectly. Nitpick: use const instead of let for `finalPrice` since it's never reassigned.

Who this is for

Builders who want their fine-tuned model to reliably perform this specific capability, as part of a broader assistant or agent.

Which models this works with

The dataset is plain JSONL, so it works with any instruction-tuned open model. On Rebiha you can train it directly on:

Training uses LoRA, or QLoRA on a 4-bit base for larger models. Your base model's weights are never modified — training produces a separate adapter.

How it works

  1. Pick this dataset and a base model.
  2. Rebiha runs the job on a managed GPU. You set up nothing.
  3. Download your model: a ready-to-run GGUF, plus developer assets (adapter, tokenizer, configs).

Fine-tuning or RAG?

Use RAG when the model needs facts that change — your prices, your documents, today's inventory. Use fine-tuning when you want consistent behavior and structure without prompting for it every time. This dataset teaches the shape of this capability, not facts to look up.

How managed fine-tuning works

  1. Inspect the dataset examples and choose a compatible base model.
  2. Rebiha runs the training job on managed GPU infrastructure using LoRA or QLoRA as appropriate.
  3. Download portable model assets for your own workflow, including the available ready-to-run and developer outputs for the selected training path.

Fine-tuning or RAG?

Use fine-tuning when the objective is recurring behavior: response structure, domain patterns, terminology, workflow, or how the model approaches a class of requests. Use RAG when answers depend on changing or proprietary facts that should be retrieved at request time. The two approaches can also work together.

Know what goes in. Own what comes out.

You can inspect real examples before training, choose the model you want to adapt, and keep portable model assets when training is complete.

Related datasets

Start fine-tuning