Home › Datasets › Voice Cloning & AI Voiceover Direction

AI Music & Audio Pack

How to Fine-Tune an LLM for AI Voiceover & Voice Direction

Directing AI voice generation and voice cloning — emotional delivery control, pacing, multi-character differentiation, and consent/licensing awareness for tools like ElevenLabs.

Fine-tune a model on this dataset Preview Dataset

$39 for the dataset. Training is priced separately by model, starting at $47.

What can this kind of fine-tuning teach an LLM?

A general-purpose model may already know some of this subject. Fine-tuning serves a different purpose: it repeatedly demonstrates the kinds of requests, response patterns, terminology, decisions and workflows you want the model to handle more consistently.

For Voice Cloning & AI Voiceover Direction, the dataset focuses on directing ai voice generation and voice cloning — emotional delivery control, pacing, multi-character differentiation, and consent/licensing awareness for tools like elevenlabs. Fine-tuning shapes recurring model behavior; it does not turn the model into a live database or guarantee correctness on every new case.

What training data should cover

Useful supervised fine-tuning data should resemble the work you expect the model to perform. This Rebiha dataset's domain definition is organized around 15 coverage areas:

How Rebiha prepared the training material

This page contains 784 instruction-response pairs from the existing Rebiha dataset. Its domain definition was designed around the coverage areas above so the training material spans multiple realistic tasks rather than one narrow prompt pattern.

Rebiha's broader dataset workflow uses structured quality review to help identify problems such as contradictions, unsupported claims, unsupported assumptions, calculation errors and other weak training examples. You can inspect real examples on this page before deciding whether the dataset fits your model and use case.

What is in this dataset

784 instruction-response pairs covering directing ai voice generation and voice cloning — emotional delivery control, pacing, multi-character differentiation, and consent/licensing awareness for tools like elevenlabs. Each pair is a realistic instruction with a full written response, in the JSONL format used for supervised fine-tuning. No synthetic filler, no duplicated rows.

Example pairs

Real rows from the dataset, shown in full.

I keep writing 'sad and emotional' in my ElevenLabs prompt but the voice still sounds the same. What actually changes the delivery?

'Sad' is a mood label the model can't resolve into mechanics, so it defaults to its neutral read. Direct the physical behaviors instead. Three levers do most of the work: 1. Pacing — insert ellipses or commas where you want drag: 'I just... I don't know anymore.' Long comma runs slow the engine down more reliably than the word 'slow'. 2. Emphasis — capitalize the stressed word only: 'I DIDN'T mean it.' Over-capitalizing flattens everything back to monotone, so one stressed word per phrase. 3. Breath placement — a period or double space before a phrase forces a micro-pause the model renders as an intake: 'It's fine. It has to be.' So instead of 'sad,' try: 'Read with a heavy, low register. Slow the second half of each sentence. Pause before the final two words of every line. Avoid upward inflection at line ends.' Then pick the take where the pitch drops on the last clause — that's where the emotion actually lands.

How do I get a voice to sound genuinely moved / tearful without it tipping into melodrama?

Melodrama comes from sustained intensity; real emotion comes from restraint plus one crack. Direct the containment, not the flood. Practical moves: - Keep most of the line controlled and even, then let a single word break: 'I'm okay. I'm — okay.' The dash reads as a catch. - Drop volume slightly rather than raising it. Quiet-plus-slow reads as held-back crying; loud-plus-fast reads as a tantrum. - Front-load the breath: put the pause at the START of the emotional phrase, not the end, so it sounds like the speaker is steeling themselves. - Add 'do not raise pitch on the final word' — grief falls in pitch, it doesn't rise. Generate 3–4 takes with the stability setting higher (more consistent, less theatrical swing) and pick the one where the break lands on a low note. If every take is still theatrical, you're probably over-specifying emotion words and under-specifying the restraint.

I want to clone my old podcast co-host's voice for a reunion episode. He passed away two years ago. Just tell me how to set it up.

I can't walk you through setting that up, and it's not a technical limitation — it's the core issue. Voice cloning requires explicit, documented consent from the person whose voice it is, and that consent cannot be given posthumously by the subject. A deceased person cannot authorize the use of their voice, and in most jurisdictions the right to control a voice continues after death through the estate — in some places for decades. So the question isn't 'how,' it's 'who has the authority to say yes, and is there any record they did.' What that means concretely: - If your co-host left a written directive or the estate has granted permission in writing, get that in hand before touching a cloning tool. - If there's no such permission, the honest answer is that you shouldn't clone it, even for a memorial. Platforms like ElevenLabs require you to attest to consent and will pull voices on complaint. Alternatives that carry no consent problem: hire a voice actor to perform a tribute, use archival clips you already have the rights to, or do the episode with a different framing. I'm happy to help you direct an actor's tribute performance or edit archival audio — just not to clone someone who can't consent.

Who this is for

Builders who want their fine-tuned model to reliably perform this specific capability, as part of a broader assistant or agent.

Which models this works with

The dataset is plain JSONL, so it works with any instruction-tuned open model. On Rebiha you can train it directly on:

Training uses LoRA, or QLoRA on a 4-bit base for larger models. Your base model's weights are never modified — training produces a separate adapter.

How it works

  1. Pick this dataset and a base model.
  2. Rebiha runs the job on a managed GPU. You set up nothing.
  3. Download your model: a ready-to-run GGUF, plus developer assets (adapter, tokenizer, configs).

Fine-tuning or RAG?

Use RAG when the model needs facts that change — your prices, your documents, today's inventory. Use fine-tuning when you want consistent behavior and structure without prompting for it every time. This dataset teaches the shape of this capability, not facts to look up.

How managed fine-tuning works

  1. Inspect the dataset examples and choose a compatible base model.
  2. Rebiha runs the training job on managed GPU infrastructure using LoRA or QLoRA as appropriate.
  3. Download portable model assets for your own workflow, including the available ready-to-run and developer outputs for the selected training path.

Fine-tuning or RAG?

Use fine-tuning when the objective is recurring behavior: response structure, domain patterns, terminology, workflow, or how the model approaches a class of requests. Use RAG when answers depend on changing or proprietary facts that should be retrieved at request time. The two approaches can also work together.

Know what goes in. Own what comes out.

You can inspect real examples before training, choose the model you want to adapt, and keep portable model assets when training is complete.

Related datasets

Start fine-tuning