[{"data":1,"prerenderedAt":730},["ShallowReactive",2],{"blog-shrinking-full-duplex-speech":3},{"id":4,"title":5,"author":6,"body":7,"categories":714,"citations":719,"date":720,"description":721,"extension":722,"hidden":723,"meta":724,"navigation":725,"path":726,"seo":727,"stem":728,"thumbnail":719,"__hash__":729},"blog\u002Fblog\u002Fshrinking-full-duplex-speech.md","Shrinking Full-Duplex Speech: Training a 600M Moshi-Style Model on a Weekend Budget","Anurag Kanade",{"type":8,"value":9,"toc":696},"minimark",[10,27,36,39,44,57,60,188,193,208,211,218,222,235,238,241,245,248,254,283,298,313,318,329,332,335,339,342,346,349,352,358,362,365,371,375,378,385,389,398,404,407,482,487,493,500,504,519,522,525,529],[11,12,13,14,18,19,26],"p",{},"Full-duplex conversation, the ability of a speech model to listen and speak at the same time, is usually discussed as a property of ",[15,16,17],"em",{},"scale",": Moshi (",[20,21,25],"a",{"href":22,"rel":23},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.00037",[24],"nofollow","Défossez et al., 2024",") demonstrated it with a 7B model pretrained on seven million hours of audio, and most systems that followed kept roughly that budget. The interesting question is whether duplexity actually requires any of that, or whether it is a data and objective problem that a much smaller model could learn: small enough to train in two GPU-days and run on a laptop.",[11,28,29,30,35],{},"There was reason to think so. PersonaPlex (",[20,31,34],{"href":32,"rel":33},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2602.06053",[24],"Roy et al., 2026",") took stock Moshi, changed nothing about the architecture, and taught it voice and role control with roughly 2,250 hours of synthetic dialog and about 48 A100-hours of fine-tuning. Their data ablation is the load-bearing result for anyone attempting this on a small budget: 25% of their data retained most of the quality. Turn-taking, barge-in, and backchanneling, the behaviors that make duplex models feel alive, appear to be learnable from a few hundred hours of the right data.",[11,37,38],{},"So the plan was a \"Kokoro-class Moshi\": a model of roughly 600M parameters, English-only, one fixed persona voice, trained end to end on a budget a single person can carry, with a recipe others could re-run for their own language. This post covers the full attempt: the synthetic data pipeline, a distillation trick I think is underused, four training runs, and a diagnosis of where the small model broke. The project is paused for capital rather than concluded, but the failure mode turned out to be specific and measurable, and that specificity is the useful part.",[40,41,43],"h2",{"id":42},"the-architecture-we-inherit","The Architecture We Inherit",[11,45,46,47,50,51,56],{},"Moshi models a two-person conversation as a grid of discrete tokens at 12.5 Hz. Audio enters through Mimi, a streaming neural codec that compresses 24 kHz speech into 8 residual vector-quantization (RVQ) codebooks per 80 ms frame. Codebook 0 is ",[15,48,49],{},"semantic",", distilled from WavLM (",[20,52,55],{"href":53,"rel":54},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2110.13900",[24],"Chen et al., 2021","), and carries phonetic content. Codebooks 1 through 7 encode successively finer acoustic residuals. Mimi is about 80M parameters, released under CC-BY-4.0, and stays frozen throughout; nothing here involves training a codec.",[11,58,59],{},"Each frame of the conversation contains seventeen tokens:",[61,62,63,88],"table",{},[64,65,66],"thead",{},[67,68,69,73,76,79,82,85],"tr",{},[70,71,72],"th",{},"Stream",[70,74,75],{},"t",[70,77,78],{},"t+1",[70,80,81],{},"t+2",[70,83,84],{},"t+3",[70,86,87],{},"Notes",[89,90,91,112,132,151,171],"tbody",{},[67,92,93,97,100,103,106,109],{},[94,95,96],"td",{},"0 · agent text",[94,98,99],{},"\"I\"",[94,101,102],{},"\"'m\"",[94,104,105],{},"PAD",[94,107,108],{},"\"here\"",[94,110,111],{},"text leads audio",[67,113,114,117,120,123,126,129],{},[94,115,116],{},"1 · agent audio cb0",[94,118,119],{},"a₀",[94,121,122],{},"a₁",[94,124,125],{},"a₂",[94,127,128],{},"a₃",[94,130,131],{},"semantic codebook",[67,133,134,137,140,142,145,148],{},[94,135,136],{},"2–8 · agent audio cb1–7",[94,138,139],{},"·",[94,141,139],{},[94,143,144],{},"a₀′",[94,146,147],{},"a₁′",[94,149,150],{},"acoustic, delayed +2",[67,152,153,156,159,162,165,168],{},[94,154,155],{},"9 · user audio cb0",[94,157,158],{},"u₀",[94,160,161],{},"u₁",[94,163,164],{},"u₂",[94,166,167],{},"u₃",[94,169,170],{},"input at inference",[67,172,173,176,178,180,183,186],{},[94,174,175],{},"10–16 · user audio cb1–7",[94,177,139],{},[94,179,139],{},[94,181,182],{},"u₀′",[94,184,185],{},"u₁′",[94,187,170],{},[11,189,190],{},[15,191,192],{},"Figure 1: The duplex token grid; one row is one stream, one column is one 80 ms frame. Stream layout and ordering follow Moshi.",[11,194,195,196,199,200,203,204,207],{},"Two design choices in this grid do most of the work. The first is the ",[15,197,198],{},"inner monologue",": the agent's own text stream is time-aligned with its audio and shifted slightly earlier, so the model decides what to say in text before it vocalizes it. This is the mechanism that lets the language modeling happen in a space the model is good at, and it yields streaming ASR and TTS behavior as side effects. The second is the RQ-Transformer factorization: a large ",[15,201,202],{},"temporal"," transformer runs across frames, consuming one summed embedding per 80 ms step, while a small ",[15,205,206],{},"depth"," transformer autoregressively predicts the seventeen tokens within the next frame. The expensive model never sees a sequence seventeen times longer than the conversation.",[11,209,210],{},"PersonaPlex sits entirely on top of this architecture. Its contribution is a conditioning scheme and a data recipe, not a change to the model. The conditioning is a hybrid system prompt: an audio clip of the target voice on the agent streams, temporally concatenated with a text role prompt forced onto the text stream, a 440 Hz sine on the user channel, and loss masked over the whole prefix. The data recipe is synthetic dialogs produced by LLM transcript generation, TTS rendering, and stitching. Their loss weighting (acoustic codebooks ×0.02, padded text tokens ×0.3) comes up again later, because inheriting it turned out to be a mistake in this setting.",[11,212,213],{},[214,215],"img",{"alt":216,"src":217},"The student model: user audio and the hybrid system prompt feed a Qwen3-0.6B-Base temporal transformer, which drives a 6-layer depth transformer; that produces the agent text stream (leading by 2 frames) and 8 agent audio codebooks (codebooks 1-7 delayed by 2), with the audio codes decoded to 24 kHz speech by a frozen Mimi decoder","\u002Fdiag_1.png",[40,219,221],{"id":220},"the-central-bet-replace-audio-pretraining-with-a-text-pretrained-backbone","The Central Bet: Replace Audio Pretraining with a Text-Pretrained Backbone",[11,223,224,225,228,229,234],{},"Moshi's seven million hours were mostly spent teaching its backbone ",[15,226,227],{},"language",", from audio, from nothing. That is the one expense a small project cannot pay and also the one it shouldn't have to: pretrained text LMs already exist at every size. So the temporal transformer here is initialized from Qwen3-0.6B-Base (",[20,230,233],{"href":231,"rel":232},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.09388",[24],"Qwen Team, 2025","), the depth transformer is trained from scratch, and the wager is that the inner monologue bridges the two worlds: the backbone already knows how to conduct a conversation in text, and training only has to teach it to read and write Mimi codes.",[11,236,237],{},"The verified model is 773M parameters, of which 156M are text embeddings tied three ways (backbone input, depth-transformer input, depth text head), leaving about 617M working parameters. Pruning Qwen's 150k-entry multilingual vocabulary to English would cut another ~130M; that stayed on the v2 list.",[11,239,240],{},"A back-of-envelope that held up throughout: at 12.5 Hz, one epoch over a thousand hours of audio is about 45 million frames. At roughly 6 FLOPs per parameter per token (forward and backward combined) and ~700M active parameters, that is on the order of a few hundred petaFLOPs, call it half an exaFLOP. On a 96 GB RTX Pro 6000 at modest utilization, that lands around two hours. Compute was never the constraint. The constraints were data hours and, as it turned out, how the loss allocates capacity across those seventeen streams.",[40,242,244],{"id":243},"building-the-data","Building the Data",[11,246,247],{},"Everything in this section ran on one rented RTX Pro 6000 and a Mac.",[11,249,250],{},[214,251],{"alt":252,"src":253},"The data pipeline: LLM-generated dialogs go through TTS rendering and duplex stitching, then a QC filter keeps 3,191 of 4,569 (70%); PersonaPlex 7B teacher rollouts add 423 more; both feed the final Mimi tokenize step producing 3,614 token grids","\u002Fdiag_2.png",[11,255,256,257,260,261,265,266,269,270,269,273,269,276,269,279,282],{},"Transcript generation follows PersonaPlex's hierarchical sampling: draw a domain, a scenario, an angle, then have an LLM write the full two-speaker exchange. The adjustment is where diversity comes from. PersonaPlex gets diversity from thousands of personas and roles; a fixed-persona model can't, so diversity here comes from ",[15,258,259],{},"conversational dynamics"," instead. Every turn carries a ",[262,263,264],"code",{},"dynamic_tag"," (",[262,267,268],{},"normal",", ",[262,271,272],{},"backchannel",[262,274,275],{},"interruption",[262,277,278],{},"overlap",[262,280,281],{},"pause_before_long",") that the stitching stage later turns into physics. 5,496 dialogs came out of this, which deduplicated to 4,569 after I discovered the generation script's resume path had been reusing dialog IDs.",[11,284,285,286,291,292,297],{},"Each turn was rendered with Higgs TTS 3 (",[20,287,290],{"href":288,"rel":289},"https:\u002F\u002Fhuggingface.co\u002Fbosonai\u002Fhiggs-tts-3-4b",[24],"Boson AI",", a 4B Qwen3-based TTS with zero-shot cloning) behind an sglang server, with the agent cloned onto Kore, one of the two clean voices in the Apache-licensed ",[20,293,296],{"href":294,"rel":295},"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fshb777\u002Fgemini-flash-2.0-speech",[24],"gemini-flash-2.0-speech"," dataset. Kore was chosen so the same 284 hours could serve as alignment data later with a consistent persona voice. Stitching then assembles two-channel duplex audio where the tags become timing: normal turns get 300–800 ms of gap, backchannels and overlaps get a small negative gap, interruptions truncate the previous turn mid-word. This sounds too crude to work, but PersonaPlex validated exactly this negative-silence trick for teaching barge-in mechanics.",[11,299,300,301,306,307,312],{},"What the plan had treated as an optional cleanup pass turned out to be structural. Raw Higgs renders carried a 10–20% per-dialog defect rate: silent turns, clipping, and runaway generations (forty seconds of audio for nine seconds of text, though most of the excess proved to be silence tails that trimming could repair rather than reject). The best failure was linguistic: the transcript generator's language filter operated on whole dialogs, so around four hundred Japanese backchannels (うん, なるほど) sat undetected inside otherwise-English conversations, because a two-token acknowledgment doesn't move a document-level language classifier. The filter pass runs re-ASR word error rate against the clean transcript using Whisper (",[20,302,305],{"href":303,"rel":304},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2212.04356",[24],"Radford et al., 2022","), ECAPA speaker similarity against Kore references, artifact detection, and loudness normalization, and kept 3,191 of 4,569 dialogs. Kokoro (",[20,308,311],{"href":309,"rel":310},"https:\u002F\u002Fhuggingface.co\u002Fhexgrad\u002FKokoro-82M",[24],"hexgrad",") is often described as an architecture achievement; I read it mainly as a curation result, and at this scale the arithmetic agrees: five hundred clean hours beat fifteen hundred noisy ones.",[314,315,317],"h3",{"id":316},"distilling-the-teachers-timing-not-its-content","Distilling the Teacher's Timing, Not Its Content",[11,319,320,321,324,325,328],{},"Hand-stitched gaps teach a model the ",[15,322,323],{},"mechanics"," of overlap but not human micro-timing: when precisely to murmur agreement, how quickly to yield when interrupted. PersonaPlex had real telephony-style data for that; I did not. But PersonaPlex itself is released under the NVIDIA Open Model License, which permits derivatives, and it is a model whose entire value in this context is that it already ",[15,326,327],{},"has"," the timing.",[11,330,331],{},"So a rollout script streams our rendered user channels into PersonaPlex 7B, conditioned with a Kore voice prompt and the dialog's own role context, and records its live agent response, capturing the full seventeen-stream token grid per frame, training-ready without re-tokenization. The teacher's learned turn-taking becomes supervision on exactly the axis where synthetic stitching is weakest. The practical numbers: each instance wants ~24 GB of VRAM (three fit on a 96 GB card; four OOM each other) and runs at about 0.7× realtime, which cut the target from 2,000 rollouts to 423. PersonaPlex's own scaling ablation suggests this regime is sufficient for a timing signal, and the training loader oversamples rollouts 2×. If a strong open model exists in your target behavior space, I now think on-policy rollouts of it are worth trying before hand-built alternatives.",[11,333,334],{},"The final dataset is 3,614 token grids, stitched corpus and rollouts in one format. Mimi tokenization of the whole ~100 hours took fifteen minutes on one GPU.",[40,336,338],{"id":337},"training","Training",[11,340,341],{},"Training ran on Modal B200 instances. Each run below cost between $2 and $15.",[314,343,345],{"id":344},"finding-one-the-text-channel-converges-almost-immediately-and-the-audio-channel-does-not","Finding One: The Text Channel Converges Almost Immediately, and the Audio Channel Does Not",[11,347,348],{},"The first run went straight to duplex dialog fine-tuning, 6,000 steps, about 95 minutes. Validation text loss bottomed out near 1.68 by step three to four thousand, and free-run samples (the model continuing a conversation given only the user's audio channel) produced coherent, on-persona text: reassurances, confirmation-email service talk, sensible responses to what the user had actually said. The central bet paid out in full on the text side; the pretrained backbone needed almost no persuading to conduct a spoken-style conversation over Mimi-clocked frames.",[11,350,351],{},"The audio told a different story. Speech was present, with roughly the right prosodic contour, but acoustically rough to the point of being hard to make out, and validation audio loss sat flat at 1.72 from step four thousand onward.",[11,353,354],{},[214,355],{"alt":356,"src":357},"Run 1, text loss (blue) keeps falling toward 1.1 while audio loss (green) flattens around 1.6 from step 3,000 onward on a 6,000-step duplex fine-tuning run on a B200; hollow markers are validation","https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fedwixx\u002Fs2s-blog-assets\u002Fresolve\u002Fmain\u002Ffigures\u002Ffig_stageB_loss.png",[314,359,361],{"id":360},"finding-two-loss-weighting-is-not-the-bottleneck","Finding Two: Loss Weighting Is Not the Bottleneck",[11,363,364],{},"The obvious suspect was the inherited ×0.02 weight on acoustic codebooks. PersonaPlex tuned that number for a model whose audio embeddings had already seen seven million hours of pretraining; the audio embeddings here started from a random initialization, so a starvation explanation fit the flat loss curve. The second run raised the weight to 0.15, keeping everything else fixed. Text quality was unchanged and the audio did not improve. The loss weight was not the bottleneck.",[11,366,367],{},[214,368],{"alt":369,"src":370},"Validation audio loss at acoustic weight 0.02 versus 0.15; the absolute levels are not directly comparable since each run's audio loss is scaled by its own acoustic weight, but both curves flatten at the same step with the same shape and the 0.15 run did not sound clearer","https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fedwixx\u002Fs2s-blog-assets\u002Fresolve\u002Fmain\u002Ffigures\u002Ffig_aw15_val_audio.png",[314,372,374],{"id":373},"finding-three-ruling-out-a-bug","Finding Three: Ruling Out a Bug",[11,376,377],{},"Before spending more on training, three checks separated a bug from a capacity problem, for about $2 of GPU time total. Decoding stored training tokens straight through Mimi produced clean speech, so the tokenization pipeline was correct. Applying the acoustic delay and then inverting it reproduced the direct decode to four decimal places, so the delay handling was correct. Overfitting a single dialog for 400 steps drove audio loss from 7.79 to 0.28, close to a full reconstruction, so the model was mechanically capable of learning audio. None of these checks needed a training run, and they should have run before the first one rather than after the second.",[11,379,380,381,384],{},"With a bug ruled out, two structural causes remained, both avoidable. First, copying Moshi's objective meant the model spent eight of its sixteen audio streams learning to ",[15,382,383],{},"predict the user's voice",", tokens that are pure input at inference time. At 7B with unlimited data that multi-task load is apparently free; at 600M with three hundred hours it starves the one voice that ships. The fix is a one-line change in the loss: user streams become input-only with zero weight. Second, I had skipped the planned alignment stage (single-speaker TTS in the same frame format) to save budget, when the general pattern with small TTS models (Kokoro-class systems get intelligible on 24–100 hours of single-voice data) says a model should learn to read aloud before it learns to converse.",[314,386,388],{"id":387},"finding-four-the-garble-lives-in-the-fine-codebooks","Finding Four: The Garble Lives in the Fine Codebooks",[11,390,391,392,397],{},"The alignment stage went in as a pivot: TTS-as-duplex on ~600 hours of single-speaker audio (gemini-flash plus LibriTTS-R, ",[20,393,396],{"href":394,"rel":395},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.18802",[24],"Koizumi et al., 2023","), agent-only prediction, acoustic weight raised to 0.5, run to 40,000 steps. The result sharpened the problem rather than solving it. Teacher-forced output remained substantially garbled, but text-forced generation (forcing the ground-truth text stream and letting the model generate only audio) was noticeably clearer, recognizable if robotic. A gap of that shape, where the model performs better the more of its own predictions you replace with ground truth, is the classic signature of exposure bias: the acoustic decoder is learning, but it is conditioned on a self-generated text and semantic stream that has not stabilized.",[11,399,400],{},[214,401],{"alt":402,"src":403},"Alignment-stage run across its resumed segments to 40,000 steps; training text loss collapses toward zero while validation text loss climbs steadily as the small read-aloud corpus gets memorized, and audio loss grinds down slowly without reaching the region where Mimi decodes cleanly","https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fedwixx\u002Fs2s-blog-assets\u002Fresolve\u002Fmain\u002Ffigures\u002Ffig_stageA_40k.png",[11,405,406],{},"The most instructive experiment of the project was also the cheapest. Freeze the Qwen backbone entirely, train only the audio embeddings and depth transformer on a 32-dialog overfit set with acoustic weight 1.0, and log cross-entropy separately for each RVQ codebook. A single aggregate audio loss number had been averaging this away the whole time.",[61,408,409,425],{},[64,410,411],{},[67,412,413,416,419,422],{},[70,414,415],{},"codebook",[70,417,418],{},"role",[70,420,421],{},"CE at step 20",[70,423,424],{},"CE at step 1500",[89,426,427,440,454,468],{},[67,428,429,432,434,437],{},[94,430,431],{},"cb0",[94,433,49],{},[94,435,436],{},"0.45",[94,438,439],{},"0.29",[67,441,442,445,448,451],{},[94,443,444],{},"cb1",[94,446,447],{},"coarse acoustic",[94,449,450],{},"3.31",[94,452,453],{},"1.25",[67,455,456,459,462,465],{},[94,457,458],{},"cb4",[94,460,461],{},"mid acoustic",[94,463,464],{},"4.64",[94,466,467],{},"2.04",[67,469,470,473,476,479],{},[94,471,472],{},"cb7",[94,474,475],{},"fine acoustic",[94,477,478],{},"5.43",[94,480,481],{},"3.21",[11,483,484],{},[15,485,486],{},"Table 1: Per-codebook audio cross-entropy during the frozen-backbone rescue run on a 32-example overfit set. The semantic codebook is essentially solved; loss rises monotonically with RVQ depth, and the finest levels resist even deliberate overfitting.",[11,488,489],{},[214,490],{"alt":491,"src":492},"The same run as Table 1, over time; codebook 0 sits near the floor from the start, each deeper acoustic level converges to a higher plateau, and codebook 7 is still above 3 nats after 1,500 steps of deliberate overfitting at acoustic weight 1.0, the ordering never inverts","https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fedwixx\u002Fs2s-blog-assets\u002Fresolve\u002Fmain\u002Ffigures\u002Ffig_rescue_codebooks.png",[11,494,495,496,499],{},"There is the garble, quantified. The model learned ",[15,497,498],{},"what"," to say acoustically (codebook 0, the phonetic content) but could not master the fine residual detail that separates clean speech from smear. This is not mysterious once stated: each RVQ level encodes the residual of the level before it, so the fine codebooks are individually the closest to noise and the hardest to predict, and they were precisely the streams that every budget decision (the ×0.02 inherited weight, the absent audio pretraining, a 100M shared-weight depth transformer) had deprioritized. Moshi could afford that allocation because its scale paid the fine codebooks' bill elsewhere. A cold-start 600M model cannot, and no aggregate loss curve will tell you so unless you ask per codebook.",[40,501,503],{"id":502},"where-this-leaves-the-project","Where This Leaves the Project",[11,505,506,507,512,513,518],{},"The open thread is specific. Stabilize the self-generated text and semantic streams, either scheduled sampling or a semantic-first curriculum that trains codebooks 0 and 1 to convergence before unlocking deeper levels. Give the fine codebooks more alignment hours (Emilia, ",[20,508,511],{"href":509,"rel":510},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2407.05361",[24],"He et al., 2024",", is the obvious source) or a depth transformer that isn't sharing 100M parameters across seven residual levels it has never heard in pretraining. After that, the rest of the original plan still stands: streaming KV-cached inference at the 80 ms frame budget, evaluation on Full-Duplex-Bench (",[20,514,517],{"href":515,"rel":516},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2503.04721",[24],"Lin et al., 2025","), and quantized exports in the 450–800 MB range. The data (3,614 duplex grids), the checkpoints, and the pipeline are intact and waiting on compute.",[11,520,521],{},"None of the findings above needed a large training run to discover. The decode round-trip, the single-dialog overfit, the frozen-backbone rescue, and the per-codebook logging together cost a few dollars, and they turned \"the audio sounds bad\" into \"codebooks 1 through 7 are under-supervised and conditioned on an unstable stream,\" a statement precise enough to act on. The lesson from the failed part is about inheritance: borrowing an architecture also means borrowing its objective and loss weighting, and those encode assumptions about scale and pretraining that don't transfer just because the layer diagram does. Moshi's decision to predict both speakers and to down-weight acoustic detail was correct for Moshi. The layer diagram transferred; the assumptions did not, and seeing the difference cost two training runs. The text half of the bet, that a pretrained text LM plus the inner monologue gives you conversation for close to free, held completely. That's why I still think a small full-duplex model is a data problem rather than a scale problem, and worth finishing.",[11,523,524],{},"If you're working on small duplex models, or want to run this recipe for your language, say hi.",[40,526,528],{"id":527},"references","References",[530,531,532,555,575,594,609,621,633,646,659,672,684],"ol",{},[533,534,535,539,540,543,544,548,549,554],"li",{},[536,537,538],"strong",{},"Défossez, A., Mazaré, L., Orsini, M., Royer, A., Pérez, P., Jégou, H., Grave, E., & Zeghidour, N."," (2024). ",[15,541,542],{},"Moshi: a speech-text foundation model for real-time dialogue",". ",[20,545,547],{"href":22,"rel":546},[24],"arXiv:2410.00037",". Code and Mimi weights: ",[20,550,553],{"href":551,"rel":552},"https:\u002F\u002Fgithub.com\u002Fkyutai-labs\u002Fmoshi",[24],"kyutai-labs\u002Fmoshi",".",[533,556,557,560,561,543,564,568,569,574],{},[536,558,559],{},"Roy, R., Raiman, J., Lee, S., et al."," (NVIDIA) (2026). ",[15,562,563],{},"PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models",[20,565,567],{"href":32,"rel":566},[24],"arXiv:2602.06053",". Checkpoint: ",[20,570,573],{"href":571,"rel":572},"https:\u002F\u002Fhuggingface.co\u002Fnvidia\u002Fpersonaplex-7b-v1",[24],"nvidia\u002Fpersonaplex-7b-v1"," (NVIDIA Open Model License).",[533,576,577,580,581,543,584,588,589,554],{},[536,578,579],{},"Qwen Team."," (2025). ",[15,582,583],{},"Qwen3 Technical Report",[20,585,587],{"href":231,"rel":586},[24],"arXiv:2505.09388",". Backbone: ",[20,590,593],{"href":591,"rel":592},"https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-0.6B-Base",[24],"Qwen\u002FQwen3-0.6B-Base",[533,595,596,599,600,603,604,608],{},[536,597,598],{},"Boson AI."," ",[15,601,602],{},"Higgs TTS 3",". Model: ",[20,605,607],{"href":288,"rel":606},[24],"bosonai\u002Fhiggs-tts-3-4b"," (Boson Higgs TTS 3 Research and Non-Commercial License, disclosed here since it rendered the training audio).",[533,610,611,599,614,616,617,554],{},[536,612,613],{},"shb777.",[15,615,296],{},", 284 h, two voices (Puck\u002FKore), Apache-2.0. ",[20,618,620],{"href":294,"rel":619},[24],"Dataset",[533,622,623,599,626,543,629,554],{},[536,624,625],{},"hexgrad.",[15,627,628],{},"Kokoro-82M",[20,630,632],{"href":309,"rel":631},[24],"Model",[533,634,635,638,639,543,642,554],{},[536,636,637],{},"Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I."," (2022). ",[15,640,641],{},"Robust Speech Recognition via Large-Scale Weak Supervision",[20,643,645],{"href":303,"rel":644},[24],"arXiv:2212.04356",[533,647,648,651,652,543,655,554],{},[536,649,650],{},"Chen, S., et al."," (2021). ",[15,653,654],{},"WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing",[20,656,658],{"href":53,"rel":657},[24],"arXiv:2110.13900",[533,660,661,664,665,543,668,554],{},[536,662,663],{},"Koizumi, Y., et al."," (2023). ",[15,666,667],{},"LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus",[20,669,671],{"href":394,"rel":670},[24],"arXiv:2305.18802",[533,673,674,539,677,543,680,554],{},[536,675,676],{},"He, H., et al.",[15,678,679],{},"Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation",[20,681,683],{"href":509,"rel":682},[24],"arXiv:2407.05361",[533,685,686,580,689,543,692,554],{},[536,687,688],{},"Lin, G.-T., et al.",[15,690,691],{},"Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities",[20,693,695],{"href":515,"rel":694},[24],"arXiv:2503.04721",{"title":697,"searchDepth":698,"depth":698,"links":699},"",3,[700,702,703,706,712,713],{"id":42,"depth":701,"text":43},2,{"id":220,"depth":701,"text":221},{"id":243,"depth":701,"text":244,"children":704},[705],{"id":316,"depth":698,"text":317},{"id":337,"depth":701,"text":338,"children":707},[708,709,710,711],{"id":344,"depth":698,"text":345},{"id":360,"depth":698,"text":361},{"id":373,"depth":698,"text":374},{"id":387,"depth":698,"text":388},{"id":502,"depth":701,"text":503},{"id":527,"depth":701,"text":528},[715,716,717,718],"TTS","Speech-to-Speech","Full-Duplex","Machine Learning",null,"2026-07-24","An attempt at a sub-1B full-duplex speech-to-speech model built on a pretrained text backbone instead of Moshi's audio pretraining, with a synthetic data pipeline, on-policy distillation from PersonaPlex, and a per-codebook diagnosis of exactly where the small model broke.","md",false,{},true,"\u002Fblog\u002Fshrinking-full-duplex-speech",{"title":5,"description":721},"blog\u002Fshrinking-full-duplex-speech","vDtn_M60iiL9SW8K7iEo-CODFJwhfpBZ6ReGKEo77HY",1785355553623]