Voice Matching is a separate audio app I built around an existing song and a target-voice reference. It takes the performance, converts the vocal character, and mixes the converted voice back in with the accompaniment.
It's related to my other AI media work, but it handles a different part of the process. Lip-match Studio works with faces and video. Voice Matching works with audio: prep, conversion, timing, previews, and the balance of the final song.
That difference matters because a converted vocal is only one piece. It still has to line up with the source, sit at a sensible level, and be something I can listen to and revise.
Choosing a useful voice reference
The app saves voice references, so I can come back to a profile without uploading the same material again. It stores reference clips separately and picks a good excerpt for each render.
The reference matters a lot. A clip with background music, another speaker, or a heavily processed performance can confuse the converter. More recordings aren't automatically better if I combine them without thinking about what's in them.
So I can choose a specific reference and where it starts. That makes the input explicit. Each render records which excerpt it used, instead of pulling from a big pile of recordings with no clear source.
This is reference conditioning. It isn't training a new model on all the clips I upload, and I don't describe it that way.
Separating the song into useful parts
When the input is a full song, the first real step is separation. The app needs a vocal track to convert and an accompaniment track to keep in the mix. If I already have a clean vocal, I can skip that step.
Those separated stems are useful after the first render too. If I want to change the balance later, I shouldn't have to separate the song and run the conversion again just to adjust the music's volume.
The project caches the separated material and keeps the outputs for later remixing. That's one of the choices that makes the model practical to use across several attempts.
Running the pipeline locally
The code uses a Seed-VC adapter for the conversion and a separate audio-separation environment. The app has to coordinate different model and library requirements, including parts that run on Apple Silicon through different compute paths.
Getting that setup working is part of the project. The interface needs a reliable way to pass the source audio, the reference, and the settings into the engine, collect the result, and report failures. It also has to keep the originals safe, so a cancelled or failed job doesn't damage the input.
Jobs save their settings and progress. The app runs one heavy job at a time, supports cancellation, and recognizes work that was interrupted when the process stopped. Those are ordinary software problems, but they matter when a render takes long enough that I might walk away from it.
Previewing before committing to a whole song
The app can generate a short preview instead of rendering the whole input right away. That gives me something small to judge before I spend time on a full track.
The preview is where the real judgment happens. Does the voice sound coherent? Can you understand the words? Is some of the original singer still audible? Does it sit naturally against the music? A job reporting success can't answer those questions.
The code also runs technical checks on duration, valid samples, and peak levels. Those checks confirm the output is a usable audio file with the expected timing. They don't prove the voice sounds like the target or that the performance is good.
Finishing the mix
After conversion, the app lines up the vocal with the expected source length and mixes it with the accompaniment. Level matching keeps the converted vocal from overpowering the backing just because the model put out a louder signal.
The remix controls cover vocal and music levels, key changes, and some finishing adjustments. They work on the saved stems wherever possible, so the expensive conversion step stays separate from later editing decisions.
That separation makes the app behave more like a media tool. Generation creates material. Listening and mixing decide what to do with it. A successful render is the start of something I can review, not the end of the creative process.
Voice Matching gave me hands-on experience connecting models to an audio workflow, with inputs, caches, previews, jobs, and exports. The finished app is worth describing because each piece has a clear job in getting from a reference clip to a mix I can actually judge.