How to Extract and Preserve Background Sound in AI-Dubbed Videos

Want to retain original background sounds or music in your AI-dubbed videos? Our background sound extraction feature lets you preserve background audio while adding AI-generated voices, creating natural-sounding results.

When you use AI dubbing, our system automatically separates background sounds from the original track. This lets us blend the AI voice seamlessly with the background audio for a professional, immersive final product.

Upload video or audio file within Create a project (requires Beta of Video→Projects integration) or Translate video, audio, subtitles flow.

For Translate video, audio, subtitles flow:

• Choose an AI voice for your dubbing.

• The Extract Background Sound option appears automatically after voice selection.

• Background sound extraction begins immediately, with a progress indicator visible.

For Create a project flow:

• Translation review stage automatically starts background sound extracting

Note: background sound extraction process doesn’t block you from translation and review actions.

You can turn background sound on or off anytime during the process.

The video preview reflects your selection:

• If background sound is ON: You'll hear both AI voice and original background audio.

• If background sound is OFF: You'll hear only the AI voice.

• Your settings carry over to the final exported video.

Export the video when you're happy with the preview.

• If background sound is ON, you'll get AI voice mixed with background audio.

• If background sound is OFF, you'll get only the AI voice.

In future updates, you will be able to adjust the volume of background sounds to fine-tune the balance between AI voice and original audio.

• Enhance AI-dubbed videos by maintaining the natural feel of original media.

• Better user experience for marketing, customer support, and content creation.

• Flexible control over background audio for professional-quality results.

While this feature enhances the AI dubbing experience, there are some limitations to be aware of:

• Quality of Separation Varies: The effectiveness of voice separation depends on the complexity of the original audio. If the background music is similar in frequency to the voice, complete separation may not be possible.

• Loss of Audio Fidelity: Some background sounds may be partially distorted or removed due to limitations in AI separation models.

• Issues with Low-Quality Audio: Noisy recordings or audio with heavy reverb may result in imperfect separation.

• Speech Overlapping with Music: If the original voice and background sound are highly blended, artifacts may be noticeable in the extracted background track.

• Processing Time: Extracting background sound may take additional processing time, especially for long videos.

• Multi-Speaker Challenges: If there are multiple speakers in the original audio, separation accuracy may decrease.