

Streamvoice
Overview :
StreamVoice is a language model-based zero-lip speech conversion model that enables real-time conversion without requiring the complete source speech. It utilizes a full causal context-aware language model combined with a time-independent acoustic predictor, allowing it to alternately process semantic and acoustic features at each time step, thereby eliminating the dependency on complete source speech. To enhance the performance degradation that may arise in streaming due to incomplete context, StreamVoice employs two strategies to augment the language model's context-awareness: 1) Teacher-guided Context Prediction, where a teacher model summarizes the current and future semantic context during training, guiding the model to predict missing contexts; 2) Semantic Masking Strategy, which promotes acoustic prediction from previously damaged semantic and acoustic inputs, enhancing the contextual learning capability. Notably, StreamVoice is the first language model-based streaming zero-lip speech conversion model that does not require any future prediction. Experimental results demonstrate that StreamVoice exhibits streaming conversion capabilities while maintaining comparable zero-lip performance to non-streaming speech conversion systems.
Target Users :
StreamVoice can be used in fields such as music production, speech synthesis, and voice conversion.
Use Cases
In music production, use StreamVoice to convert a singer's voice into different singing styles.
In speech synthesis, use StreamVoice to convert text into different speaking style voices.
In voice conversion, use StreamVoice to convert a speaker's voice into different speaking styles.
Features
Real-time Zero-Lip Speech Conversion
Streaming Processing
Context-Aware Language Modeling
Featured AI Tools

Openvoice
OpenVoice is an open-source voice cloning technology capable of accurately replicating reference voicemails and generating voices in various languages and accents. It offers flexible control over voice characteristics such as emotion, accent, and can adjust rhythm, pauses, and intonation. It achieves zero-shot cross-lingual voice cloning, meaning it does not require the language of the generated or reference voice to be present in the training data.
AI speech recognition
2.4M

Chattts
ChatTTS is an open-source text-to-speech (TTS) model that allows users to convert text into speech. This model is primarily aimed at academic research and educational purposes and is not suitable for commercial or legal applications. It utilizes deep learning techniques to generate natural and fluent speech output, making it suitable for individuals involved in speech synthesis research and development.
AI speech synthesis
1.4M