SafeEar is an innovative audio depth detection framework that can identify deep audio without relying on speech content. The framework employs a neural audio codec design to separate semantic and acoustic information from audio samples, using solely acoustic features (such as prosody and timbre) for depth detection, thereby safeguarding the privacy of speech content. SafeEar enhances the detector's capabilities by augmenting the codec in real-world scenarios, enabling it to recognize various forms of deep audio. Extensive experiments conducted on four benchmark datasets demonstrate that SafeEar is highly effective in detecting various deep technologies, achieving a minimum equal error rate (EER) of 2.02%. Moreover, it protects speech content in five languages from deciphering by both machine and human auditory analysis, as validated by our user studies revealing a word error rate (WER) exceeding 93.93%. Additionally, SafeEar has established a benchmark for anti-deep and anti-content recovery assessments, laying the groundwork for future research in audio privacy protection and depth detection.

pdf-to-podcast is an AI-powered productivity tool that transforms PDF documents into podcast episodes. It utilizes OpenAI's text-to-speech model and Google Gemini technology to process PDF content into natural dialogue suitable for audio podcasts, outputting as MP3 files. The primary advantage of this tool is its ability to convert static document content into dynamic audio content, allowing users to listen on mobile devices while also serving as a source of material for podcast episodes.

bleep_that_sht is an application written in Python that uses the Whisper transcription model to transcribe audio and then replace selected keywords, using corresponding timestamps with beeps. All processing is done locally, no data is uploaded, and user privacy is protected.

FoleyCrafter is a text-based video to audio generation framework capable of producing high-quality audio that is semantically relevant to the input video and time-synced. This technology holds significant importance in video production, especially during post-production, where it can greatly enhance efficiency and audio quality. It was jointly developed by the Shanghai Artificial Intelligence Laboratory and the Chinese University of Hong Kong, Shenzhen.

The resemble-enhance AI model supports voice noise reduction and enhancement, capable of efficiently removing background noise, restoring voice details, and improving voice quality. It includes both noise reduction and enhancement modules, which separate voice signals from noise and enhance voice quality through deep learning algorithms. The model is trained for true HI-FI 44.1kHz voice, delivering high-quality enhanced speech. Users can install it via pip, or customize and train their own model based on provided code. This powerful yet user-friendly solution is the top choice for enhancing voice quality.

NVAS3d is a project for estimating sound at any location within a scene containing multiple unknown sound sources. It achieves novel-view acoustic synthesis by using audio recordings from multiple microphones and the 3D geometry and materials of the scene.

SafeEar is an innovative audio depth detection framework that can identify deep audio without relying on speech content. The framework employs a neural audio codec design to separate semantic and acoustic information from audio samples, using solely acoustic features (such as prosody and timbre) for depth detection, thereby safeguarding the privacy of speech content. SafeEar enhances the detector's capabilities by augmenting the codec in real-world scenarios, enabling it to recognize various forms of deep audio. Extensive experiments conducted on four benchmark datasets demonstrate that SafeEar is highly effective in detecting various deep technologies, achieving a minimum equal error rate (EER) of 2.02%. Moreover, it protects speech content in five languages from deciphering by both machine and human auditory analysis, as validated by our user studies revealing a word error rate (WER) exceeding 93.93%. Additionally, SafeEar has established a benchmark for anti-deep and anti-content recovery assessments, laying the groundwork for future research in audio privacy protection and depth detection.

pdf-to-podcast is an AI-powered productivity tool that transforms PDF documents into podcast episodes. It utilizes OpenAI's text-to-speech model and Google Gemini technology to process PDF content into natural dialogue suitable for audio podcasts, outputting as MP3 files. The primary advantage of this tool is its ability to convert static document content into dynamic audio content, allowing users to listen on mobile devices while also serving as a source of material for podcast episodes.

bleep_that_sht is an application written in Python that uses the Whisper transcription model to transcribe audio and then replace selected keywords, using corresponding timestamps with beeps. All processing is done locally, no data is uploaded, and user privacy is protected.

FoleyCrafter is a text-based video to audio generation framework capable of producing high-quality audio that is semantically relevant to the input video and time-synced. This technology holds significant importance in video production, especially during post-production, where it can greatly enhance efficiency and audio quality. It was jointly developed by the Shanghai Artificial Intelligence Laboratory and the Chinese University of Hong Kong, Shenzhen.

Text to Sound Effects is ElevenLabs' latest AI audio model, capable of generating various sound effects, short music tracks, ambiences, and character voices based on text prompts. It represents a major innovation in the field of audio production, providing film and television studios, video game developers, and social media content creators with fast, economical, and scalable tools for generating rich and immersive audio environments. The product, through its collaboration with Shutterstock, leverages authorized tracks from its vast audio library, meticulously fine-tuned, to create a versatile new tool for modern creators.

The resemble-enhance AI model supports voice noise reduction and enhancement, capable of efficiently removing background noise, restoring voice details, and improving voice quality. It includes both noise reduction and enhancement modules, which separate voice signals from noise and enhance voice quality through deep learning algorithms. The model is trained for true HI-FI 44.1kHz voice, delivering high-quality enhanced speech. Users can install it via pip, or customize and train their own model based on provided code. This powerful yet user-friendly solution is the top choice for enhancing voice quality.

NVAS3d is a project for estimating sound at any location within a scene containing multiple unknown sound sources. It achieves novel-view acoustic synthesis by using audio recordings from multiple microphones and the 3D geometry and materials of the scene.

AudioSep is an open-domain audio source separation model based on natural language queries. It consists of two key components: a text encoder and a separation model. We trained AudioSep on a large-scale multimodal dataset and extensively evaluated its capabilities on many tasks, including audio event separation, instrument separation, and voice enhancement. AudioSep demonstrates strong separation performance and impressive zero-shot generalization ability, significantly outperforming previous audio query and language query sound separation models when using audio titles or text labels as queries. To ensure the reproducibility of this work, we will release the source code, evaluation benchmark, and pre-trained models.