LLaSA是一种由HKUSTAudio团队开发的基于LLaMA的文本到语音合成系统，旨在通过结合LLaMA（1B, 3B, 8B）语言模型与XCodec2语音编码器，生成高质量的语音合成。该模型能够从输入文本或语音提示生成语音，且能够支持中英双语的语音合成。LLaSA模型在一个包含250,000小时中英双语语音数据集的基础上进行了训练，并且能够有效地处理和生成自然流畅的语音。
要使用LLaSA进行文本到语音合成，首先需要安装XCodec2库。可以通过以下步骤进行安装并开始使用： 1. 创建一个新的环境并激活： ``` conda create -n xcodec2 python=3.9 conda activate xcodec2 pip install xcodec2==0.1.3 ``` 2. 使用LLaSA模型从文本生成语音： ``` from transformers import AutoTokenizer, AutoModelForCausalLM import torch import soundfile as sf llasa_1b ='HKUST-Audio/Llasa-1B' tokenizer = AutoTokenizer.from_pretrained(llasa_1b) model = AutoModelForCausalLM.from_pretrained(llasa_1b) model.eval() model.to('cuda') from xcodec2.modeling_xcodec2 import XCodec2Model model_path = \"HKUST-Audio/xcodec2\" Codec_model = XCodec2Model.from_pretrained(model_path) Codec_model.eval().cuda() input_text = 'Dealing with family secrets is never easy. Yet, sometimes, omission is a form of protection, intending to safeguard some from the harsh truths. One day, I hope you understand the reasons behind my actions. Until then, Anna, please, bear with me.' ``` 通过上述代码，您可以输入文本，模型将生成对应的语音输出。
LLaSA模型的使用和训练过程是免费的，您可以根据需要在Hugging Face平台上下载和使用。
HKUSTAudio
暂无官方邮件地址
Twitter: @HKUSTAudio, Instagram: @hkust_audio
LLaSA is an advanced system designed to scale both training and inference for LLaMA-based speech synthesis. It optimizes computational efficiency, leveraging large-scale datasets and cutting-edge machine learning frameworks to enhance text-to-speech performance.
Llasa是一款兼容Llama框架的文本到语音（TTS）基础模型，基于160k小时的标记语音数据，广泛应用于语音合成领域，支持多种语言和语音风格。
Fuse ChatTTS with OpenVoice, upload a 10-second audio clip, and clone your personalized ChatTTS voice.
Zonos-v0.1 is an advanced text-to-speech model with multilingual support, offering high-quality voice cloning and speech generation with detailed control over emotions, pitch, and speaking style.
Video-T1 explores test-time scaling (TTS) for video generation, enhancing video quality and consistency with prompts by leveraging additional inference-time computation.
