新网建立网站简约大气网站模板

新网建立网站,简约大气网站模板,龙岗网站维护,新手怎么引流推广简介说明 Infinite Talk数字人对口型图像转视频AI工具 支持无限时长视频生成 InfiniteTalk AI#xff1a;音频驱动的视频生成框架 InfiniteTalk AI是由 MeiGen AI 开发的先进音频驱动视频生成框架#xff0c; 专注于将静态图像或现有视频与音频结合#xff0c;生成无限时长…简介说明Infinite Talk数字人对口型图像转视频AI工具 支持无限时长视频生成InfiniteTalk AI音频驱动的视频生成框架InfiniteTalk AI是由 MeiGen AI 开发的先进音频驱动视频生成框架专注于将静态图像或现有视频与音频结合生成无限时长的逼真说话视频实现精准唇形同步和全身动作协调。核心功能核心优势说明精准唇形同步口型与音频毫秒级匹配自然度远超传统工具身份保持长时间生成中人物特征稳定不变情感表达能将音频情感映射到面部表情和肢体语言高效性能单GPU支持长时生成多GPU可扩展通过分析音频特征仅需生成关键稀疏帧再智能插值补全既保证效率又确保自然流畅。核心优势优势 说明精准唇形同步 口型与音频毫秒级匹配自然度远超传统工具身份保持 长时间生成中人物特征稳定不变情感表达 能将音频情感映射到面部表情和肢体语言高效性能 单 GPU 支持长时生成多 GPU 可扩展典型应用场景教育领域虚拟讲师制作、课程多语言本地化企业宣传产品解说、多语种企业介绍视频快速制作自媒体创作博主可不出镜完成内容量产影视制作配音替换、角色对话重制使用方式通过官网 (infinitetalk.app) 上传人像素材和音频选择生成参数即可获得高质量的 Talking Video。总结InfiniteTalk AI 重新定义了音频驱动视频生成为内容创作提供了前所未有的自由度让 一张照片开口说话 成为现实且支持无限时长、自然流畅的表达。快速入门1. 创建一个 conda 环境并安装 pytorch、xformersconda create -n multitalk python3.10 conda activate multitalk pip install torch2.4.1 torchvision0.19.1 torchaudio2.4.1 --index-url https://download.pytorch.org/whl/cu121 pip install -U xformers0.0.28 --index-url https://download.pytorch.org/whl/cu1212. Flash-attn installation:pip install misaki[en] pip install ninja pip install psutil pip install packaging pip install wheel pip install flash_attn2.7.4.post13. 其他依赖关系pip install -r requirements.txt conda install -c conda-forge librosa4. FFmeg安装conda install -c conda-forge ffmpeg或者sudo yum install ffmpeg ffmpeg-devel模型下载ModelsDownload LinkNotesWan2.1-I2V-14B-480P HuggingfaceBase modelchinese-wav2vec2-base HuggingfaceAudio encoderMeiGen-InfiniteTalk HuggingfaceOur audio condition weightsDownload models using huggingface-cli:huggingface-cli download Wan-AI/Wan2.1-I2V-14B-480P --local-dir ./weights/Wan2.1-I2V-14B-480P huggingface-cli download TencentGameMate/chinese-wav2vec2-base --local-dir ./weights/chinese-wav2vec2-base huggingface-cli download TencentGameMate/chinese-wav2vec2-base model.safetensors --revision refs/pr/1 --local-dir ./weights/chinese-wav2vec2-base huggingface-cli download MeiGen-AI/InfiniteTalk --local-dir ./weights/InfiniteTalk 快速推理我们的型号兼容480P和720P两种分辨率。Some tipsLip synchronization accuracy:​​ Audio CFG works optimally between 3–5. Increase the audio CFG value for better synchronization. FusionX While it enables faster inference and higher quality, FusionX LoRA exacerbates color shift over 1 minute and reduces ID preservation in videos. V2V generation: Enables unlimited length generation. The model mimics the original videos camera movement, though not identically. Using SDEdit improves camera movement accuracy significantly but introduces color shift and is best suited for short clips. Improvements for long video camera control are planned. I2V generation: Generates good results from a single image for up to 1 minute. Beyond 1 minute, color shifts become more pronounced. One trick for the high-quailty generation beyond 1 min is to copy the image to a video by translating or zooming in the image. Here is a script to convert image to video. Quantization model: If your inference process is killed due to insufficient memory, we suggest using the quantization model, which can help reduce memory usage.一些建议唇同步精度音频CFG的最佳表现为3–5。提高音频CFG值以实现更好的同步。 FusionX虽然它能实现更快的推断和更高质量但FusionX LoRA会加剧1分钟内的色移并降低视频中的识别保护。 V2V生成支持无限长度的生成。模型模仿了原始视频的摄像机运动但并不完全相同。使用SDEdit显著提升了摄像机运动的准确性但引入了色彩偏移更适合短片段。计划改进长距离摄像机控制。 I2V生成单张图像能产生长达1分钟的良好效果。超过1分钟后颜色变化会更明显。对于高质量生成超过1分钟的一个技巧是通过将图像进行翻译或放大将图像复制到视频中。这里有一个脚本可以把图片转换成视频。 量化模型如果您的推理过程因内存不足而终止我们建议使用量化模型这有助于减少内存使用。InfiniteTalk 的使用--mode streaming: long video generation. --mode clip: generate short video with one chunk. --use_teacache: run with TeaCache. --size infinitetalk-480: generate 480P video. --size infinitetalk-720: generate 720P video. --use_apg: run with APG. --teacache_thresh: A coefficient used for TeaCache acceleration —-sample_text_guide_scale When not using LoRA, the optimal value is 5. After applying LoRA, the recommended value is 1. —-sample_audio_guide_scale When not using LoRA, the optimal value is 4. After applying LoRA, the recommended value is 2. —-sample_audio_guide_scale When not using LoRA, the optimal value is 4. After applying LoRA, the recommended value is 2. --max_frame_num: The max frame length of the generated video, the default is 40 seconds(1000 frames).1. 推断1单显卡运行python generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir weights/chinese-wav2vec2-base \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --input_json examples/single_example_image.json \ --size infinitetalk-480 \ --sample_steps 40 \ --mode streaming \ --motion_frame 9 \ --save_file infinitetalk_res2运行720P版本如果你想用720P运行可以设置--size infinitetalk-720python generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir weights/chinese-wav2vec2-base \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --input_json examples/single_example_image.json \ --size infinitetalk-720 \ --sample_steps 40 \ --mode streaming \ --motion_frame 9 \ --save_file infinitetalk_res_720p3显存非常低如果你想用非常低的显存运行可以设置--num_persistent_param_in_dit 0python generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir weights/chinese-wav2vec2-base \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --input_json examples/single_example_image.json \ --size infinitetalk-480 \ --sample_steps 40 \ --num_persistent_param_in_dit 0 \ --mode streaming \ --motion_frame 9 \ --save_file infinitetalk_res_lowvram4多GPU推断GPU_NUM8 torchrun --nproc_per_node$GPU_NUM --standalone generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir weights/chinese-wav2vec2-base \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --dit_fsdp --t5_fsdp \ --ulysses_size$GPU_NUM \ --input_json examples/single_example_image.json \ --size infinitetalk-480 \ --sample_steps 40 \ --mode streaming \ --motion_frame 9 \ --save_file infinitetalk_res_multigpu5多人动画python generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir weights/chinese-wav2vec2-base \ --infinitetalk_dir weights/InfiniteTalk/multi/infinitetalk.safetensors \ --input_json examples/multi_example_image.json \ --size infinitetalk-480 \ --sample_steps 40 \ --num_persistent_param_in_dit 0 \ --mode streaming \ --motion_frame 9 \ --save_file infinitetalk_res_multiperson2. 使用 FusioniX 或 Lightx2v只需 4~8 步FusioniX需要8步Lightx2V只需4步。python generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir weights/chinese-wav2vec2-base \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --lora_dir weights/Wan2.1_I2V_14B_FusionX_LoRA.safetensors \ --input_json examples/single_example_image.json \ --lora_scale 1.0 \ --size infinitetalk-480 \ --sample_text_guide_scale 1.0 \ --sample_audio_guide_scale 2.0 \ --sample_steps 8 \ --mode streaming \ --motion_frame 9 \ --sample_shift 2 \ --num_persistent_param_in_dit 0 \ --save_file infinitetalk_res_lora3. 使用量化模型运行仅支持单显卡运行python generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir weights/chinese-wav2vec2-base \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --input_json examples/single_example_image.json \ --size infinitetalk-480 \ --sample_steps 40 \ --mode streaming \ --quant fp8 \ --quant_dir weights/InfiniteTalk/quant_models/infinitetalk_single_fp8.safetensors \ --motion_frame 9 \ --num_persistent_param_in_dit 0 \ --save_file infinitetalk_res_quant4. 与Gradio一起奔跑python app.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir weights/chinese-wav2vec2-base \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --num_persistent_param_in_dit 0 \ --motion_frame 9或python app.py \--ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir weights/chinese-wav2vec2-base \ --infinitetalk_dir weights/InfiniteTalk/multi/infinitetalk.safetensors \ --num_persistent_param_in_dit 0 \ --motion_frame 9图片预览效果预览https://www.bilibili.com/video/BV1pBqHB4EoW/下载地址https://github.com/MeiGen-AI/InfiniteTalkhttps://pan.quark.cn/s/f425d805167f