Text-To-Animation: Generating 3D Animation Using Textual Descriptions

Abstract

The quick progress in generative AI Opened new doors in 3D animation by adding allowing automated animation generation from text description. This paper presents Text to Animation (TTA), a new system that fine-tunes the DeepSeek-Coder-5.7BMQA-Base model to convert natural language inputs into Blender Python Scripts. By incorporating Chainlit, the system provides real-time script generation and execution inside Blender with minimal human effort and improved user feedback. For performance rendering, animations are rendered on the cloud, maximizing computational effectiveness. The AI-based solution streamlines the animation pipeline for artists, teachers and creators by automating Blender scripting, hence reducing the technical barrier to producing high-quality 3D animations. It opens 3D content creation to a broader audience by minimizing the necessity of advanced technical skills. The model is fine-tuned with the DeepSeek-Coder-7.7BMQA-Base model with LoRA (Low-Rank Adaptation) for optimizing performance on tasks of generating Blender scripts. Possible users vary from training modules and educational simulation to gaming and cinematic previsualization. This article emphasizes how large language models transform creative industries by closing the gap between AI and 3D animation.Future research will concentrate on enhancing the model’s comprehension of intricate motion dynamics and user interaction during the animation process.

Introduction

The Text-to-Animation (TTA) system is an AI-driven framework designed to simplify the creation of 3D animations in Blender by converting natural language descriptions into executable Python scripts. This approach democratizes animation production, making it accessible to educators, artists, and content creators without extensive programming expertise.

???? System Overview

The TTA system leverages the DeepSeek-Coder-5.7B-MQA language model, fine-tuned using Low-Rank Adaptation (LoRA) techniques to specialize in Blender's Python API. Users input natural language prompts, such as "Create a rotating cube with a bouncing motion," and the system generates corresponding Blender Python scripts. These scripts are executed in real-time within Blender's console via Chainlit integration, allowing for immediate preview and interaction. Cloud-based rendering ensures efficient processing without taxing local hardware resources.

???? Methodology

The TTA system comprises four primary modules:

1. Data Preparation: Curates and preprocesses a dataset of Blender Python scripts, focusing on procedural logic and animation control.

2. Model Training: Fine-tunes the DeepSeek-Coder-5.7B-MQA model on the prepared dataset using LoRA for efficient adaptation to Blender-specific tasks.

3. Script Generation: Transforms user prompts into Blender Python scripts through the fine-tuned model, facilitated by Chainlit for real-time interaction.

4. Rendering: Executes the generated scripts within Blender and utilizes cloud-based infrastructure for rendering, ensuring scalability and performance.

?? Technical Architecture

• Multi-Query Attention (MQA): Enhances inference efficiency by reducing computational overhead while maintaining model expressiveness.

• SWISH-GLU Feed-Forward Network (FFN): Improves the model's capacity to learn intricate patterns in Blender scripting operations.

• Cloud Rendering: Integrates with platforms like Google Colab to offload rendering tasks, ensuring high-performance output.

???? Results and Applications

The TTA system enables the generation of dynamic 3D animations from textual descriptions, facilitating the creation of educational content, medical visualizations, and interactive simulations. Its integration with Chainlit allows for an intuitive user experience, while cloud rendering supports complex tasks without local hardware constraints.

???? Limitations

• Script Accuracy: The quality of generated scripts depends on the clarity and specificity of user prompts.

• Animation Complexity: Current capabilities are limited to basic 3D transformations and animations; advanced physics-based simulations are not yet supported.

• Rendering Speed: Rendering performance is contingent on the computational resources of the cloud platform utilized.

???? Future Directions

• Enhanced Animation Capabilities: Support for advanced motion dynamics, character rigging, and physics simulations.

• Domain-Specific Optimization: Fine-tuning the model with specialized datasets to improve script accuracy for various industries.

• Real-Time Rendering: Integration with cloud-based GPU rendering for faster animation previews within the Chainlit interface.

• Multimodal AI Approaches: Incorporation of image-based guidance to enhance animation precision and realism.

Conclusion

The envisioned Text-To-Animation System showcases the generative power of AI by turning text descriptions into fully rendered 3D animation. Through the fine -tuning of the DeepSeek-Coder-5.7bmqa-base model on Blender script data, the system streamlines the creation, execution, and rendering of Blender scripts. This streamlining greatly minimizes the complexity and time invested in animation creation, rendering it a useful technology for creative and technical applications. The incorporation of Chainlit as an interactive interface enables real - time preview of scripts, opening the system to users with little or no background in Blender scripting. This friendly interface democratizes animation production, making it possible for anyone to turn abstract ideas into rich visual representations with ease. Such a feature is especially powerful in learning environments such as medical education, where interactive 3D animations can enhance conceptual understanding and learning outcomes. Additionally, using Google Colab for GPU rendering makes it scalable and cost-effective without the requirement for Expensive local hardware. This is thus possible for educational institution, researchers & independent creators. Lastly, the Text-to-Animation System is a testament to the capability of generative AI to animation operations. Its capacity to automate animation design from text inputs has profound implications for new frontiers in education, research & creative sectors, where interactive visualizations are crucial to successful communication comprehension.

References

[1] F. Zheng, Y Zhu, \"Exploring the Fusion of Animation and Computer Vision for Enhanced Realism in Virtual Character Interaction,\" IEEE Access, vol. 12, pp. 194816– 194828, 2024, doi: 10.1109/ACCESS.2024.3519292.Available:https://doi.org/10.1109/ACCESS.2024.3519292 [2] N. Bouali and V. Cavalli-Sforza, \"A Review of Text-to-Animation Systems,\" IEEE Access, vol. 11, pp. 86071– 86087, 2023, doi: 10.1109/ACCESS.2023.3304903. Available: https://doi.org/10.1109/ACCESS.2023.3304903 [3] Blender Foundation,BlenderPython API Documentation, Blender 4.3, 2024. Available: https://docs.blender.org/api/current. [4] J. Jonathan Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, \"Video Diffusion Models,\" arXiv preprint, arXiv:2204.03458, 2022.Available:https://arxiv.org/abs/2204.03458 [5] Saharia, C., Chan,W., Saxena,S., Li, L., Whang, J.,Denton, E., SeyedGhasemipourS. K.,Ayan, B. K.,Mahdavi, S. S., Gontijo Lopes, R., Salimans, T., Ho, J.,Fleet, D. J., & Norouzi, M. (2022). Imagen: PhotorealisticText –to -ImageDiffusion Models with LargePretraine dModels. Available:https://imagen.research.google/ [6] U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni, D. Parikh, S. Gupta, and Y. Taigman, \"Make-A-Video: Text-to-Video GenerationwithoutText-VideoData,\"arXivpreprint,arXiv:2209.14792,2022.Available:https://arxiv.org/abs/2209.14792 [7] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, \"High-Resolution Image Synthesis with Latent Diffusion Models,\" arXiv preprint, arXiv:2112.10752, 2022. Available: https://arxiv.org/abs/2112.10752 [8] Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y. Yang, W. Hong, X. Zhang, G. Feng, D. Yin, Y. Zhang, W. Wang, Y. Cheng, B. Xu, X. Gu, Y. Dong, and J. Tang, \"CogVideoX: Text-to-Video Diffusion Models with AnExpert Transformer,\" arXiv preprint, arXiv:2408.06072,2025Available: https://arxiv.org/abs/2408.06072 [9] Z. Liu, Y. Meng, H. Ouyang, Y. Yu, B. Zhao, D. Cohen-Or, and H. Qu, \"Dynamic Typography: Bringing Text to LifeviaVideoDiffusionPrior,\"arXivpreprint,arXiv:2404.11614,2024.Available:https://arxiv.org/abs/2404.11614 [10] OpenAI.(2023).SORA:Self-Optimizing Reinforcement Agent. OpenAI. https://openai.com/ [11] M. Liu, Y. Liu, G. Krishnan, K. S. Bayer, and B. Zhou, \"T2M-X: Learning Expressive Text-to-Motion Generationfrom Partially Annotated Data,\" arXiv preprint,arXiv:2409.13251,2024.Available:https://arxiv.org/abs/2409.13251 [12] P. Goel, K.-C. Wang, C. K. Liu, and K. Fatahalian, “Iterative Motion Editing with Natural Language,” inProceedings of the ACM SIGGRAPH ’24 Conference onComputer Graphics and Interactive Techniques, Jul. 2024,pp.1–9.Available:https://doi.org/10.1145/3641519.3657447

Copyright

Copyright © 2025 Dr. A. Kousalya, Dhivya R, Kalaivani P, Revathi R, Dharani V. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.