Forecasting and Visualization of RSS Feeds Trends and Historical News Data Using Clustering and Prophet

Abstract

The exponential growth of digital news media has led to an overwhelming influx of unstructured information, posing challenges in extracting actionable insights and identifying emerging trends. This paper presents an Advanced News Analytics System designed to automate the analysis, clustering, and forecasting of news data aggregated from real-time RSS feeds and historical archives. Leveraging machine learning and time-series forecasting techniques, the system employs Feedparser for data acquisition, Scikit-learn for clustering via K-Means and DBSCAN, and Facebook Prophet for trend forecasting. The platform, implemented using Streamlit, provides an intuitive dashboard that visualizes historical topic dynamics, real-time distributions, and future trend predictions. Unlike conventional aggregators, this system transcends basic content display by enabling automated topic modeling, trend evolution tracking, and predictive analytics, thereby empowering users with data-driven insights into media trends. The study underscores the system’s potential to enhance strategic decision-making through scalable, unbiased, and interpretable news trend forecasting.

Introduction

The paper presents an Advanced News Analytics System designed to process, analyze, and forecast trends in digital news media. Traditional news aggregators focus on collection but offer limited analytical insight, whereas this system leverages machine learning, neural topic modeling (BERTopic), and time-series forecasting (Facebook Prophet) to provide deeper understanding of news evolution.

The system architecture is modular and scalable, built with FastAPI, Streamlit, and Chart.js, and integrates real-time RSS feeds with historical news archives. Data is preprocessed using Pandas, NumPy, and NLP techniques, then clustered via K-Means and DBSCAN, stored in a centralized database, and used for predictive modeling.

Key modules include:

1. RSS and Historical Data Collection – Aggregates real-time and past news, ensuring data integrity and continuity.

2. Data Processing and Clustering – Cleans and structures text, identifies thematic clusters, and uncovers emerging topics.

3. Centralized Data Storage – Maintains structured, reliable, and scalable storage for all processed and forecasted data.

4. Visualization and Trend Forecasting – Provides interactive dashboards with charts and predictions, enabling users to monitor topic evolution, identify trends, and make data-driven decisions.

Conclusion

The News Analytics and Forecasting System presents a comprehensive framework for automating the collection, processing, and analysis of news data from multiple sources. By integrating RSS feeds and historical datasets, the system ensures continuous and reliable access to real-time and archival information. This combination strengthens the analytical foundation by enabling both immediate insights and long-term trend evaluation, offering users a holistic perspective on news patterns over time. The inclusion of data processing, clustering, and centralized storage allows for efficient handling of vast textual information. Through the application of intelligent clustering algorithms, the system identifies topic-based groupings that enhance content organization and retrieval. Furthermore, the structured database design ensures scalability, security, and seamless integration with analytical tools for advanced operations like anomaly detection and temporal trend monitoring. Finally, the visualization and forecasting modules bridge the gap between raw data and actionable insights. Using advanced graphical dashboards and predictive modeling techniques such as Prophet or ARIMA, users can visualize historical trends and forecast potential future developments. This end-to-end workflow transforms unstructured news data into an intelligent, data-driven analytical system capable of supporting informed decision-making, proactive media monitoring, and enhanced understanding of evolving news dynamics.

References

[1] Facebook Research. (2023). Prophet: Forecasting at Scale. Retrieved from: https://facebook.github.io/prophet/ [2] Grootendorst, M. (2023). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794. Retrieved from: https://arxiv.org/abs/2203.05794. [3] Muennighoff, N., et al. (2023). MTEB: Massive Text Embedding Benchmark. arXiv preprint arXiv:2305.13823. Retrieved from: https://arxiv.org/abs/2305.13823 [4] Pedregosa, F., et al. (2023). *Scikit-learn: Machine Learning in Python (Version 1.3).* Journal of Machine Learning Research, 24(1), 1–8. Retrieved from: https://jmlr.org/papers/v24/23-0348.html. [5] PostgreSQL Global Development Group. (2023). PostgreSQL 15.2 Documentation. Retrieved from: https://www.postgresql.org/docs/15/index.html [6] Redis Ltd. (2023). Redis Documentation (Version 7.0). Retrieved from: https://redis.io/documentation/. [7] Celery Project. (2023). Celery: Distributed Task Queue. Documentation. Retrieved from: https://docs.celeryq.dev/en/stable/ [8] FastAPI. (2023). FastAPI Documentation (Version 0.100.0). Retrieved from: https://fastapi.tiangolo.com/ [9] Meta Open Source. (2023). React Documentation. Retrieved from: https://react.dev/ [10] Chart.js Community. (2023). Chart.js Documentation (Version 4.0). Retrieved from: https://www.chartjs.org/docs/latest/ [11] Facebook Research. (2023). Prophet: Forecasting at Scale. Retrieved from: https://facebook.github.io/prophet/ [12] Grootendorst, M. (2023). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794. Retrieved from: https://arxiv.org/abs/2203.05794 [13] Muennighoff, N., et al. (2023). MTEB: Massive Text Embedding Benchmark. arXiv preprint arXiv:2305.13823. Retrieved from: https://arxiv.org/abs/2305.13823 [14] Pedregosa, F., et al. (2023). *Scikit-learn: Machine Learning in Python (Version 1.3).* Journal of Machine Learning Research, 24(1), 1–8. Retrieved from: https://jmlr.org/papers/v24/23-0348.html [15] PostgreSQL Global Development Group. (2023). PostgreSQL 15.2 Documentation. Retrieved from: https://www.postgresql.org/docs/15/index.html [16] Redis Ltd. (2023). Redis Documentation (Version 7.0). Retrieved from: https://redis.io/documentation/ [17] Celery Project. (2023). Celery: Distributed Task Queue. Documentation. Retrieved from: https://docs.celeryq.dev/en/stable/ [18] FastAPI. (2023). FastAPI Documentation (Version 0.100.0). Retrieved from: https://fastapi.tiangolo.com/ [19] Meta Open Source. (2023). React Documentation. Retrieved from: https://react.dev/ [20] Chart.js Community. (2023). Chart.js Documentation (Version 4.0). Retrieved from: https://www.chartjs.org/docs/latest/

Copyright

Copyright © 2025 S. Venkata Laksmi, N. U. Ajita, T. S. Dharshini. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.