Ijraset Journal For Research in Applied Science and Engineering Technology

Abstract

The end-to-end business solution is helpful for any business trying to move their data to cloud for further processing, visu-alizing, monitoring, scaling their business/ business data. Most applications start as small, focused services that solve a single problem and solve it well. But, as a company grows, the needs of its customers multiply. One feature becomes two, which becomes four, then eight and so on. This is manageable, but only to a point. With each additional feature, the com-plexity of the application infrastructure expands. Figuring out how to properly manage the costs of a growing development environment is nearly as difficult as figuring out how to manage the development environment itself. In fact, the two issues are intertwined. All of these problems can hence be solved by using cloud technology like Flexibility in Operations, Reduction in cost, In-crease in Transparency, Enhanced Risk Management and Improved Security. The proposed system is divided into four modules: Ingestion of raw data into the database by creating pipeline, Connecting the database to Machine learning ser-vice, Building the model for accuracy in prediction of the model, Visualizing the dataset into PowerBI.

Introduction

I. INTRODUCTION                                                                    

The major purpose behind this project is to develop an end to end business solution to forecast the sales for a business by using Azure resources, to give them insights & for helping them in supply chain management. This project is developed for any business who finds difficult to deal with bulky reports, improper visualization of sales, gross margin, products sold and are facing difficulty in analyzing data. Their data is ingested on cloud and a predictive model is built which further connected to PowerBI to provide proper visualization and predictions.

A. Aim

The major purpose behind this project is to develop an end to end business solution to forecast the sales for a business by using Azure resources, to give them insights and for helping them in supply chain management.

B. Motivation

The motivation for doing this project was primarily an interest in undertaking a challenging project in an interesting area of research. The opportunity to learn about a new area of computing covered in lectures was appealing.

C. Objectives

To give the business an end to end solution  including ingestion of data from storage to SQL database.

Implementing a predictive model using Azure machine learning to predict total sales.

Visualizing data in Power BI after sales prediction.

To help the business in supply chain optimization.

II. LITERATURE SURVEY

A. Existing System

1. In the existing system they dealt with on premises data which was difficult to manage.
2. The reports got bulky due to many customers added in their database.
3. The system on premises became difficult to manage as they dealt with physical servers.
4. Their prediction couldn’t sync very well with the updates in data.

B. Proposed System

1. We are ingesting their data on Azure Cloud provided by Microsoft to give them a better experience in visualizations by saving their expenses in their complicated system.
2. This system will scale their data automatically by saving a lot of computations and manual work.
3. Our security best practices describe how to ensure that the confidentiality, integrity, and availability of your application aren't compromised by malicious actors.

Table 1 – Pros and Cons of Present System

Thus, though the pros and cons weigh out to be equal. The cons are quite critical and can cause troubled situations in longer run.

Thus, the technology we plan to implement will reduce time, human error, the human effort and will make more accurate future predication thus reducing time, effort and cost while implementing the system.

The following table presents our basic findings from each paper which provides a direction to our idea.

III. SYSTEM ARCHITECTURE

A. Store

1. The raw data of the supermarket is initially stored in blob storage in Azure.
2. Create an empty table in the Azure SQL database.

B. Ingest

1. Using Azure data factory, create copy activity to ingest the data from blob storage to the Azure SQL database.
2. Deploy the pipeline, store in GitHub.

C. Build Model

1. Using Azure Machine learning studio connect the database to the datastore.
2. In the Notebook, code for the predictive model
3. Using the KNN algorithm we are going to build the predictive model.
4. By deploying the model, we will store the REST API endpoint.

D. Serve the model

1. Using the REST API endpoint connect it to the PowerBI to show the visualizations.

IV. PROBLEM SOLUTION AND DATA FLOW

The steps involved int the system operation are :

1. Ingest
2. Prepare
3. Transform & Analyze
4. Publish

a. Initially the raw data is stored in blob storage, which is ingested into the Azure SQL via pipeline, We used the copy activity in the Azure Data Factory.

b. The Copy activity in Azure Data Factory is used to copy data between the different data stores.

c. Azure Data Factory uses the Integration Runtime (IR) as a secure compute infrastructure to run the copy activity across the different network environments and make sure that this activity is performed in the closest possible region to the data store. You can imagine it as a bridge between the copy activity and the linked services.

d. Once the server instance, as well as Azure SQL Database, is in place, the next step is to create Azure Machine Learning Workspace, so that using the same, we can access the machine learning studio.

e. After uploading a dataset to Power BI, a click of a button can be used to invoke this feature that automatically builds many reports based on its analysis of the data, without any human intervention being required.

V. ALGORITHMS

A. Algorithms used in Previous Work

To forecast sales, several types of algorithms have been proposed with neural networks  and auto-regression being the most prominent. This is expected as the problem was, in essence, a time series problem. ARIMA and Long Short Term Memory (LSTM) have yielded much discussion and promising results in the academic literature [4, 5, 6].

B. Current Algorithm

We are using KNN algorithm for the prediction of sales, building a predictive model as it gives the best accuracy.

VI. SCOPE

The project is developed for any business who finds difficult to deal with bulky reports, improper visualization of sales, gross margin, products sold and facing difficulty in analyzing data. Their data is ingested on cloud and a predictive model is built which further connected to power BI which can provide proper visualization and predictions.

VII. UNIQUE FEATURES

1. Gives the business flexibility to use any Azure services at optimal cost.
2. Automatic patch management on your virtual machines frees up time you would otherwise spend managing your infrastructure, allowing you to  focus on improving your app’s core features.
3. The visualizations can give insights automatically, which will help business forecast the sales.
4. The solution on azure ensures that applications can scale dynamically based on traffic or usage activity.

VIII. ACKNOWLEDGEMENTS

This project and the research behind it would not have been possible without the exceptional support of our project guide, Prof. Subhash A. Nalawade ,His enthusiasm, knowledge and attention to detail have been an inspiration and kept our work on track.

We are also grateful for the insightful comments offered by Prof. Sonali Patil – Project Coordinator Information Technology and Dr. L. K. Wadhawa Principal - Dr. D. Y. Patil Institute of Technology for his constant support and motivation. The generosity and expertise of one and all have improved this study in innumerable ways and saved us from many errors.

Conclusion

We have partially implemented the business problem of their bulky reports and improper visualization of data by moving their data to cloud and performed ingestion, built predictive model for forecasting sales of visualizing data to help in the sup-ply chain optimization.

References

[1] https://learn.microsoft.com/en-us/azure/machine-learning/ [2] https://learn.microsoft.com/en-us/azure/app-service/app-service-web-tutorial-dotnet-sqldatabase [3] https://learn.microsoft.com/en-us/azure/data-factory/connector-sql-server?tabs=data- factory [4] https://naadispeaks.wordpress.com/2021/01/29/connecting-azure-sql-server-with- azure-machine- learning/ [5] https://learn.microsoft.com/en-us/azure/data-factory/tutorial-copy-data-dot-net

Copyright

Copyright © 2023 Prof. S. A. Nalawade, Rashi Singh, Amisha Jha. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.