Authors: Remya Stephen
Certificate: View Certificate
The term big data refers to data that is diverse, huge, and incredibly fast. Aside from the volume, big data is also complex, making conventional data management tools unable to handle it. There is an explanation of big data in this abstract, as well as various applications of this technique. Big data is precious and an important dogfight for all sectors moment. It\'s used by nearly all associations and has numerous use cases.
Large Quantities of data are stored in an systematized manner on a big data platform. Added up data sets are generally stored on big data platforms, which combine tackle and software tools for data operation.
Multitudinous sophisticated and largely scalable pall data platforms have surfaced to store and reuse the continuously growing volume of data from colorful sources because the data affluence is patient and only anticipated to increase in intensity. Big data platforms are the name given to these kinds of platforms.
A big data platform organizes and stores this volume of data in a way that makes it easy to draw perceptive conclusions. Big data platforms use a combination of tackle and software tools for data operation to aggregate data on an enormous scale, generally onto the pall.
The constant influx of information from colorful sources is becoming more violent, especially with advances in technology. And this is where big data platforms come into the store and dissect the ever- adding a mass of information.
A big data platform is an intertwined computing result that combines multitudinous software systems, tools, and tackle for big data operation.
It's a one- stop armature that solves all the data requirements of a business, anyhow because of the volume and size of the data at hand. Due to their effectiveness in data operation, enterprises are decreasingly espousing big data platforms to gather tons of data and convert it into structured, practicable business receptivity.
Presently, the business is swamped with multitudinous Open source and commercially available big data platforms. They boast different features and capabilities for use in a big data terrain.
A. Characteristics of big data
Big data generally has a multidimensional structure and can be characterised by five.
II. METHODS AND TOOLS USED IN BIG DATA
With the elaboration of technology and the increased multitudes of data flowing in and out of associations daily, there has come a need for faster and more effective ways of assaying similar data. Having piles of data on hand is no longer enough to make effective opinions at the right time. Similar data sets can no longer be fluently anatomized with traditional data operation and analysis ways and architectures. thus, there arises a need for new tools and styles specialized for big data analytics.
In this context, the Big-Data Analytics and Decisions framework is proposed, which incorporates big-data analytics tools and styles into decision-making. In this frame, different big data storage, operation, and processing tools, analytics tools, and visualization and evaluation tools are mapped to the different phases of the decision-making process. Data and analytics recycling, as well as big data analyses that can be applied for knowledge discovery and informed decision-making, are three main areas associated with big data analytics.
A. Big Data Warehouse and governance
The conventional styles of structured data storehouse and reclamation include relational databases, data marts, and data
storages.Although, the big data domain calls for magnetic, nimble, Deep analysis chops, which differ from the aspects of a traditional Enterprise Data Warehouse( EDW) terrain. Consequently, several results, ranging from distributed systems and Massive Pa-rallel Processing( MPP) databases for furnishing high query performance and platform scalability, to non-relational or in- memory databases, have been used for big data.
On the other hadoop is a frame for performing big data analytics which provides trustability, scalability, and manageability by furnishing an perpetration for the MapReduce paradigm. The Hadoop platform consists of two main components: HDFS for storage and MapReduce for analytics. As a redundant and reliable distributed file system, HDFS is optimized for large files, dividing each file into blocks and distributing them across cluster nodes. In addition to this, a replication mechanism ensures the data's availability and reliability even if one of the nodes fails. HDFS nodes can be categorized into two types: Data Nodes and Name Nodes. Across multiple Data Nodes, data is replicated in file blocks, and the Name Node acts as a regulator between the client and the Data Node, directing the client to the Data Node that contains the requested information.
B. Analyzing large amounts of data
There are four essential aspects for efficient big data processing. A key factor is a fast loading speed as disk and network traffic can disrupt query executions during data loading. Another critical requirement is the quickness of query processing, to be able to meet the demands of heavy workloads and on-time requests. Response time is a must for any queries. Consequently, the data placement structure must be able to maintain rapid query processing speeds as the number of queries significantly grows. In addition, efficient utilization of storage space is a critical requirement for effective big data processing. With the fast-growing user activities demanding scalable storage capacity and computing power, it is essential that storage space be managed effectively during processing due to limited disk space availability.
Ultimately, the fourth requirement is robust adaptivity to enormously dynamic workload styles. The underlying machine should be relatively adaptive to unexpected dynamics in data processing, and not precise to sure workload styles.
III. VARIOUS APPLICATION OF BIGDATA
Industries,, academics, and other prominent stakeholders certainly agree that big data is revolutionizing most, if not all, modern industries in recent years. As big data continues to permeate our daily lives, the focus has shifted significantly away from the hype surrounding big data to finding real value in using big data.
A. Big Data in Banking and Securities Industry
Big Data has made banks more efficient in every industry. The use of technology has overcome user difficulties, increased bank revenues, and made insights more transparent and understandable than ever before. From detecting fraud to analyzing and streamlining trade processing to better understanding users, perfecting trade processing, and improving the user experience, Big Data has a wide range of uses.
The purpose of Big Data in banking is that people expect banks to provide realistic and practical solutions. They expect a faster and more streamlined process, personalized advice, accurate, complete and up-to-date information on topics that are informative and relevant, such as applying for a traditional loan to buy a house.
Advanced analytics and the use of cutting-edge technologies such as artificial intelligence and machine learning can leverage data in areas such as personalization and convenience, protection, and the creation of relevant offers. Artificial intelligence and machine learning will play an important role in the future of finance. They will help banks analyze data, predict customer behavior and personalize financial services.
???????B. Big Data in Media and Entertainment Industry
As the current strategy of using gadgets to consume online content becomes the latest trend, enthusiasm for traditional media consumption methods is slowly fading. Big data is winning in this industry due to the enormous amount of data generated. From predicting what audiences want in terms of genres, music and content by age group, to providing insights into customer churn, big data has made media houses' jobs much easier.
Organizations in this industry simultaneously analyze customer and behavioral data to create detailed customer profiles that can be used to:
Spotify, an on-demand music service, uses Hadoop big data analytics to collect data from millions of users around the world and uses the analyzed data to make informed music recommendations to individual users.
Amazon Prime, which aims to provide a superior customer experience by offering a one-stop shop for videos, music, and Kindle books, is also making heavy use of big data.
Another apt example of big data playing a key role in transforming media platforms would be Netflix. Technology not only affects the series Netflix invests in, but also how the series is presented to viewers. Your search and viewing history (including where you paused videos) influences everything from personalized thumbnails to shows you watch on Netflix's popular section.
???????C. Big Data in the Healthcare Sector
The health care sector has access to vast amounts of data, but is plagued by deficiencies in using data to contain rising health care costs and inefficient systems that hinder prompt and better health care overall. Due to the high medical costs, and health ignorance, the health care industry is in crisis. We need a system that collects real-time data from patients and predicts disease risk. Such a system can uses efficient machine learning algorithms and big data analysis technology to enable patients to remotely monitor their health status anytime, anywhere.
In developing countries like India, healthcare is a major concern, including elderly health, maternity, diabetes, asthma, cardiac arrest, and cancer. Due to the high cost of treatment, there is a growing need for big data analytics for healthcare. This cost is much higher and has been increasing for 20
years. only 20% of individuals are wealthy enough to afford such medical care. But the remaining 80% of people who can't even think of going to multiple reputable specialty hospitals for treatment . With this technology, the system could be made intelligent enough to predict the frequency of disease outbreaks and inform patients early about their condition.
Even though, there are some challenges in big data One of the major obstacles to big data analytics in healthcare is that the data is spread across many sources and assimilating these data sources requires the development of new strategies and policies. However, the healthcare industry faces tremendous challenges around data storage, security, visualization, management, reporting, querying, sharing, and many data integrity concerns, to name a few.
Advantage of big data in health care
Big data analytics have the potential to significantly improve the patient experience, including treatment quality, satisfaction, cost savings and timely predictions. With Medical Data Analytics. Here, we could get,
D. Big Data in Education
Big data is increasingly being used in education to improve teaching and learning, improve student achievement, and inform decision-making. Here are some ways to use big data in education.
???????E. Big Data in Transportation
Big data plays a key role in making transportation more perfect and efficient. Whether managing revenue generated, maintaining earned reputation, or pursuing strategic marketing, big data is making an impact in this area. It also helps in planning routes according to user needs, efficiently managing waiting times, and identifying accident-prone areas to increase traffic safety levels.
A perfect example of the use of big data in the transportation industry is Uber. The platform creates and uses a variety of data about vehicles, drivers, locations, and travel by each vehicle, and tests that data for use in predicting demand, supply, and accurate driver locations and fares.
Here are some of the tools listed that big data using in transportation
Big data analytics is employed to improve the efficiency and quality of public transportation systems. By analyzing passenger data, fare collection records, and scheduling information, transportation authorities can optimize routes, adjust service frequency, and enhance overall passenger experience. Tools used in public transportation optimization include:
a. Automated fare Collection Systems: These systems collect data on passenger journeys, boarding and alighting patterns, and payment information, enabling transportation authorities to analyze passenger flows and make service adjustments.
b. Real-time Passenger Information Systems: These systems provide travelers with real-time updates on arrival times, service disruptions, and alternative routes, improving the overall passenger experience.
IV. HOW BIG DATA RELATED WITH AI
Big data and artificial intelligence have a synergistic relationship. AI requires vast amounts of data to learn and improve decision-making processes, and big data analytics leverages AI for better data analysis.
AI can assist users at every stage of the big data cycle, or the process of aggregating, storing, and retrieving different kinds of data from different sources. These include data management, pattern management, context management, decision management, behavior management, goal management and risk management.
AI can identify data types, find possible connections between data sets, and recognize knowledge using natural language processing. It can be used to automate and speed up data preparation tasks such as data model generation to aid in data exploration.
You can learn patterns of common human errors and identify and correct potential information errors. It also learns by observing users interact with analytics programs, quickly discovering unexpected insights from massive data sets. AI can also learn semantic nuances and context-specific nuances to help users better understand numerical data sources. It can also alert users to anomalies and unexpected patterns in data, pro actively monitor events, and identify potential threats using system logs, social networking data, and more.
Here's how big data and AI are interconnected:
Overall, big data provides the necessary raw material for AI systems to learn, adapt, and make intelligent decisions. The abundance of data, along with advancements in AI algorithms, has led to significant breakthroughs in various fields, driving innovation and improving outcomes.
In conclusion, big data has become a powerful tool with widespread applications across various industries. Its ability to process and analyze vast amounts of data has transformed the way businesses operate, governments make decisions, and researchers gain insights. As big data continues to grow in volume and complexity, its applications are expected to expand further, transforming industries and driving innovation. However, it is important to address privacy concerns, ensure data security, and adhere to ethical practices when utilizing big data for various purposes.
 Yasmeen Shaikh, S. R. Biradar and V. K. Parvati, “Role of IoT and Bigdata Analytics in Healthcare for Disease Prediction” 2020 IEEE International Conference on Convergence to Digital World – Quo Vadis (ICCDW 2020)  Anku Jaiswal and Purrushottam Bagale, 3rd “A Survey on Big Data in Financial Sector” 2017 International Conference on Networking and Network Applications  Nii Attoh-Okine “Big Data Challenges in Railway Engineering” 2014 IEEE International Conference on Big Data  Margaret. Rouse, ‘Remote Patient Monitoring (RPM)’, April 2019.[Online].Available:https://searchhealthit.techtarget.com/definition/remote-patient-monitoring-RPM. [Accessed: 13- Jun- 2019]  Md Ileas Pramanik, Raymond Y.K. Lau, Haluk Demirkan, Md.AbulKalam Azad, “Smart health: Bigdata enabled Health paradigmwithin Smart Cities”, An international journal of Expert Systems with Applications 87 (2017) 370–383, ScienceDirect, Elsevier.  Jagadish, H.V. et al. Big Data and Its Technical Challenges.Communications of the ACM, July 2014.  Chen, C.L. P. and Zhang, C. Y . Data-intensive applications, challenges,techniques and technologies; A Survey on Big Data. Information Science, Vol 275, 2014, 314-327.  https://www.qlik.com/us/augmented-analytics/big-data-ai#:~:text=are%20transforming%20BI-,Relationship%20between%20AI%20and%20Big%20Data,AI%20for%20better%20data%20analysis.  https://www.simplilearn.com/tutorials/big-data-tutorial/big-data-applications  https://www.interviewbit.com/blog/applications-of-big-data/
Copyright © 2023 Remya Stephen. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.