Challenges and prospects in big data analytics: a comprehensive review of developments, hurdles, and future research directions

Big data and business analytics are trends that are positively affecting the business world. This comprehensive review article explores the shifting paradigms and dynamic trends within Big Data Technology (BDT), predominantly for last 5 years, based on an extensive literature review and comparative analysis methodology. It elucidates the transformative influence of big data analytics (BDA) in various sectors, emphasizing the rapid ascendance of cloud computing, Artificial Intelligence (AI) integration, and development of sophisticated analytics tools. The review leverages a wealth of academic literature and market research to underscore the predicted expansion of the big data market. This projected growth indicates the widespread adoption of BDT across industries, with healthcare becoming a significant consumer, motivated by the demand for personalized medicine and improved patient care. The review then navigates emerging trends such as open data usage and ethical concerns surrounding big data, indicating the increasing necessity for stringent guidelines for data use and robust individual data control mechanisms. This is derived from a methodical analysis of recent scholarly articles and industry reports. The article also scrutinizes the evolving definition of "big data" through comparative study of the 3V model and the expanded 7V model in various literature sources, reflecting the evolving nature of data and the unique challenges introduced by modern big data analytics. The review also outlines the challenges for successful implementation of big data projects and highlights the current open research directions of big data analytics. The reviewed areas of big data suggest that good management and manipulation of the large data sets using the techniques and tools of big data can deliver actionable insights that create business values.

1. Berisha, B., Mëziu, E., & Shabani, I. (2022). Big data analytics in Cloud computing: an overview. Journal of Cloud Computing, 11(1), 24.

2. Davenport, T.H., & Ronanki, R. (2021). Artificial Intelligence for the real world (2018). Harvard Business Review.

3. Mannering, F., Bhat, C.R., Shankar, V., & Abdel-Aty, M. (2020). Big data, traditional data and the tradeoffs between prediction and causality in highway-safety analysis. Analytic methods in accident research, 25, 100113.

4. Big Data Market. Online source: https://www.marketdataforecast.com/market-reports/big-datamarket

5. Himanen, L., Geurts, A., Foster, A. S., & Rinke, P. (2019). Data‐driven materials science: status, challenges, and perspectives. Advanced Science, 6(21), 1900808.

6. Chen, W., & Quan-Haase, A. (2020). Big data ethics and politics: Toward new understandings. Social Science Computer Review, 38(1), 3-9.

7. Berisha, B., Mëziu, E. & Shabani, I. Big data analytics in Cloud computing: an overview. J Cloud Comp 11, 24 (2022). https://doi.org/10.1186/s13677-022-00301-w

8. González García, C., & Álvarez-Fernández, E. (2022). What Is (Not) Big Data Based on Its 7Vs Challenges: A Survey. Big Data and Cognitive Computing, 6(4), 158. https://doi.org/10.3390/bdcc6040158

9. Ajah, I. A., & Nweke, H. F. (2019). Big Data and Business Analytics: Trends, Platforms, Success Factors and Applications. Big Data and Cognitive Computing, 3(2), 32. https://doi.org/10.3390/bdcc3020032

10. Lee, I., & Mangalaraj, G. (2022). Big Data Analytics in Supply Chain Management: A Systematic Literature Review and Research Directions. Big Data and Cognitive Computing, 6(1), 17. https://doi.org/10.3390/bdcc6010017

11. Borges do Nascimento I., Marcolino M., Abdulazeem H., Weerasekara I., Azzopardi-Muscat N., Gonçalves M., Novillo-Ortiz D. Impact of Big Data Analytics on People’s Health: Overview of Systematic Reviews and Recommendations for Future Studies J Med Internet Res 2021;23(4):e27275 URL: https://www.jmir.org/2021/4/e27275 DOI: 10.2196/27275

12. Seyedan, M., Mafakheri, F. Predictive big data analytics for supply chain demand forecasting: methods, applications, and research opportunities. J Big Data 7, 53 (2020). https://doi.org/10.1186/s40537-020-00329-2

13. What is Prescriptive Analytics? Online source: https://www.talend.com/resources/what-isprescriptive-analytics/

14. Bhattarai, B.P., Paudyal, S., Luo, Y., Mohanpurkar, M., Cheung, K., Tonkoski, R., Hovsapian, R., Myers, K.S., Zhang, R., Zhao, P., Manic, M., Zhang, S. and Zhang, X. (2019), Big data analytics in smart grids: state-of-the-art, challenges, opportunities, and future directions. IET Smart Grid, 2: 141-154. https://doi.org/10.1049/iet-stg.2018.0261

15. Tawalbeh, L. A., Muheidat, F., Tawalbeh, M., & Quwaider, M. (2020). IoT Privacy and security: Challenges and solutions. Applied Sciences, 10(12), 4102

16. Ferraris, A., Mazzoleni, A., Devalle, A., & Couturier, J. (2019). Big data analytics capabilities and knowledge management: impact on firm performance. Management Decision, 57(8), 1923-1936

17. Amani, M., Ghorbanian, A., Ahmadi, S.A., Kakooei, M., Moghimi, A., Mirmazloumi, S. M., ... & Brisco, B. (2020). Google earth engine cloud computing platform for remote sensing big data applications: A comprehensive review. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, 5326-5350.

18. Mohammadpoor, M., & Torabi, F. (2020). Big Data analytics in oil and gas industry: An emerging trend. Petroleum, 6(4), 321-328.

19. Jabbar, A., Akhtar, P., & Dani, S. (2020). Real-time big data processing for instantaneous marketing decisions: A problematization approach. Industrial Marketing Management, 90, 558-569.

20. Završnik, A. (2021). Algorithmic justice: Algorithms and big data in criminal justice settings. European Journal of criminology, 18(5), 623-642.

21. Amazon Redshift – The New AWS Data Warehouse by Jeff Barr. Online source: https://aws.amazon.com/ru/blogs/aws/amazon-redshift-the-new-aws-data-warehouse/