The influence of current results in a event­oriented data collection system

DOI: 10.31673/2412-9070.2024.031822

Abstract

The article discusses synchronous and asynchronous procedures in the implementation of the microservices management algorithm in an adaptive system for processing large data flows when collecting information on the main event-oriented approach in the implementation of the architecture of a software system that processes information in real-time. This approach is important when processing large volumes of data from heterogeneous information sources, especially when the task is to minimize the total processing time of large data streams and reduce the total number of calls to data sources. The proposed approach makes it possible, based on its adaptability, to manage the selection of the composition and the number of calls of microservices to the sources, by the events that occur during the collection of information. Thus, it is possible to determine the choice of sources based on the assessment of the effectiveness of obtaining relevant information from them. This is especially important when processing large data flows from heterogeneous information sources when the task is to minimize the total time of collection and processing of large data flows. In turn, this poses the task of minimizing the number of requests to information sources to obtain a sufficient number of data units that are relevant to the search query. The creation of effective big data processing systems requires constant development of approaches to the architecture of building software applications. The event-oriented microservice architecture of the system makes it possible to adapt the operation of the system to the loads on individual microservices and the efficiency of their work by forming and responding to relevant events based on the analysis of relevant events that occur during the collection and initial processing of the received data. Depending on the specific task, it is possible to use both a synchronous and an asynchronous microservice management algorithm. The article provides an analysis of the effectiveness of obtaining relevant data depending on the degree of consideration in the evaluation of results and the formation of events, both the depth of the previous history of the results of requests and the size of their quantitative impact. The use of event-oriented microservice architecture can be especially effective when developing various information and analytical systems that need, according to user requests, to access various sources of information, analyze their data for relevance according to the request, and process large data streams.

Keywords: microservices; adaptation; event-driven architecture; big data.

References
1. Richardson C. Microservices. Development and refactoring patterns. S.-P.: Peter, 2019. 544 p.
2. Ford N., Parsons R., Kua P. Building Evolutionary Architectures: Support Constant Change. O'Reilly Media, 2017. 332 p.
3. Belnar A. Building Event-Driven Microservices: Leveraging Organizational Data at Scale. USA, O'Reilly Media, 2020, 324 p.
4. Improving the Efficiency of Typical Scenarios of Analytical Activities / O. V. Koval, V. O. Kuzminykh, I. I. Husyeva [et al.] // CEUR Workshop Proceedings, ISSN: 1613-0073. 2021. Vol. 3241. P. 123–132.
5. Wolff E. Microservices, Flexible Software Architecture. Boston: Addison-Wesley, 2016. 436 p.
6. Ashley D. Bootstrapping Microservices with Docker, Kubernetes, and Terraform: A project-based guide. Shelter: Manning Publications Co., 2021. 442 p.
7. Pethuru Raj, Jeeva S. Chelladhurai, Vinod Singh. Learning Docker. Packt Publishing, 2015. 240 p.
8. Hugo Filipe Oliveira Rocha. Practical Event-Driven Microservices Architecture: Building Sustainable and Highly Scalable Event-Driven Microservices. Ermesinde: Apress, 2021. 472 p.
9. Adaptive Software System for International Activity Level Assessment / O. V. Koval, V. O. Kuzminykh, I. I. Husyeva [et al.] // CEUR Workshop Proceedings. 2022. Vol. 3503. P. 52–61.