Abstract
The implementation of a set of measures aimed to regular monitoring and analyzing the activity of users of the National Research Computer Network of Russia in the inter-network interaction, evaluation of the level of its involvement in joint research projects, the intensity of using the technological infrastructure of Russian and the world’s national research and education networks are discussed in the paper. Information on the developed methods, hardware and software solutions based on NetFlow data and provided collecting, analysis and visualization of network traffic is given; some results of work in identifying and visualizing of data exchange and the main directions of network connectivity are presented and discussed.
Similar content being viewed by others
REFERENCES
P. Haag, Nfdump—NetFlow Processing Tools. https://github.com/phaag/nfdump. Accessed May 14, 2020.
V. T. Guimarães, C. M. D. S. Freitas, R. Sadre, L. M. R. Tarouco, and L. Z. Granville, ‘‘A survey on information visualization for network and service management,’’ IEEE Comm. Surv. Tutor. 18, 285–323 (2016). https://doi.org/10.1109/COMST.2015.2450538
C. Allocchio, L. Balint, V. Berkhout, J. Bersee, Y. Izhvanov, et al., A History of International Research Networking: The People who Made it Happen (Wiley-VCH, New York, 2010).
J. Ryan, A history of the Internet and the Digital Future (Breaktion Books, London, 2015).
A. G. Abramov and A. V. Evseev, ‘‘RUNNet: Infrastructural and service basis of the national research and education network of the Russian Federation,’’ in Proceedings of the 8th International Conference on Distributed Computing and Grid-technologies in Science and Education GRID 2018, Sept. 10–14, 2018 (Dubna, 2018), pp. 52–57. Urn:nbn:de:0074-2267-5. http://ceur-ws.org/Vol-2267/52-57-paper-8.pdf.
A. G. Abramov and A. V. Evseev, ‘‘Network RUNNet: Towards the state-of-the-art challenges in the field of telecommunications in science and education,’’ Inform. Educat. Sci. 4 (40), 3–15 (2018).
A. G. Abramov and A. V. Evseev, ‘‘Conceptual aspects of creating a new generation national research computer network in the Russian Federation,’’ Inform. Technol. 25, 724–733 (2019).
G. I. Savin, B. M. Shabanov, A. V. Baranov, A. P. Ovsyannikov, and A. A. Gonchar, ‘‘On the use of the federal research telecommunications infrastructure for supercomputer computations,’’ Vestn. YuUrGU, Ser.: Vychisl. Mat. Inform. 9 (1), 20–35 (2020). https://doi.org/10.14529/cmse200102
Cisco IOS NetFlow. https://www.cisco.com/c/en/us/products/ios-nx-os-software/ios-netflow/. Accessed May 14, 2020.
R. Bejtlich, The Practice of Network Security Monitoring: Understanding Incident Detection and Response (No Starch Press, San Francisco, 2013).
C. Sanders and J. Smith, Applied Network Security Monitoring: Collection, Detection, and Analysis (Syngress, Waltham, MA, USA, 2014).
R. Hofstede, P. Čeleda, B. Trammell, I. Drago, R. Sadre, A. Sperotto, and A. Pras, ‘‘Flow monitoring explained: From packet capture to data analysis with NetFlow and IPFIX,’’ IEEE Comm. Surv. Tutor. 16, 2037–2064 (2014). https://doi.org/10.1109/COMST.2014.2321898
I. Ghafir, J. Svoboda, and V. Prenosil, ‘‘A survey on network security monitoring implementations,’’ in Proceedings of the 2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), Aug. 22–24, 2016, Vienna, Austria (2016), pp. 77–82. https://doi.org/10.1109/W-FiCloud.2016.30
Z. Tian, ‘‘Management of large scale NetFlow data by distributed systems’’ Master’s Thesis (Norwegian Univ. Sci. Technol., Trondheim, 2016). http://hdl.handle.net/11250/2410246. Accessed May 14, 2020.
M. Čermák, D. Tovarňák, M. Laštovička, and P. Čeleda, ‘‘A performance benchmark for NetFlow data analysis on distributed stream processing systems,’’ in Proceedings of the IEEE/IFIP Network Operations and Management Symposium (NOMS), Apr. 25–29, 2016, Istanbul, Turkey (2016), pp. 919–924. https://doi.org/10.1109/NOMS.2016.7502926
A. Raulot and S. Zaidi, Large-Scale NetFlow Information Management (University of Amsterdam, Amsterdam, 2018). https://delaat.net/rp/2017-2018/p83/report.pdf. Accessed May 14, 2020.
F. Fischer, F. Mansmann, D. A. Keim, S. Pietzko, and M. Waldvogel, ‘‘Large-scale network monitoring for visual analysis of attacks,’’ in Proceedings of the 5th International Workshop on Visualization for Computer Security (VizSec 2008), Sept. 15, 2008, Cambridge, USA (2008), pp. 111–118. https://doi.org/10.1007/978-3-540-85933-8_11
B. Li, J. Springer, G. Bebis, and M. H. Gunes, ‘‘A survey of network flow applications,’’ J. Network Comput. Appl. 36, 567–581 (2013).
A. I. Getman, E. F. Evstropov, and Yu. V. Markin, ’’Real-time network traffic analysis: A review of applied tasks, approaches and solutions,’’ Preprint ISP RAS No. 28 (ISP RAS, Moscow, 2015). https://www.ispras.ru/preprints/docs/prep_28_2015.pdf. Accessed May 14, 2020.
NfSen—Netflow Sensor: A graphical web based front end for the nfdump netflow tools. http://nfsen.sourceforge.net. Accessed May 14, 2020.
K. Lakkaraju, W. Yurci, and A. J. Lee, ‘‘NVisionIP: Netflow visualizations of system state for security situational awareness,’’ in Proceedings of the 2004 ACM Workshop on Visualization and Data Mining for Computer Security (VizSec/DMSec 2004), Oct. 29, 2004, Washington DC, USA (2004), pp. 65–72.
P. Minařík and T. Dymáček, ‘‘NetFlow data visualization based on graphs,’’ in Proceedings of the 5th International Workshop on Visualization for Computer Security (VizSec 2008), Sept. 15, 2008, Cambridge, USA (2008), pp. 144–151. https://doi.org/10.1007/978-3-540-85933-8_14
A. G. Abramov and A. V. Evseev, ‘‘Monitoring of user activity of the Russian research and education network RUNNet in inter-network interaction: Methods, tools and results,’’ Inform. Educat. Sci. 3 (39), 34–49 (2018).
Cytoscape.js—Graph Theory (network) Library for Visualisation and Analysis. https://js.cytoscape.org. Accessed May 14, 2020.
ACKNOWLEDGMENTS
The author is grateful to the employees of the SPb branch of JSCC RAS V.A. Porkhachev and I.V. Vasilyev, who took an active part in the development and implementation of the system and applied software solutions.
Funding
The paper was prepared within the framework of the state assignment of Federal State Institution ‘‘Scientific Research Institute for System Analysis of the Russian Academy of Sciences’’ on the topic ‘‘Research and development of methods and means of organizing high-performance computing, creating, processing, storing and distributing big data and digital content in distributed information and computing environments’’ (project no. 0065-2019-0014).
Author information
Authors and Affiliations
Corresponding author
Additional information
(Submitted by A. M. Elizarov)
Rights and permissions
About this article
Cite this article
Abramov, A.G. Collection, Analysis and Interactive Visualization of NetFlow Data: Experience with Big Data on the Base of the National Research Computer Network of Russia. Lobachevskii J Math 41, 2525–2534 (2020). https://doi.org/10.1134/S1995080220120021
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S1995080220120021