Computational experiments in Computer Science: A bibliometric study
DOI:
https://doi.org/10.56294/dm2025188Keywords:
computational experiment, computer science, operations research, optimization, bibliometric analysisAbstract
Introduction: Computational Experiments are crucial in various fields, including biological sciences, engineering, social sciences, etc., and are a powerful tool for understanding complex systems, optimizing processes, and driving innovation. Their importance lies in their ability to integrate with experimental methods, facilitate simulation-based learning, and provide cost-effective, scalable, and flexible solutions for analyzing complex systems. The purpose of this study is to make a bibliometric analysis of the research related to Computational Experiments in Computer Science. Methods: This bibliometric analysis was performed using information from 2013 and 2024 from the Scopus and Web of Science databases, with published articles This bibliometric study followed the guidelines proposed in the publication “How to conduct a bibliometric analysis: An overview and guidelines” by the author Gonthu N. To answer the research questions, the number of articles per year, number of articles per country, number of articles per subject area, list of main journals, and citation analysis were analyzed. Results: The results show that Scopus has more publications on the subject, China is the country that publishes more on the subject, Mathematics is the predominant subject area, finally, a co-occurrence analysis was performed where a total of 27 clusters were found in Scopus and 10 clusters in WoS. From this, the 10 most relevant keywords in each of the databases were identified. Conclusions: This review can be a basis in order that researchers to have a starting point for the current state of publications on Computational Experiments for future research.
References
1. Yilmazlar IO, Kurz ME. Adaptive local search algorithm for solving car sequencing problem. J Manuf Syst. 2023;68:635 – 643.
2. Barr RS, Golden BL, Kelly JP, Resende MGC, Stewart WR. Designing and reporting on computational experiments with heuristic methods. J heuristics. 1995;1:9–32.
3. Landeta-López P, Guevara-Vega C. Computational Experiments in Computer Science Research: A Literature Survey. IEEE Access. 2024;12:132254–70.
4. Santner TJ, Williams BJ, Notz WI, Williams BJ. The design and analysis of computer experiments. Vol. 1. Springer; 2003.
5. Jones B, Johnson RT. Design and analysis for the gaussian process model. Qual Reliab Eng Int. 2009;25(5):515 – 524.
6. Law AM, Kelton WD, Kelton WD. Simulation modeling and analysis. Vol. 3. Mcgraw-hill New York; 2007.
7. Gönül-Sezer ED, Ocak Z. Comparison of system dynamics and discrete event simulation approaches. Adv Intell Syst Comput. 2016;442:69 – 81.
8. Kleijnen JPC. Design and analysis of simulation experiments. In: Springer Proceedings in Mathematics and Statistics. 2018. p. 3 – 22.
9. Craig PS, Goldstein M, Rougier JC, Seheult AH. Bayesian forecasting for complex systems using computer simulators. J Am Stat Assoc. 2001;96(454):717 – 729.
10. Cavendish JC. A framework for validation of computer models. In: SIAM Proceedings Series [Internet]. 2005. p. 87 – 99. Available from: https://cutt.ly/pw0LGnt7
11. Bayarri MJ, Paulo R, Berger JO, Sacks J, Cafeo JA, Cavendish J, et al. A framework for validation of computer models. Technometrics. 2007;49(2):138 – 154.
12. Guevara-Vega C, Bernárdez B, Durán A, Quiña-Mera A, Cruz M, Ruiz-Cortés A. Empirical Strategies in Software Engineering Research: A Literature Survey. In: 2021 Second International Conference on Information Systems and Software Technologies (ICI2ST). 2021. p. 120–7.
13. Črepinšek M, Liu S-H, Mernik M. Replication and comparison of computational experiments in applied evolutionary computing: Common pitfalls and guidelines to avoid them. Appl Soft Comput J. 2014;19:161 – 170.
14. Beaulieu-Jones BK, Greene CS. Reproducibility of computational workflows is automated using continuous analysis. Nat Biotechnol. 2017;35(4):342 – 346.
15. Cruz M, Bernárdez B, Durán A, Galindo JA, Ruiz-Cortés A. Replication of Studies in Empirical Software Engineering: A Systematic Mapping Study, From 2013 to 2018. IEEE Access. 2020;8:26773–91.
16. Guevara-Vega C, Bernárdez B, Cruz M, Durán A, Ruiz-Cortés A, Solari M. Research artifacts for human-oriented experiments in software engineering: An ACM badges-driven structure proposal. J Syst Softw. 2024;218.
17. Xue X, Yu X, Zhou D, Wang X, Bi C, Wang S, et al. Computational Experiments for Complex Social Systems: Integrated Design of Experiment System. IEEE/CAA J Autom Sin. 2024;11(5):1175 – 1189.
18. Donthu N, Kumar S, Mukherjee D, Pandey N, Lim WM. How to conduct a bibliometric analysis: An overview and guidelines. J Bus Res. 2021;133:285 – 296.
19. Wu M-J, Zhao K, Fils-Aime F. Response rates of online surveys in published research: A meta-analysis. Comput Hum Behav Reports. 2022;7.
20. Mahmoodi E, Fathi M, Ghobakhloo M. The impact of Industry 4.0 on bottleneck analysis in production and manufacturing: Current trends and future perspectives. Comput Ind Eng. 2022;174.
21. Karimi S, Iordanova I. Integration of BIM and GIS for Construction Automation, a Systematic Literature Review (SLR) Combining Bibliometric and Qualitative Analysis. Arch Comput Methods Eng. 2021;28(7):4573 – 4594.
22. Li H, Li B. The state of metaverse research: a bibliometric visual analysis based on CiteSpace. J Big Data. 2024;11(1).
23. Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F. An approach for detecting, quantifying, and visualizing the evolution of a research field: A practical application to the Fuzzy Sets Theory field. J Informetr. 2011;5(1):146 – 166.
24. Ishibuchi H, Imada R, Setoguchi Y, Nojima Y. Reference Point Specification in Inverted Generational Distance for Triangular Linear Pareto Front. IEEE Trans Evol Comput. 2018;22(6):961 – 975.
25. Zhu Y, Zheng Z, Zhang X, Cai K. The r-interdiction median problem with probabilistic protection and its solution algorithm. Comput Oper Res [Internet]. 2013;40(1):451 – 462. Available from: https://doi.org/10.1016/j.cor.2012.07.017
26. Assis LS De, Franca PM, Usberti FL. A redistricting problem applied to meter reading in power distribution networks. Comput Oper Res [Internet]. 2014;41(1):65–75. Available from: http://dx.doi.org/10.1016/j.cor.2013.08.002
27. Dell’Amico M, Furini F, Iori M. A branch-and-price algorithm for the temporal bin packing problem. Comput Oper Res [Internet]. 2020;114:104825. Available from: https://doi.org/10.1016/j.cor.2019.104825
28. Yaseen M, Abbas M. An efficient computational technique based on cubic trigonometric B-splines for time fractional Burgers’ equation. Int J Comput Math. 2020;97(3):725 – 738.
29. Pessoa LS, Resende MGC, Ribeiro CC. A hybrid Lagrangean heuristic with GRASP and path-relinking for set k-covering. Comput Oper Res. 2013;40(12):3132 – 3146.
30. Sarin SC, Sherali HD, Judd JD, Tsai P-F. Multiple asymmetric traveling salesmen problem with and without precedence constraints: Performance comparison of alternative formulations. Comput Oper Res. 2014;51:64 – 89.
31. Clempner J. Verifying soundness of business processes: A decision process Petri nets approach. Expert Syst Appl. 2014;41(11):5030 – 5040.
32. Soo Kim J, Il Park S, Young Shin K. A quantity flexibility contract model for a system with heterogeneous suppliers. Comput Oper Res. 2014;41(1):98 – 108.
33. Marti R, Martinez-Gavara A, Sanchez-Oro J, Duarte A. Tabu search for the dynamic Bipartite Drawing Problem. Comput & Oper Res. 2018 Mar;91:1–12.
34. Rauchecker G, Schryen G. Using high performance computing for unrelated parallel machine scheduling with sequence-dependent setup times: Development and computational evaluation of a parallel branch-and-price algorithm. Comput Oper Res. 2019;104:338 – 357.
35. Zheng R, Dai T, Sycara K, Chakraborty N. Automated multilateral negotiation on multiple issues with private information. INFORMS J Comput [Internet]. 2016;28(4):612–28. Available from: https://doi.org/10.1287/ijoc.2016.0701
36. Myklebust TGJ, Sharpe MA, Tunçel L. Efficient heuristic algorithms for maximum utility product pricing problems. Comput Oper Res [Internet]. 2016;69:25 – 39. Available from: https://doi.org/10.1016/j.cor.2015.11.013
37. Kergosien Y, Giret A, Neron E, Sauvanet G. An Efficient Label-Correcting Algorithm for the Multiobjective Shortest Path Problem. INFORMS J Comput. 2022;34(1):76 – 92.
38. López-Sánchez AD, Sánchez-Oro J, Laguna M. A new scatter search design for multiobjective combinatorial optimization with an application to facility location. INFORMS J Comput. 2021;33(2):629 – 642.
39. Moncel J, Thiery J, Waserhole A. Computational performances of a simple interchange heuristic for a scheduling problem with an availability constraint. Comput Ind Eng. 2014;67(1):216 – 222.
40. Bahig HM, Fathy KA. An efficient parallel strategy for high-cost prefix operation. J Supercomput. 2021;77(6):5267 – 5288.
41. Cai Q, Deng Y. A fast Bayesian iterative rule in amoeba algorithm. Int J Unconv Comput. 2019;14(5–6):449 – 466.
42. Mongeon P, Paul-Hus A. The journal coverage of Web of Science and Scopus: a comparative analysis. Scientometrics. 2016;106(1):213 – 228.
43. Haddaway NR, Collins AM, Coughlin D, Kirk S. The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PLoS One. 2015;10(9):e0138237.
44. Kitchenham B, Brereton OP, Budgen D, Turner M, Bailey J, Linkman S. Systematic Literature Reviews in Software Engineering: a Systematic Literature Review. Inf Softw Technol. 2010;51(1):7–15.
45. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A. Experimentation in Software Engineering. Springer Berlin, Heidelberg; 2024.
46. Di Vaio A, Palladino R, Hassan R, Escobar O. Artificial intelligence and business models in the sustainable development goals perspective: A systematic literature review. J Bus Res. 2020;121:283 – 314.
47. Kan Yeung AW, Tosevska A, Klager E, Eibensteiner F, Laxar D, Stoyanov J, et al. Virtual and augmented reality applications in medicine: Analysis of the scientific literature. J Med Internet Res. 2021;23(2).
48. Merediz-Solá I, Bariviera AF. A bibliometric analysis of bitcoin scientific production. Res Int Bus Financ. 2019;50:294 – 305.
49. Yu D, Xu Z, Wang W. Bibliometric analysis of fuzzy theory research in China: A 30-year perspective. Knowledge-Based Syst. 2018;141:188 – 199.
50. Shukla AK, Janmaijaya M, Abraham A, Muhuri PK. Engineering applications of artificial intelligence: A bibliometric analysis of 30 years (1988–2018). Eng Appl Artif Intell [Internet]. 2019;85:517 – 532. Available from: https://doi.org/10.1016/j.engappai.2019.06.010
51. Yu D, Xu Z, Kao Y, Lin C-T. The structure and citation landscape of IEEE transactions on fuzzy systems (1994-2015). IEEE Trans Fuzzy Syst. 2018;26(2):430 – 442.
52. Niu J, Tang W, Xu F, Zhou X, Song Y. Global research on artificial intelligence from 1990-2014: Spatially-explicit bibliometric analysis. ISPRS Int J Geo-Information. 2016;5(5).
53. Centobelli P, Cerchione R, Esposito E, Oropallo E. Surfing blockchain wave, or drowning? Shaping the future of distributed ledgers and decentralized technologies. Technol Forecast Soc Change. 2021;165.
54. Landeta-López P, Guevara-Vega C. Supplemental Material: Computational experiments in Computer Science: A bibliometric study [Internet]. Zenodo; 2024. Available from: https://doi.org/10.5281/zenodo.12701189
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Pablo Landeta-López , Cathy Guevara-Vega (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The article is distributed under the Creative Commons Attribution 4.0 License. Unless otherwise stated, associated published material is distributed under the same licence.
