Computational experiments in Computer Science: A bibliometric study
DOI:
https://doi.org/10.56294/dm2025188Keywords:
computational experiment, computer science, operations research, optimization, bibliometric analysisAbstract
Introduction: Computational Experiments are crucial in various fields, including biological sciences, engineering, social sciences, etc., and are a powerful tool for understanding complex systems, optimizing processes, and driving innovation. Their importance lies in their ability to integrate with experimental methods, facilitate simulation-based learning, and provide cost-effective, scalable, and flexible solutions for analyzing complex systems. The purpose of this study is to make a bibliometric analysis of the research related to Computational Experiments in Computer Science. Methods: This bibliometric analysis was performed using information from 2013 and 2024 from the Scopus and Web of Science databases, with published articles This bibliometric study followed the guidelines proposed in the publication “How to conduct a bibliometric analysis: An overview and guidelines” by the author Gonthu N. To answer the research questions, the number of articles per year, number of articles per country, number of articles per subject area, list of main journals, and citation analysis were analyzed. Results: The results show that Scopus has more publications on the subject, China is the country that publishes more on the subject, Mathematics is the predominant subject area, finally, a co-occurrence analysis was performed where a total of 27 clusters were found in Scopus and 10 clusters in WoS. From this, the 10 most relevant keywords in each of the databases were identified. Conclusions: This review can be a basis in order that researchers to have a starting point for the current state of publications on Computational Experiments for future research.
References
1. Yilmazlar IO, Kurz ME. Adaptive local search algorithm for solving car sequencing problem. J Manuf Syst. 2023;68:635 – 643. DOI: https://doi.org/10.1016/j.jmsy.2023.05.018
2. Barr RS, Golden BL, Kelly JP, Resende MGC, Stewart WR. Designing and reporting on computational experiments with heuristic methods. J heuristics. 1995;1:9–32. DOI: https://doi.org/10.1007/BF02430363
3. Landeta-López P, Guevara-Vega C. Computational Experiments in Computer Science Research: A Literature Survey. IEEE Access. 2024;12:132254–70. DOI: https://doi.org/10.1109/ACCESS.2024.3458808
4. Santner TJ, Williams BJ, Notz WI, Williams BJ. The design and analysis of computer experiments. Vol. 1. Springer; 2003. DOI: https://doi.org/10.1007/978-1-4757-3799-8_1
5. Jones B, Johnson RT. Design and analysis for the gaussian process model. Qual Reliab Eng Int. 2009;25(5):515 – 524. DOI: https://doi.org/10.1002/qre.1044
6. Law AM, Kelton WD, Kelton WD. Simulation modeling and analysis. Vol. 3. Mcgraw-hill New York; 2007.
7. Gönül-Sezer ED, Ocak Z. Comparison of system dynamics and discrete event simulation approaches. Adv Intell Syst Comput. 2016;442:69 – 81. DOI: https://doi.org/10.1007/978-3-319-31295-8_5
8. Kleijnen JPC. Design and analysis of simulation experiments. In: Springer Proceedings in Mathematics and Statistics. 2018. p. 3 – 22. DOI: https://doi.org/10.1007/978-3-319-76035-3_1
9. Craig PS, Goldstein M, Rougier JC, Seheult AH. Bayesian forecasting for complex systems using computer simulators. J Am Stat Assoc. 2001;96(454):717 – 729. DOI: https://doi.org/10.1198/016214501753168370
10. Cavendish JC. A framework for validation of computer models. In: SIAM Proceedings Series [Internet]. 2005. p. 87 – 99. Available from: https://cutt.ly/pw0LGnt7
11. Bayarri MJ, Paulo R, Berger JO, Sacks J, Cafeo JA, Cavendish J, et al. A framework for validation of computer models. Technometrics. 2007;49(2):138 – 154. DOI: https://doi.org/10.1198/004017007000000092
12. Guevara-Vega C, Bernárdez B, Durán A, Quiña-Mera A, Cruz M, Ruiz-Cortés A. Empirical Strategies in Software Engineering Research: A Literature Survey. In: 2021 Second International Conference on Information Systems and Software Technologies (ICI2ST). 2021. p. 120–7. DOI: https://doi.org/10.1109/ICI2ST51859.2021.00025
13. Črepinšek M, Liu S-H, Mernik M. Replication and comparison of computational experiments in applied evolutionary computing: Common pitfalls and guidelines to avoid them. Appl Soft Comput J. 2014;19:161 – 170. DOI: https://doi.org/10.1016/j.asoc.2014.02.009
14. Beaulieu-Jones BK, Greene CS. Reproducibility of computational workflows is automated using continuous analysis. Nat Biotechnol. 2017;35(4):342 – 346. DOI: https://doi.org/10.1038/nbt.3780
15. Cruz M, Bernárdez B, Durán A, Galindo JA, Ruiz-Cortés A. Replication of Studies in Empirical Software Engineering: A Systematic Mapping Study, From 2013 to 2018. IEEE Access. 2020;8:26773–91. DOI: https://doi.org/10.1109/ACCESS.2019.2952191
16. Guevara-Vega C, Bernárdez B, Cruz M, Durán A, Ruiz-Cortés A, Solari M. Research artifacts for human-oriented experiments in software engineering: An ACM badges-driven structure proposal. J Syst Softw. 2024;218. DOI: https://doi.org/10.1016/j.jss.2024.112187
17. Xue X, Yu X, Zhou D, Wang X, Bi C, Wang S, et al. Computational Experiments for Complex Social Systems: Integrated Design of Experiment System. IEEE/CAA J Autom Sin. 2024;11(5):1175 – 1189. DOI: https://doi.org/10.1109/JAS.2023.123639
18. Donthu N, Kumar S, Mukherjee D, Pandey N, Lim WM. How to conduct a bibliometric analysis: An overview and guidelines. J Bus Res. 2021;133:285 – 296. DOI: https://doi.org/10.1016/j.jbusres.2021.04.070
19. Wu M-J, Zhao K, Fils-Aime F. Response rates of online surveys in published research: A meta-analysis. Comput Hum Behav Reports. 2022;7. DOI: https://doi.org/10.1016/j.chbr.2022.100206
20. Mahmoodi E, Fathi M, Ghobakhloo M. The impact of Industry 4.0 on bottleneck analysis in production and manufacturing: Current trends and future perspectives. Comput Ind Eng. 2022;174. DOI: https://doi.org/10.1016/j.cie.2022.108801
21. Karimi S, Iordanova I. Integration of BIM and GIS for Construction Automation, a Systematic Literature Review (SLR) Combining Bibliometric and Qualitative Analysis. Arch Comput Methods Eng. 2021;28(7):4573 – 4594. DOI: https://doi.org/10.1007/s11831-021-09545-2
22. Li H, Li B. The state of metaverse research: a bibliometric visual analysis based on CiteSpace. J Big Data. 2024;11(1). DOI: https://doi.org/10.1186/s40537-024-00877-x
23. Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F. An approach for detecting, quantifying, and visualizing the evolution of a research field: A practical application to the Fuzzy Sets Theory field. J Informetr. 2011;5(1):146 – 166. DOI: https://doi.org/10.1016/j.joi.2010.10.002
24. Ishibuchi H, Imada R, Setoguchi Y, Nojima Y. Reference Point Specification in Inverted Generational Distance for Triangular Linear Pareto Front. IEEE Trans Evol Comput. 2018;22(6):961 – 975. DOI: https://doi.org/10.1109/TEVC.2017.2776226
25. Zhu Y, Zheng Z, Zhang X, Cai K. The r-interdiction median problem with probabilistic protection and its solution algorithm. Comput Oper Res [Internet]. 2013;40(1):451 – 462. Available from: https://doi.org/10.1016/j.cor.2012.07.017 DOI: https://doi.org/10.1016/j.cor.2012.07.017
26. Assis LS De, Franca PM, Usberti FL. A redistricting problem applied to meter reading in power distribution networks. Comput Oper Res [Internet]. 2014;41(1):65–75. Available from: http://dx.doi.org/10.1016/j.cor.2013.08.002 DOI: https://doi.org/10.1016/j.cor.2013.08.002
27. Dell’Amico M, Furini F, Iori M. A branch-and-price algorithm for the temporal bin packing problem. Comput Oper Res [Internet]. 2020;114:104825. Available from: https://doi.org/10.1016/j.cor.2019.104825 DOI: https://doi.org/10.1016/j.cor.2019.104825
28. Yaseen M, Abbas M. An efficient computational technique based on cubic trigonometric B-splines for time fractional Burgers’ equation. Int J Comput Math. 2020;97(3):725 – 738. DOI: https://doi.org/10.1080/00207160.2019.1612053
29. Pessoa LS, Resende MGC, Ribeiro CC. A hybrid Lagrangean heuristic with GRASP and path-relinking for set k-covering. Comput Oper Res. 2013;40(12):3132 – 3146. DOI: https://doi.org/10.1016/j.cor.2011.11.018
30. Sarin SC, Sherali HD, Judd JD, Tsai P-F. Multiple asymmetric traveling salesmen problem with and without precedence constraints: Performance comparison of alternative formulations. Comput Oper Res. 2014;51:64 – 89. DOI: https://doi.org/10.1016/j.cor.2014.05.014
31. Clempner J. Verifying soundness of business processes: A decision process Petri nets approach. Expert Syst Appl. 2014;41(11):5030 – 5040. DOI: https://doi.org/10.1016/j.eswa.2014.03.005
32. Soo Kim J, Il Park S, Young Shin K. A quantity flexibility contract model for a system with heterogeneous suppliers. Comput Oper Res. 2014;41(1):98 – 108. DOI: https://doi.org/10.1016/j.cor.2013.08.012
33. Marti R, Martinez-Gavara A, Sanchez-Oro J, Duarte A. Tabu search for the dynamic Bipartite Drawing Problem. Comput & Oper Res. 2018 Mar;91:1–12. DOI: https://doi.org/10.1016/j.cor.2017.10.011
34. Rauchecker G, Schryen G. Using high performance computing for unrelated parallel machine scheduling with sequence-dependent setup times: Development and computational evaluation of a parallel branch-and-price algorithm. Comput Oper Res. 2019;104:338 – 357. DOI: https://doi.org/10.1016/j.cor.2018.12.020
35. Zheng R, Dai T, Sycara K, Chakraborty N. Automated multilateral negotiation on multiple issues with private information. INFORMS J Comput [Internet]. 2016;28(4):612–28. Available from: https://doi.org/10.1287/ijoc.2016.0701 DOI: https://doi.org/10.1287/ijoc.2016.0701
36. Myklebust TGJ, Sharpe MA, Tunçel L. Efficient heuristic algorithms for maximum utility product pricing problems. Comput Oper Res [Internet]. 2016;69:25 – 39. Available from: https://doi.org/10.1016/j.cor.2015.11.013 DOI: https://doi.org/10.1016/j.cor.2015.11.013
37. Kergosien Y, Giret A, Neron E, Sauvanet G. An Efficient Label-Correcting Algorithm for the Multiobjective Shortest Path Problem. INFORMS J Comput. 2022;34(1):76 – 92. DOI: https://doi.org/10.1287/ijoc.2021.1081
38. López-Sánchez AD, Sánchez-Oro J, Laguna M. A new scatter search design for multiobjective combinatorial optimization with an application to facility location. INFORMS J Comput. 2021;33(2):629 – 642. DOI: https://doi.org/10.1287/ijoc.2020.0966
39. Moncel J, Thiery J, Waserhole A. Computational performances of a simple interchange heuristic for a scheduling problem with an availability constraint. Comput Ind Eng. 2014;67(1):216 – 222. DOI: https://doi.org/10.1016/j.cie.2013.08.017
40. Bahig HM, Fathy KA. An efficient parallel strategy for high-cost prefix operation. J Supercomput. 2021;77(6):5267 – 5288. DOI: https://doi.org/10.1007/s11227-020-03473-x
41. Cai Q, Deng Y. A fast Bayesian iterative rule in amoeba algorithm. Int J Unconv Comput. 2019;14(5–6):449 – 466.
42. Mongeon P, Paul-Hus A. The journal coverage of Web of Science and Scopus: a comparative analysis. Scientometrics. 2016;106(1):213 – 228. DOI: https://doi.org/10.1007/s11192-015-1765-5
43. Haddaway NR, Collins AM, Coughlin D, Kirk S. The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PLoS One. 2015;10(9):e0138237. DOI: https://doi.org/10.1371/journal.pone.0138237
44. Kitchenham B, Brereton OP, Budgen D, Turner M, Bailey J, Linkman S. Systematic Literature Reviews in Software Engineering: a Systematic Literature Review. Inf Softw Technol. 2010;51(1):7–15. DOI: https://doi.org/10.1016/j.infsof.2008.09.009
45. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A. Experimentation in Software Engineering. Springer Berlin, Heidelberg; 2024. DOI: https://doi.org/10.1007/978-3-662-69306-3
46. Di Vaio A, Palladino R, Hassan R, Escobar O. Artificial intelligence and business models in the sustainable development goals perspective: A systematic literature review. J Bus Res. 2020;121:283 – 314. DOI: https://doi.org/10.1016/j.jbusres.2020.08.019
47. Kan Yeung AW, Tosevska A, Klager E, Eibensteiner F, Laxar D, Stoyanov J, et al. Virtual and augmented reality applications in medicine: Analysis of the scientific literature. J Med Internet Res. 2021;23(2). DOI: https://doi.org/10.2196/25499
48. Merediz-Solá I, Bariviera AF. A bibliometric analysis of bitcoin scientific production. Res Int Bus Financ. 2019;50:294 – 305. DOI: https://doi.org/10.1016/j.ribaf.2019.06.008
49. Yu D, Xu Z, Wang W. Bibliometric analysis of fuzzy theory research in China: A 30-year perspective. Knowledge-Based Syst. 2018;141:188 – 199. DOI: https://doi.org/10.1016/j.knosys.2017.11.018
50. Shukla AK, Janmaijaya M, Abraham A, Muhuri PK. Engineering applications of artificial intelligence: A bibliometric analysis of 30 years (1988–2018). Eng Appl Artif Intell [Internet]. 2019;85:517 – 532. Available from: https://doi.org/10.1016/j.engappai.2019.06.010 DOI: https://doi.org/10.1016/j.engappai.2019.06.010
51. Yu D, Xu Z, Kao Y, Lin C-T. The structure and citation landscape of IEEE transactions on fuzzy systems (1994-2015). IEEE Trans Fuzzy Syst. 2018;26(2):430 – 442. DOI: https://doi.org/10.1109/TFUZZ.2017.2672732
52. Niu J, Tang W, Xu F, Zhou X, Song Y. Global research on artificial intelligence from 1990-2014: Spatially-explicit bibliometric analysis. ISPRS Int J Geo-Information. 2016;5(5). DOI: https://doi.org/10.3390/ijgi5050066
53. Centobelli P, Cerchione R, Esposito E, Oropallo E. Surfing blockchain wave, or drowning? Shaping the future of distributed ledgers and decentralized technologies. Technol Forecast Soc Change. 2021;165. DOI: https://doi.org/10.1016/j.techfore.2020.120463
54. Landeta-López P, Guevara-Vega C. Supplemental Material: Computational experiments in Computer Science: A bibliometric study [Internet]. Zenodo; 2024. Available from: https://doi.org/10.5281/zenodo.12701189 DOI: https://doi.org/10.56294/dm2025188
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Pablo Landeta-López , Cathy Guevara-Vega (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The article is distributed under the Creative Commons Attribution 4.0 License. Unless otherwise stated, associated published material is distributed under the same licence.
