The IRoNS method is based on similar works from different applications [1-7], as we adopt some common points: the planning, where/how to implement a simulation study, a validation process and how to do results analysis. The overall diagram is presented bellow, with a briefly description of its items afterwards.
Diagram of IRoNS Method Steps
Each box represents the action's result (action = arrow) and they are indicated by numbers ranging from 1 to 10. The box 0 is the only exception as it represents an "initial condition"
These topics are individually detailed in the next pages, but they may be summarized as bellow.
Note: We use "§" symbol for method's steps references and "[ ]" for literature references.
✅ §0 - General Problem
➤ This is the starting point which defines a general concept of the problem that we want to study. Is it just a network + cooperative robotics study? What kind of network or robots we gonna deal with it?
✅ §1 - Problem Definition
➤ After we have a general concept, we need to describe the problem in details, specifying explicitly what is the object of study. We also define which topics we want in the simulation/study. E.g: Network? Cooperative strategies? Energy consumption? Topology control? Obstacles? Sensors?
✅ §2 - Possible Solutions
➤ If we don't have a pre selected set of techniques that covers every topic from step 1, a research about possible algorithms/techniques is necessary. Documentation of the theory behind these techniques is also welcome, being specially useful in implementation stage.
✅ §3 - Definitions, objectives and assumptions
➤ With the problem defined and with the set of selected algorithms, it is time to define how these algorithms will interact to solve the problem. It is also necessary to establish objectives for the study and to make a document about assumptions made so far.
✅ §4 - Simulation Specification
➤ The fourth step is to plan the simulation implementation through conceptual and communicative modelling. The main idea is to understand the simulator, simulation routines, the expected interaction between algorithms and the libraries/add-ons needed in the simulation.
✅ §5 - Candidate Standard Simulation
Note: We use "§" symbol for method's steps references and "[ ]" for literature references.
✅ §0 - General Problem
➤ This is the starting point which defines a general concept of the problem that we want to study. Is it just a network + cooperative robotics study? What kind of network or robots we gonna deal with it?
✅ §1 - Problem Definition
➤ After we have a general concept, we need to describe the problem in details, specifying explicitly what is the object of study. We also define which topics we want in the simulation/study. E.g: Network? Cooperative strategies? Energy consumption? Topology control? Obstacles? Sensors?
✅ §2 - Possible Solutions
➤ If we don't have a pre selected set of techniques that covers every topic from step 1, a research about possible algorithms/techniques is necessary. Documentation of the theory behind these techniques is also welcome, being specially useful in implementation stage.
✅ §3 - Definitions, objectives and assumptions
➤ With the problem defined and with the set of selected algorithms, it is time to define how these algorithms will interact to solve the problem. It is also necessary to establish objectives for the study and to make a document about assumptions made so far.
✅ §4 - Simulation Specification
➤ The fourth step is to plan the simulation implementation through conceptual and communicative modelling. The main idea is to understand the simulator, simulation routines, the expected interaction between algorithms and the libraries/add-ons needed in the simulation.
✅ §5 - Candidate Standard Simulation
➤ This step uses all the documents elaborated before to implement the first simulation (Candidate Standard Simulation) with all the desired characteristics to conduct simulations to achieve the study objectives.
✅ §6 - Standard Simulation
➤ After implementation of the first simulation, is necessary to validate it. We assume that each algorithm may be validate separately and conduct a validation process in relation to a real application, in relation to another simulation (in another software) or comparing with similar results from the literature.
✅ §7 - Study Cases
➤ Using a valid standard simulation, we conduct an experimental design with the algorithms/parameters of interest. This experimental design consist in modifying the original simulation, one parameter/algorithm at time, and evaluate its impact on the results.
✅ §8 - Simulation Results
➤ This step is just de data acquisition from the experimental design established from the last step. It is important to have a correct number of data points (samples) to construct a statistical validation in the next step.
✅ §9 - Final Results
➤ After acquiring all the necessary data, it is time to study it and interpret the information. Here we propose a statistical procedure to help understand the relationship between the selected parameters/algorithms.
✅ §10 - Conclusion
➤ The last step is to see if the obtained results are in fact the ones that we were aiming in the begging of the study. Here we check the relation between results, the problem, assumptions and objectives, to determine if the study is valid and/or useful. If the results are the ones expected, they can be further improved in relation to its presentation.
⏪ Introduction Step §0-3 ⏩
References
[1] Balci,Osman. "Principles and Techniques of Simulation Validation, Verification and Testing",Proceedings of the 1995 Winter Simulation Conference, 1995.
[2] Siegfried, R. "Modeling and Simulation of Complex Systems", Springer Vieweg, ISBN 978-3-658-07528-6, 2014.
[3] Robinson, S. Simulation: The Practice of Model Development and Use", 2ª edição, ISBN 9781137328021, Palgrave Macmillan, 2014.
[4] Law, A. M. “Simulation Modelling and Analysis”, McGraw-Hill, 5th Edition, pp. 804, 2015.
[5] Grimm, V. et al. “The ODD protocol: A review and first update. In: Ecological Modelling”, Vol. 221(23), pp. 2760-2768, DOI: 10.1016/j.ecolmodel.2010.08.019, 2010.
[6] Sargent, R. G. “Verification and validation of simulation models”, in Journal of Simulation, 7, 12-24, 2013.
[7] Klugl, F. et al. “A validation methodology for agent-based simulations”, in Proceedings of the 2008 ACM Symposium on Applied Computing (SAC), DOI: 10.1145/1363686.1363696, 2008.
Page Release: 17/04/17
Last Update: 11/08/17
