Optimisation in Network Engineering: Challenges and Solutions in Research Data Management

In the complex realm of network engineering design, optimisation methods have been instrumental, using a range of components across different systems and scenarios. However, this complexity presents a dual challenge: first, managing, tracking and combining thousands of optimisation calculations, including the specifics of component data, system classifications, scenarios considered, and settings applied. Second, integrating diverse data from multiple sources that do not all reside in one place. Third, the possibility of collaboration (in this case with students, potentially with more people). Such challenges emphasise the need for rigorous research data management. Questions such as “which component data was used in which system?” or the provenance of component data come to the fore. To answer these questions, DataLad is used to store disparate data, models, settings and results in an effective and distributed manner. DataLad’s provenance reduces the redundancy of storage and the effort required for publication, while increasing confidence in the results. This is done in the context of a research project, but the same questions arise for the industrial application of what has been researched.