DTOcean Development: Organising the Packages

23 Feb 2017

A Gentle Renaming

Before commencing a new wave of development I want to describe the purpose of each of the packages in the DTOcean project. As part of this process of description it has become clear that some of the names of both the repositories and the packages themselves could be improved.

One of the main areas of confusion is which of the packages is the principal one. Although many people thing that it is dtocean-core, in fact it’s dtocean-gui which provides the final user interface. One reason that this confusion could have occurred is that dtocean-core was in development for significantly longer than dtocean-gui as it provides the control aspects that dtocean-gui operates. Indeed DTOcean can be run with dtocean-core alone, but there will be no graphical aspect other than output graphs.

Thus, to reduce confusion it has been decided to rename dtocean-gui as dtocean-app. Hopefully, this will make the package appear of more importance in the hierarchy than dtocean-core. This and all the other changes to package names are as follows:

• dtocean-gui to dtocean-app: As explained above.
• pandas-qt to dtocean-qt: This is a fork of the pandas-qt package that has been modified to work with the DTOcean GUI. To avoid confusion with the original package (although depreciated) the name will be changed.
• dtocean-operations to dtocean-maintenance: The naming here is misleading as this package deals only with maintenance operations, not, for example, installation operations which is dealt with by dtocean-installation.
• dtocean-environmental to dtocean-environment: This is to match the naming used within the package itself.

In addition to the above changes to the packages the Github repositories names will also be modified to match.

Package Descriptions

For development of the DTOcean tool, it is necessary to understand the purpose of the 14 packages that make up the project. They can be divided into 4 groups:

1. Support: These packages provide basic functionality that may be shared among many packages.
2. Core: These packages control the execution of the design and assessment modules, control the data flows between the database, user and modules and provide interactive access for the user.
3. Modules: The design modules execute the scientific algorithms.
4. Assessments: The assessment modules provide metrics for the outputs of individual modules or the entire design.

The packages in each of these groups will now be discussed in turn. Further details can be found in the DTOcean technical manual.

Support Packages

polite

The polite package contains a number of shared functions for working with configuration files and the python logging system. It makes deploying and utilising a logging configuration file extremely “polite”. The package is utilised by almost all the other DTOcean packages.

dtocean-qt

A fork of the now defunct pandas-qt package, this has been modified to work with the DTOcean (Anaconda) Qt system and provides some additional functionality for cherry-picking how pandas tables can be edited, such as allowing new rows but not columns.

Core Packages

aneris

The package aneris is the underlying data coupling and action scheduling framework. It provides all the pieces required for the logical structure of dtocean-core, however it could be used for different applications that required any functionality where generic interfaces are sharing data. It also allows for extensions through the use of plugins.

dtocean-core

The dtocean-core applies the functionality of aneris to a more strict structure. It contains the concepts of executing a number of modules in order which will then be immediately assessed by a number of thematic assessments for the module itself and the global, cumulative state of the data. It also provides interfaces to the database and to the user in forms of data input and output and plots. It uses the plugin architecture of aneris to allow for easy extensions, including defining advanced execution strategies for optimisation.

dtocean-app

This package provides the Qt4 GUI for interacting with the dtocean-core. Although it does not contain any additional logic, visualising the data provides significantly greater insight over using dtocean-core on its own and increases productivity.

Modules

dtocean-hydrodynamics

The hydrodynamics module for DTOcean. It encapsulates 4 python modules, dtocean-wave, dtocean-tidal, dtocean-hydro and dtocean-wec. The first three provide the solvers and optimisation routines for designing a wave or tidal array and dtocean-wec provides a tool for creating a complex input for the wave solver by utilising the open source code NEMOH.

dtocean-electrical

This module develops the electrical network for the array up to and including the onshore landing point. It also selects an electrically appropriate umbilical cable should the selected devices be floating.

dtocean-moorings

The moorings and foundations module will design foundations for all devices and furthermore design mooring for floating devices. If an umbilical cable has been chosen (either by dtocean-electrical or the user) then the module will adjust the mooring and foundation system so that it is compliant with the cable.

dtocean-logistics

This module is not exposed to the user but provides all the logistics logic (such as selections of vessels and ports, scheduling and duration information) for the installation and maintenance modules.

dtocean-installation

The module generates an installation solution for the device array layout, electrical and moorings and foundations networks. It also indicates when the maintenance phase of operations can commence.

dtocean-maintenance

This module generates a maintenance strategy for the array layout and electrical and moorings and foundations networks. This strategy can be tuned by the user depending on whether they prefer a calendar based, condition monitoring or unplanned corrective strategy. The module not only requires dtocean-logistics but is also dependent on dtocean-reliability to provide the likelihood of failure of the components and device subsystems. Failure events are estimated in the time domain and a maintenance schedule and adjusted power production are produced for the lifetime of the array.

Assessments

dtocean-economics

The first assessment module is used to calculate the levelised cost of energy for any given input. A lot of additional work is done within the dtocean-core interface to this theme to create additional outputs for the user.

dtocean-reliability

Generates mean time to failure and other metrics for the networks produced by the electrical and moorings and foundations module or can be used with custom networks, as is the case with the maintenance module.

dtocean-environment

This assessment generates two numerical environmental impact scores based on the array design, operations details and inputs from the user. The first score considers any negative impacts from the array whilst the second score considers any potential positive impacts.

Interrelationships

Now that the purpose of each package has been briefly discussed, it is important to understand the relationships between them. Of particular interest is visualising the impact that changes in one package will have on those packages that depend on it. A new release of a package at a low level in the chain could necessitate at least a new release of packages higher up and potentially require changes in response.

As can be seen from the graph above, a change in any of the design modules may necessitate a change in both the dtocean-core and dtocean-app packages. As dtocean-logistics is a dependency for two other modules, changes to it may require changes in the 4 packages which are upstream of it. Also note that a change in dtocean-reliability affects 4 packages as it may trigger changes to dtocean-maintenance.

Missing from the above diagram is polite. As polite is a dependency for almost all of the other packages, it was too untidy to include it in the graph. Also, it is more mature than the other packages and therefore less likely to be changed significantly and require changes from upstream. However, there remains a risk that a significant change in polite could require new releases for numerous other packages in the project.

Managing Change

From the interrelationships shown above, it is clear that managing changes in the DTOcean project is a challenging task. This topic will be discussed in another blog post where the procedures for developing the modules will be detailed. The important lesson from this post is that communication between the developers working on each module is key to ensuring that changes are beneficial to the entire project.

DTOcean Installation Solution

15 Feb 2017

Sporadic Installation Issues Reported

Following the release of DTOcean (see DTOcean: An Introduction) a significant number of users reported issues with the installer crashing. This occurred after the main code installer had finished, when the hydrodynamic data installer fails to find the chosen installation directory.

Missing Registry Key

The symptom of this error is clear. The hydrodynamic data installer is looking for a registry key that contains the installation directory. This key should be written by a small script which is executed at the termination of the first installer. The registry key that should be written is HKEY_CURRENT_USER\Software\DTOcean, subkey INSTALL_DIR, however it is evident that this key was not written for those suffering this bug.

Uncertain Causes

It is currently not clear why the post-install script is not working for some systems whilst working for others. Some possibilities include:

• The script is failing. This could be from the commands not being recognised on certain systems (or within certain shells).
• The user does not have sufficient rights to either run the script or write to the registry
• Anti-virus software is blocking the execution of the script.

Without some logging from the script to see why it failed to execute properly, it’s hard to accurately diagnose the problem. To this end, I created an issue for Anaconda Constructor here.

Temporary Solution

As the post-install script is not working in all cases, to solve this issue the user must write the installation directory to the registry prior to installing DTOcean.

To facilitate this, I have written a batch script which, when executed, will prompt the user to enter the chosen directory.

The script can be downloaded using this link. Once extracted, double click the dtocean_pre_install.bat script and follow the prompts, as shown in the image above.

Note, that the directory entered in the pre-install script must match exactly the directory entered into the installer. Also, the installation directory must not exist prior to beginning the installation. If the directory exists following a failed installation, either attempt to uninstall or delete it manually.

This solution should allow the current version of DTOcean (1.0) to be installed while a more robust fix is developed for future releases.

Notes

• The top image is courtesy of Sterling Olson of Sandia National Labs
• Follow the related issue on Github: DTOcean/dtocean-issues#3
• Join the DTOcean mailing list

DTOcean: An Introduction

30 Jan 2017

What is DTOcean?

The official launch of the DTOcean software tool was announced on the 9th of January 2017. This was the final official output of a 3 year, 6 million Euro project to deliver a software tool to improve the decision making around designing arrays of ocean energy (wave and tidal) devices. Alongside the press release an installable graphical desktop tool (as seen above) and all its associated source code was made available to download by the general public under various open source licenses.

DTOcean is an ocean energy converter array planning tool that can be used to evaluate the impact of design decisions against quantitative metrics such as the levelised cost of energy. Various design choices can be evaluated using the 5 design modules: Hydrodynamics, Electrical Sub-systems, Moorings and Foundations, Installation and Operations and Maintenance. The results of varying parameters across these 5 modules can then be directly compared.

Who Developed it?

DTOcean was developed across 18 institutions across Europe (please see the official DTOcean website for further details). The bulk of the software development fell to about 8 of these, with Tecnalia Research and Innovation, of the Basque Country, playing a key role. In late 2014, I started a Marie Curie fellowship with the marine energy group of Tecnalia and I was principally working on DTOcean until the end of my contract in December 2016.

My role on the project was to coordinate the integration of the design modules and develop the framework for handling data between the modules, scheduling their execution, and interacting with the user through a graphical user interface. I also defined the unified data model that is observed and populated by the user of the tool.

I was not responsible for the development of the computational modules themselves or the supplementary database (they were led by other partners), although I did contribute to these projects and tried to coordinate their interactions with the other components of the tool. I was also not the sole contributor to the integration process: Vincenzo Nava, David Bould, Adam Collin, Rui Duarte and Francesco Ferri all contributed to giving the software as much functionality as possible by configuring and building upon the underlying framework.

Building a Community

Currently the installers, source code and supplementary data can be downloaded from SETIS, all of which corresponds to the 1.0 release at the end of the project. Although this portal provides a permanent home for these files it does not facilitate community engagement or future development.

To begin the process of developing a user community around DTOcean, I have placed all the source code onto the GitHub repository service. The “organisation” DTOcean contains a repository for each component (and subcomponent) of the tool and an important additional repository called dtocean-issues.

The dtocean-issues repository facilitates community engagement by allowing users of the integrated graphical tool to submit bugs or questions or even to suggest future improvements to the software. If you are using DTOcean and want to provide some feedback I would encourage you to open an issue.

Although there is no formal, funded support for DTOcean at this point, collecting the experiences of the users will be invaluable for assessing how much interest the tool has among the ocean energy community and how it could be improved should resources become available in the future.

Building a Future

A lot of work remains to be completed to organise the future development and exploitation of DTOcean. Currently, I am available to do a small amount of unpaid work to help disseminate the tool and advise potential users, but it is important for the future of the tool that some funding can be secured in order to improve the stability and accuracy of the software and respond to issues raised by the user community.

Although DTOcean has garnered significant interest from various actors such as research institutions and device developers, neither the tool itself, or the industry it is designed for, is mature enough to appeal to the utilities that would pay for its full exploitation.

Even though the software itself is free, the investment of time to develop and improve it is not. Ultimately, the success of DTOcean will be judged on whether it can attract the funding required to secure its future. Developing a strong user community is the first step to achieving that goal.