ValidITy (Validation of Intelligent Terrain Feature Recognition Methods for Hydrographic Data) is a technology transfer - validation project funded by the Helmholtz Impuls- und Vernetzungsfonds (Funding indicator KA-TVP 44). It has a duration of two years, starting 01st July 2023 and ending 30th June 2025.

In the ValidITy project, we aim to improve the way unterwater object annotation is performed.
Current industry-wide deployed processes lean heavily on human annotation and manual processing.
In this endeavour, we will harness modern technology and UX development insights to create an all-on-one tool to perform the following tasks:

Explore

Good data quality is the first step towards a great result. To ensure that products like multibeam echo soundings or side-scan-sonar imagery can be used to reliably detect underwater objects, we will implement automatic data quality controls.

Furthermore, we want to make manual inspection of hydrographic data as effortless as possible. Experience working with our industry partners has shown that common pain points for this are long waiting times, visually indistinct colourizations and a lack of repeatability.

Using both, the experience our team has gained during the development of the Digital Earth Viewer, a best-in-class 3D visualization software for raw geodata products, and modern technologies such as General Purpose Computation on Graphics Processing Unit (GPGPU), we aim modernize and streamline the process.

Detect

Manual annotation in tools such as QGIS or SAGA can be strenuous as each object necessitates a series of user interactions. We envision three main ways in which this process can be improved:

• Marking an object should only take a minimum of interactions. For point-shaped annotations, this means each click should create a new object annotation.
• Annotating features can be vastly simplified if the colourization of the annotation surface already highlights the objects. This can be accomplished through mixture and thresholding of different derivatives, such as Terrain Position Index (TPI), aspect or slope.
• Using the feature map created in the previous step, we can propose new object locations to the user.

After a set of annotations has been created, a manual review is in order. A prototype streamlining this process has already been built. This prototype presents snippets from both multibeam and side-scan-sonar together with additional statistics to the user and allows them to approve or reject each site with just one keypress. We also aim to incorporate statistical measures such as t-distributed stochastic neighbor (t-SNE) clustering and visual hashing to allow the user to make a more informed decision or even assign classes to object groups based on geomorphic attributes alone.

Recognize

The annotations from the previous step can be used to train a machine learning algorithm to automatically find similar objects in hydrographically analogous terrain. Such a tool can for example be used in a workflow, in which a small representative area of the total domain is annotated manually, after which the algorithm can annotate the remaining area. In the BASTA project a technology demonstrator that uses the U-Net (2015) segmentation algorithm has been created and evaluated by our industry partners. We now aim to incorporate further proven object detection algorithms like YOLO (2016) and ViT(2020) into this application.

Since even the best machine learning algorithm will make mistakes, we will also add tools to verify the confidence levels and object similarity for each finding, building upon the review process used for manual annotations.

Finally, in some application cases, only a limited class of objects need to be identified. In this case, we want to build tooling that can correlate 3D models of the objects to be found with the input morphology.