The Naval Architect: October 2018
Our experience of the world around us depends on a plethora of sensory inputs. When one of these inputs is lost, our situational awareness is diminished. This is particularly true of touch; although its value may be harder to describe than sight, it is an essential interface between us and our environment.
Recognition of this fact underscores the broad field of haptics research, and especially haptics technology, which attempts to recreate the sense of touch that has been lost in the transition from direct human-powered control to electrically and hydraulically actuated systems. Most people’s experience of haptics is via their smartphone, which incorporate haptic feedback in the form of vibrations, rumbling and clicking to provide a more reactive experience for the user.
Industrial applications in control systems offer greater possibilities and subsequently greater benefits. Incorporating a sense of touch through dampening, resistance and other techniques to reflect changes in the outside environment and vehicle operation is claimed to enhance control ability, improve situational awareness and increase safety. With haptic feedback, users don’t need to switch their focus between displays nor rely as heavily on visual cues, which have a coupling time between perception and action 10 times slower than haptics: 200 milliseconds versus 20 milliseconds.
Haptics in Holland
The Delft Haptics Lab, based at TU Delft and led by Professor David Abbink, has explored haptics in relation to robotics, cars, and aerospace, conducting a number of industry-funded projects. More recently, it has delved into the maritime world as Arthur Vrijdag, assistant professor at the Maritime and Transport Technology department of Delft, recognised the potential of haptics for vessel control systems – particularly in light of developments in remote operation. A recently-concluded cohesion project was soon undertaken to develop a test setup coupling haptic levers featuring actuation motors with VSTEP bridge simulation software. Using a haptic algorithm, different forces and torque can be applied to the motors in the control levers, allowing users to ‘feel’ the operation of the vessel.
The researchers like to say that “feeling is believing,” as “it is difficult to fully appreciate the ideas based on words.” However, examples include a control lever feeding back greater resistance as a vessel approaches the quayside, or providing counter-torque to make an operator aware of the actual position of an azimuthing propeller in relation to the position of the lever.
Abbink and Vrijdag are keen to point out that haptics goes beyond attention-grabbing vibrations which act as a warning that a threshold has been exceeded. More subtle types of haptic feedback are preferable because they act in relation to these thresholds. Abbink explains: “We think that the real power is not so much in vibrations, but in low-frequency forces. These can push you back, or make it more difficult to exceed certain operational limits. Feedback is given at the place where action is required, making things very intuitive.”
Another benefit of using forces is that they reduce subjectivity, says Abbink: “When you get the advice not to exceed a certain RPM, you have to interpret that advice. But if you feel it is more difficult to move the lever in that direction, it’s very easy to understand.”
Out of touch
In order for haptic feedback to reflect vessel operation, various inputs must be measured, filtered and ‘presented’ to the operator through a haptic algorithm. Whilst the researchers are still perfecting this algorithm, Vrijdag notes that the measurement process does not add much complexity as vessels are replete with sensors: “An incredible amount of sensor data is already being collected, processed and presented to the operator. One could say that for many possible applications of haptic technology onboard, the main sensors are already there, but the haptic levers provide a different way to communicate their outputs to the operator.”
In fact, a more significant issue is resistance from mariners who question the necessity of the technology, arguing that vessels can be operated safely without it. To this, Vrijdag retorts that “without haptics, of course you can still do tasks, but just slower with more effort and visual load,” thus placing more pressure on the operator. Abbink adds that this response is often grounded in the fear of ‘losing one’s job to a robot or algorithm’. However, he points out that haptics “really is about co-operation between the human and the algorithm. Haptics frees up brain space to think about what is coming in the future.” Despite its technical status, the researchers suggest that haptics actually provide a more natural way to operate a vessel, as it involves the human body in the process.
Remote operation
The improvement in situational awareness offered by haptics becomes especially desirable in the development of remote operation systems. Many sensory inputs are lost when operators are taken off the vessel, explains Vrijdag: “You take in so much more information from the environment than what you see on displays – ship motions, vibrations, the spray of water on your face. They all give you pieces of information that you put together and say either ‘everything is ok’ or ‘something is wrong and I need to check it out’. You lose all of these things if you are sitting in a control station.” Haptics can be applied to mitigate this problem: “We cannot replace all of these cues, but if we can partly put them back, we truly believe this will be powerful. Soft information is far more important than you realise.”
In the design of remote operator stations, the pair says that ‘multi-sensory integration’ should therefore be paramount: “When we design interfaces for humans, quite often we just do the visual sense: displays upon displays. To do this better is important and haptics has a very strong contribution.”
With increased distance between the operator and ship/sensor, the problem of latency arises – the delay between an input and output in the form of haptic feedback. However, the idiosyncrasies of maritime render this a minor issue as, in Abbink’s words: “A lot of maritime operations revolve around avoiding contact rather than rendering contact.” Therefore, small delays in transmission matter less than in ‘hard’ haptic operations where touch is actively applied.
As for the type of vessels that will benefit most from the incorporation of haptics, Vrijdag admits that this is still an open question. However, he suggests: “The more multifunctional ships are, the more options there are for haptic feedback.” Examples include offshore wind MPVs, ASD tugs, and inland vessels, all of which require operators to use levers almost continuously. Vessels of this type have been mooted as candidates for remote operation, exemplified by the demonstration of a remotely controlled Svitzer tug in Copenhagen harbour last year, which was fitted with Rolls-Royce technology.
More generally, haptics will likely aid remote operators piloting ships in port, as vessels of all sizes must be carefully manoeuvred to avoid the multiple and ever-shifting hazards. Therefore, if haptic feedback can deliver on its promise of improving control ability, it may result in an efficiency and safety boost in port settings, especially when coupled with the intelligent awareness systems being developed by the likes of Rolls-Royce Marine (now owned by Kongsberg) and ABB (see p14). For maritime and harbour pilots, haptic technology can also compensate somewhat for ever-more congested ports and growing vessel size.
Within reach
Having developed their test setup, Vrijdag and Abbink have recently conducted a project looking to reduce impacts on operators of small fast ships in waves, and have provided demonstrations of their levers at numerous maritime events. According to the researchers, several companies have also visited the lab to test the equipment and discuss possible applications.
Vrijdag acknowledges that the technology remains at a low readiness level. However, the researchers hope to use their levers with a full-size bridge simulator soon, involving third-party users to gain feedback and improve the system, particularly the algorithm. Ultimately, their goal is to work in full-scale in a safe zone or port and undertake a proof-of-concept both on board and remotely.
Initial reactions in a simulated environment, they say, have been positive, and often joyful: “When they feel what it’s like when they have it, you see their eyes light up!” Whether haptics delivers in full-scale remains to be seen, and this is paired with the task of convincing an often-sceptical industry. Regardless, Abbink insists that one conclusion can be drawn with confidence: “Maybe haptics isn’t necessary, but it sure helps!”
