Technology Achievements and Pilots

Mass evacuation vessels need to keep up with the trends of passenger shipbuilding. The number of passengers are increasing while demographics state that the passengers average age increases and the participation of people with mobility and other problems are following suite. To address this two types of MEV’s are going to be designed.

MEV I will be designed to be retrofitted to existing ships. MEV I will be a composite floating structure that will be a recreational space during travel and will accommodate 60% of passengers on each side, exceeding SOLAS requirements. Battery operated hydro jets and an inflatable hull create a seaworthy lifeboat.

The advantages of this MEV concept are the following: i) high passenger capacity suitable for large passenger ships, ii) the MEVs will be functional spaces during the voyage, iii) low weight structures with manoeuvring and propulsive capabilities even in adverse weather conditions, iv) quick, unobstructed access in the MEVs suitable for people from a wide range of demographics, v) low-maintenance system, and vi) low levels of training required.

MEV II is intended to be part of the next generation ships, comprising of a modular superstructure and multiple MEV’s integrated in it. The difference with the MEV-I concept is that the MEV-II will be completely integrated with the rest of the ship superstructure and may therefore be used as fully functional passenger cabins during normal operation that will transform into MEVs during an emergency situation. As a result, access to these MEVs will be even easier, compared to the MEV-I,

as people will simply walk into the MEV from the deck. The MEV-II will also provide an even higher passenger capacity (estimated at about 120% of the total persons onboard on each side of the ship).

PALAEMON will develop a hybrid methodological design approach for implementing inclusive design principles with ergonomics considerations in accordance with maritime safety regulations as they are defined by IMO. The approach will focus on the core value of accessibility for the various aspects of the design and on making informed decisions to maximise the ‘Product performance indicators’ for the target end users. Within an inclusive design approach, we will acknowledge the mismatches between the needs of the whole spectrum of potential passengers and the design of the interior of the MEV. Potential passengers can include people with disabilities, passengers with affected cognitive, attentive, emotional, and motor functioning or factors that can negatively impact one’s ability to respond, i.e. alcohol consumption.

Technology development guidelines can be summed up as following:

1.  A MEV with an interior space that can satisfy the needs of people with any kind of impairment will result in a paradigm for designing such spaces that work better for everyone.
2.  An approach that will not result in a segregated, specialised and unsustainable design that does not serve the passengers and crew in the long run.

Based on SOLAS and STCW conventions supporting guidelines that focus on fire prevention, navigation safety, training and contingency planning, a study will be made where already existing safety systems on board are analysed and the implementation of new improved technology to support easier and faster safety system communication is planned according to the SOLAS chapters II-1 and II-2.

The concept proposed within this project will evolve in various stages as follows:

– Developing an interface that is efficient and user friendly about the current situation with coloration with IMO Conventions and beyond;

– Developing a simulation process to train and test the new proposed systems;

– Research of various past maritime accidents to provide insights about various alternatives that can be implemented within the system. This could be a software-based assistance for users in case of emergencies, based on various algorithms;

– Testing the new developed safety interface and improving the needs according to analysis of the tests;

– Estimating and analysing flaws and stress keys on the user, who is using the developed system;

– Developing a training concept in cooperation with Cruise companies, based on various interviews with the end user, and their experiences with the new developed safety system.

All trainings are going to be supported by a VR training system that is providing the chance to do “on sight” training and evacuations in a virtual environment with full free degree of movement. For virtualisation the software solution of M2D with a ship-model will be used.

The academy is going to be built on three modules:

Prevention: How to identify critical areas and potential threats. SOTA evacuation tactics on ships. Security technologies for dismemberment and high sea rescue operations.

Action: First aid training dedicated to water-related emergencies and typical injuries of mass evacuations, training of evacuation tactics and operations.

Mitigation: How to recover after an incident. What should be done after the first emergency evacuation. How to take care until final rescue to a harbour. Cooperation with legal authorities.

PALAEMON will introduce hardware accelerated cameras with autonomous capabilities to more accurately locate trapped victims. Hardware accelerated cameras are cameras with embedded smart capabilities (support of Lightweight analytics, video transcoding, video compression, smart behaviour). In PALAEMON will be also used legacy cameras (cameras that already exists in the ship’s) that will became smart and accelerated by connected with embedded system at the edge in order to include them in PALAEMON framework and cover the wide area PALAEMON surveillance. Hardware accelerated cameras will be designed and developed in order to handle:

• Sensor management tasks. Connectivity and information retrieval from different cameras and other sensors (microphones, etc.).
• Video processing in order to achieve unified video format.
• Lightweight video analytics (e.g. intrusion detection, motion, IR).
• Dynamic change of algorithms depending on the event that should be detected.
• Intelligent management of Lightweight analytics and embedded system operations (storage efficiency, network adaptation etc.).
• Intelligent trigger of the nearby cameras in order to cover fast a large area and prevent attacks.
• Two function modes, Smart and conventional CCTV
• Small area – single camera people detection. Counting, unusual behaviour, movement info in field of view.
• Large Area – multi camera people detection. Counting, unusual behaviour, people movement trajectory and obstacle detection and registration.

Life Cycle assessment of PALAEMON mass evacuation ecosystem will be performed with a sophisticated assessor tool that embeds a multi-criteria multi group decision making analysis and constitutes a dynamic safety framework seamlessly connected with risk assessment. The tool has the following key functionalities and characteristics:

1. Performance of PALAEMON evacuation system will be evaluated against a diverse set of attributes namely cost factors, security, safety, environmental issues, health etc. Significance of attribute over the aggregate performance of the evacuation system will also be set with well-defined weight factors;
2. A multi subject group of experts and innovators from different disciplines will play the role of evaluators. Importance of Innovators will also be weighed associating expert speciality with each attribute under valuation;
3. Each criteria assessment either beneficiary or costly, objective or subjective will support fuzzy, relative numbers and linguistic type of grading. Cost-benefit analysis will be used for performance output;
4. Performance assessment will be extended across the development of PALAEMON evacuation ecosystem bundled with the level of readiness of the system. Efficiency and effectiveness of evacuation system against a multi-criteria index will be constantly updated following maturity of solution delivered and degree of data analysis;
5. Each instance of assessment analysis will suggest improvements and adjustments in overall approach of system development;
6. LCA will be associated with risk assessment correlating hazard likelihood or severity assessment with readiness and effectiveness of project objective towards an automated holistic approach in evacuation process.

PALAEMON LCA tool will indicate an active and dynamic safety framework against currently state of the art passive and static framework.

A crew member must be aware of the different types of emergency situations that can arise on board the ship. This would help in understanding and tackle emergency in a better way and would also lead to taking correct actions to save lives, property, and environment. Procedures and guidelines to act in an emergency situation is a functional requirement and main component of safety management system on board. PALAEMON perspective towards a systematic and fully automated approach to operational planning of the evacuation process will be put in comparison with existing emergency situation guidelines and training manuals suggesting improvements and alterations. In this context PALAEMON will develop and utilize an ISM compliant SMS administrator tool for automation in controlling and updating emergency guides following PALAEMON holistic evacuation ecosystem suggestions.

Risk Management isn’t tied in with making enormous paperwork, but instead about distinguishing sensible measures to control the dangers in a work environment. Finding a way to secure crew and passengers with the least effort and reducing the working load on the crew without arising in flaws within the safety. The process would be developing a Smart Risk Assessment Platform (SRAP) on Board, which can be managed by both shore and offshore side in real time update. The system should give users the possibility to appraise new ideas or observations made by both shore and offshore sides.

Following steps will be needed to develop (SRAP):

– safety procedures according to IMO conventions BRM and ISM implementations

– design of a new strategy in Risk Assessment that is suitable for the evacuation system in development and other safety related issues.

– design of a paperless platform, which is both efficient and user friendly. Using new technologies (e.g. Apps, Smart watch etc.)

– implement communication guidelines between decision-makers (shore side and offshore side)

– bridge team resource management

Motion detectors based on piezoelectric accelerometers will be used to monitor the global response of the vessel at all times. Passive acoustic emission (AE) sensors will be employed to evaluate the structural integrity of the hull as well as the ingress of water from leaking parts of the vessel after a collision or grounding event. In case of an accident the information from the motion detectors and AE sensors will be fed to the PALAEMON control system of the vessel in order to accurately evaluate the structural integrity and global response time changes with respect to nominal-range values.

The motion sensors do not require a separate power supply and the signals transmitted are digital, which means two things:

• Plug and record capabilities of the system with no extra power source;
• No loss of signal quality since it’s digital.

With this system the following actions can be accomplished:

• Measurement and real-time monitoring of the actual condition (heel, trim, yaw) and deflection and torsion on the vessel during any ship operation;
• Storage of the log files for reporting and post-processing of motion data;
• Assessment and post-processing of recorded data.

The ship stability motion prediction module will use accumulated data from an interconnected wireless network of sensors and visual observation data from UAV’s, to determine the damaged condition of the ship. In case of faulty sensors, the data will be added manually to the system. To identify the expected dynamic motion behaviour of the vessel a simulation/calculation module will be implemented, which determines the maximum potential motions in six degrees of freedom for the ship, based on its hydrodynamic parameters. Weather forecasts predicting frequency, direction and significant wave height15 of the sea state, as well as the location of flooded compartments and their filling level will be taken into account. The main goal of this module is to provide data about the ships present and future performance. The intended simulation/calculation module will provide an as close to real-time ability as possible and will be connected to the on-board decision support system.

The main screen of COncORDE EMS comprises the central part of the UI. It is set up as a dashboard, from which all operational information can be accessed with one step click. It is a cloud-based platform which divides the emergency response in 5 spaces of work:

1. The PSAP/112 centre managing the initial alert phase

2. The emergency medical vehicles on their way to the incident scene

3. The Field/Incident scene

4. The patient transport vehicles to First Receiver

5. The First Receiver

The platform has an open source / community version which will be offered by KT and will be evolved further through PALAEMON to bridge the gap between the on-deck world consisting of trained individuals on the crew and the strategic one (dispatching centre, hospitals etc.)

The incident window in the dashboard allows access to the entire information related to incident management – PSAP info, Sitreps and Teamtable the platform can be accessed from http://concorde.konnektable.com/.

The current version of COncORDE EMS features the following subsystems/ tools and services:

Decision Support Service: PALAEMON partner KT will design and develop a decision support tool that can be used for efficient real-time resource allocation. Machine learning algorithms are going to be implemented to estimate the number of expected victims/patients in an emergency incident. The Decision Support Services Module implements four algorithms and offers them as services. Three of the services exploit optimisation techniques to recommend allocation of certain supplied resources to relevant demand, while the fourth one exploits machine learning techniques to learn from data and predict/ estimate required resources in a new emergency incident. For convenience, these services are listed below:

• Recommended (optimal) allocation of available crew units to incidents, depending on estimated needs
• Recommended (optimal) allocation of patients to master stations based on given order of evacuation and triage results for present injuries
• Recommended (optimal) allocation of tasks to available actors on the field, given demand pre-defined by the field commander.
• Estimation of expected casualties and demanded resources (EMS units), given historical data on emergency incident recordings

The Decision Support Services integrate with the semantic knowledge management module. This integration facilitates the semantic mapping between ship passengers (considering their status and injuries being the output of the medical treatment on the field) and First Receivers’ (Hospitals) reported capacity on medical specialities and beds.

Passengers Mustering and Evacuation Process Automation System (PaMEAS) is the system tracking and monitoring the position of customers within the ship (in the various areas of a ship, from the lower decks, where the state rooms locate, to the uppermost levels, which include the promenade and activity decks) and automatically launching, in the case of an emergency, a pre-defined evacuation plan that will be communicated to passenger’s mobile phones. Maritime safety standards require that all passengers on board should have the ability to escape in the case of an emergency from which the ship cannot recover. The PaMEAS system implements this regulation by streamlining and automating the processes of mustering and evacuation, and by creating a mobile interface between the ship evacuation plan and the passengers. The System will be able to identify

the location of passengers within the ship, and based on this information, to provide augmented safety functionality and new types of evacuation services such as, a) High accuracy indoor people tracking and analytics functionality, b) Passenger navigation in emergency situations and, c) Mustering and evacuation monitoring in real time and, d) SAR (Search and Rescue) specific services that can be used by the dispatched rescue forces to search and find survivors, promptly approach and rapidly assist them). Essentially, PaMEAS is designed to apply a pre-defined (by the Project) ship evacuation plan, to provide automatic guidance in emergency situations with the use of a mobile application (which may be of great help when people should cross dark and smoked filled areas).

PaMEAS will seamlessly integrate, through specific APIs, with other PALAEMON back-end infrastructure systems and onboard tool and services, while building on top of existing ship IT infrastructure (e.g. Reservation and Check-in, Crew Management, Shipboard Property Management etc.). In fact, PaMEAS System will cooperates through APIs and connectors with other physical and cyber systems that are included in PALAEMON project infrastructure, specifically with:

i. MEV – Mass Evacuation Vessel: MEV Sensoring Tool has an interconnection with MaPEAS System

ii. SSS – Smart Safety System: MaPEAS receives evacuation scenarios from SSS – MaPEAS implements part of the DSS functionality, i.e. Communicate effectively with passengers during an emergency and Demonstrate the use of personal life-saving appliances

iii. AR – Augmented Reality Glasses for Ship Crew: MaPEAS System provides feedback to AR Core

iv. COncORDE – COncORDE Emergency Management System: MaPEAS provides aggregate information to COncORDER for global ship monitoring needs)

v. ITML – Enhanced data fusion and analysis techniques: MaPEAS provides information to ITML

vi. UAV – UAV Inspection Module: MaPEAS provides information to UAV (directly or through other PALAEMON IT Systems)

Essentially, the functionality of PaMEAS System is based on the deployment of a commercially available off-the-shelf (COTS) Indoor Position System, tailored to the needs of a passenger ship. An indoor positioning system (IPS) is a system to locate people and objects inside edifices where GPS stops working, by using the capacity of the telecom equipment to identify users (such as, for example, the ability of Wi-Fi Access points to know what device is connected), along with the information collected by typical sensing devices (such as beacons etc.) and by context-specific mobile apps, installed in mobile devices. By incorporating an IPS System, PaMEAS system will use the indoor tracking functionality available by these systems to create and deploy within the ship a new layer of service functionality consisting of passengers’ location monitoring for safety purposes, mustering and emergency navigation; and, enable the deployment of a unique service point for on-board safety coordination, from where passengers’ safety can be operated, verified and monitored (the indoor positioning functionality can be also used by the ship management, in regular time periods, to create new customer experience-based services, a possibility that creates an additional, important, investment incentive). Obviously, the System will be also able to monitor the location of seafarers, according to the applying legislation, and provide them critical information and real-time task-based guidance and in the case of an emergency.

As explained before, PaMEAS is designed to apply a (pre-defined) ship evacuation plan and provide guidance to passengers with the use of a mobile application (i.e. real time navigation) to improve the operational effectiveness of the mustering, evacuation and rescue processes. PaMEAS System is a flexible infrastructure which extends the functionality of the existing ship IT systems. It is made of

several network hardware and IT software components. Hardware refers to the part of the System that should be implemented in the different decks of a ship and includes: Wi-Fi Access Points, 4G(LTE)/5G Radio Dots, beacons and other sensors. Software refers to a software suite that will be deployed locally

Figure 13: PaMEAS Hardware & Software Components.

This proposal version was submitted by Philippe CHROBOCINSKI on 19/09/2018 15:36:06 Brussels Local Time. Issued by the Participant Portal Submission Service.

Marine Accident Response (subtopic B) MG-2-2-2018

PALAEMON page 18

(within the ship) and remotely (in the cloud), to process the collected information from the hardware components and provide location analytics, emergency navigation for the passengers and rules-based safety policy decisions for the ship management.

PaMEAS hardware: The hardware equipment implemented in the ship will follow the ongoing technology evolution path, by exploring two different technology trajectories (Wi-Fi16 vs 4G LTE/5G17) while studying and promoting their coexistence. In fact, PaMEAS will establish:

i. A core wireless infrastructure based on state of-the-art technologies, using mostly Wi-Fi Access Points, beacons and other sensors,

ii. An augmented wireless infrastructure that will complete the core infrastructure, a 4G LTE/5G Radio Dot Network18 (consisting of indoor small cells implemented in priority in the most difficult to monitor areas of the ship, such as decks hosting staterooms and corridors), in the objective to further improve people location accuracy and service efficiency.

Eventually, these two infrastructures may co-exist for a longer period, in a context where the core wireless infrastructure (Wi-Fi) will cover the activity decks of the ship, while the Radio Dot Network (augmented wireless infrastructure) serving the staterooms areas and the corridors leading to the passengers’ cabins (lower ship decks). The coexistence of these technologies will also allow for a safe transition from the state-of-the art to more advanced technologies, such as Radio Dots, which may however have a significant market penetration only in a few years from now, perhaps close to the end of the project.

PaMEAS software: The hardware equipment will be completed with a software suite that offers to the ship management automatic people and objects tracking (i.e. position identification) based on the processing of information collected form network antennas, different types of sensors placed in the ship and mobile devices, along with: location analytics, navigation guidance in emergency conditions and automated mustering and evacuation, safety-related decision policies based on automated rules etc. A machine learning component completes the above suite, to offer the possibility of improving location efficiency based on the analysis of past location paths, essentially by identifying structures and routines hidden in users’ past locations. Finally, people privacy is ensured by a specific module that establishes user-control over the activation of location services, process certification in the case of an automatic launching of the location tracking software (in the case of an emergency), and transparent exchange of personal data between the user and the ship IT management.

The components of the PaMEAS software suite are listed below and illustrated in the following Figure: a) RTLS System (Location identification), b) Rules-Based Engine (RBE), c) Service Portal and

Mobile App, d) Privacy Management Tool, e) Analytics Engine and Machine Learning, f) PaMEAS System APIs.

Augmented Reality Glasses for Ship Crew (AR)

Interoperable Communication Platform

Enhanced data fusion and analysis techniques

Passengers Localisation

UAV Inspection Module

Data Management

Data Analytics Engine & Weather Forecasting Toolkit

Advanced Augmented Reality (AR) technologies to enable the crew to be aware of the full extent of the damage occurred to the ship in real time, provide them with information regarding on-going mustering and ship stability and structural issues, while aiding them in steering all passengers to the designated evacuation areas and MEVs.

The application will adapt the resources according to the technical specifications of the running systems and will be:

Scalable: All software modules will scale according to the technical specifications of the devices they are running.

Modular and extensible: It will be easily extended so that it adapts to new functionalities without affecting existing ones.

Standardized: Will be based on standards of development, communication, graphical standards and usability.

PALAEMON will leverage on some of the most relevant components of FIWARE platform as the baseline IoT/ICT platform to implement the smart digital onboard system:

• IoT Agents to ingest heterogeneous data from diverse sensors.

• NGSI data model and Context Broker to digitally represent virtual entities and their attributes following a standard approach that ensures interoperability.

• Cygnus and supported datastore technologies (i.e., MongoDB, Hadoop, Elasticsearch, etc.) to collect, distribute, aggregate and persist information.

Additional components will be developed, integrated and/or extended to support the specific requirements of regulations, stakeholders and use-cases:

• Interconnection of devices with specialized communication protocols and features: BLE for smart bracelets, video streaming from smart cameras, etc.

• State of the art data-analytics tools that can lead to the refinement of the smart application deployed on top of the digital system or the design and implementation of new ones.

• Advanced data streaming and Artificial Intelligence (AI) technologies (i.e. Apache Kafka, TensorFlow, Keras) so that end-user applications may exploit machine learning techniques over high-performance, low-latency, robust and secure platform.

• Identity and Access Management (IAM) and proxies to enforce security at the different tiers of the system relying on standards like Oauth2.0 and OpenID Connect (OIDC).

A flexible hybrid cloud-edge infrastructure will be designed to execute PALAEMON digital onboard system considering scalability and reliability as key features.

Enhanced data fusion and analysis techniques to improve the system performance and present to crisis managers and decision makers a visual representation of the situation.

In more detail, advanced data analytics will be designed based on machine learning technologies, in order to realize the provision of:

• Data fusion from different data sources (sensors, cameras);

• Data pre-processing technologies;

• Real time data clustering and classification;

• Predictive analyses in order to provide suggestions and to support decisions on crisis management situations;

• Advanced and interactive visualisation tools to ensure that crisis managers and decision makers have a clear and thorough view of any situation at any time.

The above enable the system to:

• provide real time smart situation awareness

• identify the crisis level of each situation

• assess effectiveness of all alternatives regarding potential evacuation pans

• deliver decision support functionalities

• deliver an advanced visualisation toolkit for PALAEMON.

PALAEMON’s technical approach will be based on the fusion of different technologies, i.e. a hybrid solution. Although there are several types of technologies ready to be used, the features of ship forces to select only a few of them:

• Bluetooth low energy (BLE) is one of the best candidates to be implemented. This type provides a good balance between energy and connectivity. Several devices can be used to implement a prototype to monitor people.

• For outdoor areas the deployment of beacons is not required, and GPS can be used.

• According to the ship facilities, other technologies such as Wi-Fi could be used to locate people according to the RSSI of the received signal in each person and positioning by triangulation.

PALAEMON, as an integrated ecosystem of autonomous inspection platforms that will support several evacuation & mustering scenarios in an autonomous manner will exploit well-established technologies, such as distributed communication networking, UV navigation strategies, signal and vision-based event detection and actively include end-users to its design to promote the swift incorporation of PALAEMON’s framework to the evacuation operations.

In more detail, PALAEMON will include:

• Airborne Platforms: Fixed wings UAV equipped with EO/IR gimbal and a set of different type of cameras.

• Remote Command & Control capabilities: Generic Ground Control Station (GCS) with piloting module, mission, monitoring module and data exploitation module. The GCS is also used as an information system combining all the data to produce a common operational picture that can be customized (situation and interpretation).

• Command and Control System represents the main interface for the end users, allowing the definition and planning of field operations, as well as monitoring and managing these operations.

• Heterogeneous data gathering sensors: A dedicated Payload Management system allows quick plug-and-play hardware re-configuration, so that different sensor payloads can be installed on-board the UVs depending on the specific operation needs.

• Advanced cooperative navigation capabilities: The platform will feature advanced methodologies to allow cooperative navigation for homogeneous agents but also for groups of heterogeneous agents in combined operations.

• Communications framework and Data Links: A Communication Manager will be installed on all the robotic platforms allowing them to be provided with different, inter-changeable communication front-ends, supporting a multitude of different links enabling direct communication with the main Command and Control System (CCS).

The Data Management services of the platform will convert the data in motion to data at rest by delivering functionality that aggregates the data from various types of sources and then passes them to the Data Analytics module for analysis and action. The Data Management services deliver the following functionality:

• Identify data sources of interest and determine which pieces of data are of interest to higher layers in the architecture, based on the semantic infrastructure of the PALAEMON platform.

• Determine if data must be persistent and the type of storage needed (e.g., a big data system, or a relational database).

• Determine if data must be recombined or recomputed: Data might be combined, recomputed, or aggregated with previously stored information.

• Filter, transform the data and integrate it with other data sources of interest (into a composite whole) de-duplicating the resulting composite data set.

A state-of-the-art deep learning approach will be used to harness the power of training data generated by the sensor network, the PaMEAS system, the information from intelligent framework and the communication tool.

Concerning the sensor network a Recurrent Network architecture successfully captures the temporal dimension of the data.

Concerning the Extreme Weather Events Evaluation Tool will consist of a short-term weather prediction system. State-of-Art techniques like Stochastic Gradient Descent, Adagrad, Adadelta, Nesterov’s momentum, etc. will be implemented for data handling.