Home

INSIGHT

1. Extra Terrestrial Disaster Meteorit Technological Disaster Traffic Jam Football Blackout Special Critical Unknown Twitter Gruppe A Gruppe B Figure 17 Ontology Provides all possible types of an event Only events of selected highlighted types are shown D6 2 Version 1 0 31 08 2015 19 INSIGHT FP7 318225 3 7 Feedback collection and evaluation results In the following the results and findings of the evaluation scenario are described 3 7 1 Evaluation Results The nation wide scenario has been evaluated in two independent groups Both groups used the same INSIGHT system and blackout scenario The system automatically detected relevant events based on mobile network sensor data and Tweet streams The events have been assigned to one event type based on the event ontology Additionally it was possible to also integrate crowd sourced information This included information from the external trusted organizations such as the experts from the control room or members of emergency services police and firefighters The results with respect to functionality are The goal was to detect relevant events during crisis situations early in advance and thereby put crisis managers in the position to pro actively manage a situation The event detection features of the ISA are well developed and work as designed but there is room for optimization Especially events during emergencies that are locally focused and contained s
2. All the functions are logically located It was easy to understand the different functions O O O O O The size of the map was big enough O O O O O It was easy to zoom in on events on the map O O O O O I could understand the events on the timeline O O O O O It is easy to recognize the type of event O O O O O It was clear what the numbers on the vertical line of the timeline indicated i j The visible timeframe was big enough O O O O O The timeline was easy to use O O O O O The layer function was clear O O O O O The layer function was easy to use O O O O O I could understand the function of the heat map O O O O O It was clear how to use the navigator function O O O O O e U to save locations with the navigator 0 0 O 0 O eur ne the events shown on the O 0 O 0 D It was easy to search for events O O O O O D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 I could locate the events easily O O O O O I could quickly understand the type of event O O O O O Enough events were shown on one page O O O O O I could understand the function of event details O O O O O I could understand the weight of an event O O O O O I could understand the event details of the mobile phone usage R s I could understand the event details of the tweets O O O O O I could understand the two different lists in the 0 O 0 0 O event list I used the upper event list more than the lower event list Visualization and Usability fully mostly partially do not
3. completely agree agree disagree agree agree I could change the order of the events The usability of the ontology was easy The ontology was listed in a logical way There were enough types of ontology Usage during the exercise fully nos le agree agree agree agree disagree I selected the area with the most events first I selected the area which was red on the heat O O O O O map first I often navigate to a specific area on the map O 0 O 0 O by using the map I often navigate to a specific area on the map O O O O O by using the navigator I often navigate to a specific area on the map i O O O O O by using the column lat lon in the event list I often navigate to a specific area on the map l O O O O O by using the lat lon in the event details The way I selected events was random O O O O O I often clicked on the peaks shown on the 0 o 0 0 0 timeline I often used the function to focus on a specific 0 O 0 0 0 time D6 2 Version 1 0 31 08 2015 10 INSIGHT FP7 318225 The event marker function was turned on O O O O O during the entire exercise The heat map function was turned on during O O O O O the entire exercise The areas layer function was turned on during O O O O O the entire exercise The traffic layer function was turned on during O O O O O the entire exercise I often used the navigator function O O O O O I saved locations in the navigator function O O O O O I often used the search func
4. The Scats ISA identifies traffic congestion events using the static SCATS sensors of the Dublin city The goal of its evaluation is the check if the alarms of the SCATS ISA correspond to real traffic events Evaluation windows 30 minute windows 09 00 09 30 13 00 13 30 and 22 00 22 30 or the latest someone is in the traffic control room For both evaluation days DCC please define different and more appropriate evaluation windows Instructions In a new INSIGHT page or after reloading an existing page enable the Layer ISA Anomalies and ensure that the Icons button is enabled as shown in the image bellow D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 RT Events 2 ISA Anomalies Live Bus Live Scats data size 10 Last Anomaly 8 24 2015 12 37 30 Ph Cams Sensors Cluster Task During the above evaluation windows the DCC operators should look at the screen of INSIGHT and for every SCATS anomaly high degree of saturation marked by this icon note down the following information from the popup window that looks like this High Degree of Saturation DS observed for a particular SCATS sensor Details presented bellow Generate Plots Date 3 24 2015 12 54 00 PM AID 1 451584 89 Please close the window by clicking the x mark after you finish working with an anomaly u The Approve Reject and Don t Know
5. disasters on top of the situation Usability refers to the fact that the system must be usable by professionals in order to unfold the benefits of the INSIGHT project Transferability to other scenarios and adoption by professionals is a success measure of the project The prototype was assessed by the target group in terms of testing three hypotheses 1 Participants used the prototype during the entire exercise 2 Participants utilized the information given by INSIGHT 3 Participants became earlier aware of the situation because of INSIGHT The target group are professionals and participants who have not been familiar with the system and without prior knowledge on the system s capabilities Our intention is to simulate real world conditions under which the BBK operates Here the professionals working in the situation room are typically delegated from other positions to temporarily work for the GMLZ This imposed additional challenges during the implementation phase of INSIGHT to make the system even more intuitively usable D6 2 Version 1 0 31 08 2015 11 INSIGHT FP7 318225 The following sections introduce the AKNZ and describe the testing scenario development and evaluation during the summer academy in detail 3 1 The AKNZ The AKNZ is the central educational institution of the federal state regarding risk and crisis management and civil protection The Federal Ministry of the Interior has commissioned it to develop itself further into
6. discovery by interacting directly with citizens INSIGHT FP7 318225 2 2 2 Twitter Event Detection The DCC team examined independently a complete list of Tweets that the INSIGHT Twitter ISA identified as problematic Traffic related or Flood or Fire related The Twitter event detection has been an excellent feature to INSIGHT With some additional negative keyword lists such as counties outside of the Greater Dublin Area e g Cork Limerick Galway the relevancy of the alerts will be more accurate The partner responsible for the Twitter ISA for the DCC case implemented this black list feature as a filtering step Table 2 Information on the accuracy of the Twitter ISA Number of Tweets identified as Relevant of Confirmed Tweets with Relevant Text 91 163 tweets of Confirmed Tweets with Relevant Text and Useful 63 113 tweets Location in or near Dublin Examples Some examples of Tweets that were identified correctly during the evaluation period are the following 10 58 Did you know y all flooded Palmerston Road with ya shifty workmanship 15 01 t sin the fast lane Fire brigade amp ambulance on the way towards it Just passed incident Condition are poor amp wet 15 12 There is debris on the M11 Dublin Wexford Rd southbound between J14 Coyne s Cross and J15 Glendalough blocking the left pane 16 20 Car smashed the barrier southbound n11 Large tailbacks down to 1 lane
7. found from INSIGHT by considering aggregated information from the ISA Anomalies Live Bus Real time location of delayed buses Live Scats SCATS sensors reporting high degree of saturation during their last measurement It is recommended that at every time only one layer should be used The interface has two options for displaying information on the map Icons and Polygons At any time both the options could be used The Icons option provides a visual representation of the event or anomaly type while the Polygon option depicts the affected area Icons Polygons IMPORTANT Any change in settings is applied on the next screen update This is usually every 15 seconds D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 Task System Description Please provide the following details 1 Screen Size 2 Screen resolution 3 CPU i3 i5 i7 or AMD 4 Available RAM e g 4GB Task Usability Questions V1 Do you find it easy to navigate inside the map If No please comment Yes Comment V2 Does the refresh rate provide a smooth experience on your browser Yes No V3 Do the default settings provide you a smooth and not overloaded with icons interface Yes No V4 Does the interface display correctly on your screen size For example no overflows or overlapping buttons Yes No V5 Do you find the plot panel easy to use If no please provide comments Yes Comment V6 Do you find the help pa
8. icons are again used in order to send immediate feedback to the INSIGHT database about the event Important notes 1 The x mark refers to the panel the scats information are presented and not to the browser x mark 2 Please evaluate only the appear on the screen probably the Live SCATS layer is open too In this case please disable the Live SCATS layer icons If red scats icons The required information is the following 1 Anomaly ID AID D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 e S2 Isthe road congested Relevancy Please provide the significance of the con gestion 1 No Congestion 5 Heavy Congestion e S3 Was congestion anomaly captured by other conventional means from DCC cameras SCATS etc Novelty no scats cctv etc e S4 Other comments 4 6 Evaluation of the Crowdsourcing Component Functionality The Crowdsourcing component runs on mobile devices through the CrowdAlert application in order to display ongoing events found by INSIGHT to the end user and to extract information from the users The goal of its evaluation is i to ensure the integrity of the app with the web interface of Insight ii to ensure that it is fault tolerant and easy to use iii to evaluate the usefulness of the crowdsourcing tasks and ix to ensure that the app works according to the defined user settings Evaluation windows The traffic inspectors should interact with the INSIGHT mobile application using 30
9. minute windows and test its functionality 09 00 09 30 13 00 13 30 and 22 00 22 30 or the latest someone is in the traffic control room For both evaluation days DCC please define different and more appropriate evaluation windows Tasks T1 The inspector at random times should check the integrity of the events among components e C1 Functionality Are there any events showing in the map yes no C2 Similarity Are they the same with the ones illustrated in the Web Interface yes no e C3 Delay Note that due to the refresh rate there might be a small delay among the appearances of the events Is the delay acceptable yes no D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 ppt 2 RT Events y ISA Anomalies M z en INSIGHT N Live Bus C1 Functionality C2 Similarity C3 Delay T2 The inspector at random times eg every 30 minutes should check the functionality of the application and reportits behavior e C4 Time Date and time of the evaluation of the functionality e C5 Function What was the evaluated function eg Login Report Information Answer Task etc C6 Behavior Was the behavior as expected yes no C7 Other comments please include smartphone model as well T3 For each task that arrives at the crowdsourcing application the inspector should evaluate the following e C8 Relevancy Was the question relevant to an ongoing event at the area 1 Not Relevant 5 Relevant
10. only 16 23 Long delays on N5 between Wesport amp Castlebar in Mayo Car accident Emergency services at scene 16 27 hugh I turned right just after St Peters church and then left and kept going straight and very little traffic 16 34 bad news very very very very heavy traffic 16 42 Sorry folks I m late to the rosannadavison party Held up in traffic There was a gluten spillage at the Port Tunnel 17 06 Traffic is gridlocked on Howth Rd Malahide Rd amp Clontarf Rd better off staying at work a bit longer Dublin traffic Of course there were cases that the Twitter ISA misclassified some tweets 16 18 Dad just told me to go and play with the traffic 2 2 3 BUS Anomaly Detection The Bus Anomaly Detection proved useful in giving users information related to bus anomalies It proved more difficult to confirm anomalies using CCTV as by the time users tried to confirm the anomaly the time may have lapsed in some cases so verifying these anomalies was more difficult that the other anomalies The information given with each bus anomaly was very useful in trying to isolate the location of the anomaly in particular the link view the bus stops as this allows the user to know the direction ofthe bus and the current and next stop due Generally users found using the Live Bus layer to assist with the bus anomaly layer helped to understand if there was issues of congestion occurring in a certain area of t
11. the same days For each tweet the following information will be required to be filled by DCC T1 Was this tweet relevant yes no T2 Was this event captured by other conventional means from DCC cameras SCATS etc no scats cctv etc T3 In case the tweetis relevant did the incident appear in the LiveDrive stream UoA will provide the LiveDrive tweets as well yes no T4 Other comments free form text Text of Tweet T1 Relevancy T2 Novelty T3 LiveDrive T4 Comments 4 4 Evaluation of the BUS ISA Functionality The Bus ISA identifies traffic events using the GPS data ofthe Dublin buses The goal of this evaluation is the check if the alarms of the Bus ISA correspond to real traffic congestion events Evaluation windows 30 minute windows D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 09 00 09 30 13 00 13 30 and 22 00 22 30 or the latest someone is in the traffic control room For both evaluation days DCC please define different and more appropriate evaluation windows Instructions In a new INSIGHT page or after reloading an existing page enable the Layer ISA Anomalies and ensure the Icons option is enabled as show from the image bellow RT Events ISA Anomalies Live Bus Live Scats data size 10 7 Last Anomaly 8 24 2015 12 37 30 PM Bang Polygons Cams Sensors Cluster Task During the above evaluation windows the DCC operators should look at the
12. used more than cell phones events Tweets were useful to prepare on future situations Using Twitter to inform the public was very helpful The Dashboard was useful to verify the information given by the administration board The Dashboard should fill multiple monitors map is to small Location of events helpful to get an orientation within an unknown city Not clear when is event defined as critical Itis necessary to have one operator for the Dashboard To many tweets Need more time to get comfortable with the Dashboard some functions are not found Sometime we get the information from Insight faster than from our administration center Insight Dashboard could be very helpful for any kind of disaster scenario We need Social Media It already exist Group B We used the dashboard only at starting time later very rare To less information on this complex tool It was easy to locate the events Too much information Grouping the tweets on content could be helpful If we had use the Dashboard more maybe we found the dam break faster Twitter was very helpful A blackout is an inconvenient scenario for the Dashboard Sometimes we didn t know how reliable is the information All information on one map is very helpful D6 2 Version 1 0 31 08 2015 22 INSIGHT FP7 318225 3 8 Hypotheses Testing With regards to the questions raised at the beginning of this section following the answe
13. wo diese eingesetzt werden k nnten IKRY T Emscher LAGEMELDUNG POLDERGEBIET Gesellschaft Die Lage im Poldergebiet hat sich dramatisch verschlimmert Durch den Anstieg des Grundwasserspiegels und den anhaltenden Regen kann das Wasser kaum abflie en erste Keller im Bereich Duisburg Beeckerwerth sind vollgelaufen 3 5 Scenario data Blackout The Duisburg blackout starts in the south and rolls over the city to the north of Duisburg The mobile network ISA detects anomalies within the mobile network data stream to detect unexpected drop outs or abnormal low activities for a given location time and contextual information such as neighboring cells The blackout starts in the south of Duisburg and reached the north Car accident The car accident on the BAB40 caused a traffic jam in western direction All mobile network data used originated from traffic jams on BAB 40 Autobahn For traffic D6 2 Version 1 0 31 08 2015 17 INSIGHT FP7 318225 jams the Mobile Network ISA provides the event abnormal high activity along the BAB40 Twitter message contain information about an accident and massive traffic jam are collected Crowdsourcing During such a disaster scenario digitally linked emergency personnel becomes an important source of information Via crowdsourcing they are connected to the INSIGHT system to provide real time insights into local situations 3 6 Implementation Timeline Event Details ROS lat ion detail
14. 1 08 2015 DCC s personnel are automatically notified through the INSIGHT interface Traffic Alert is raised when INSIGHT identifies abnormal traffic behaviour in the area INSIGHT detects congested detectors and automatically plots them on a Map INSIGHT detects delayed buses and plots them on a map in near real time INSIGHT detects flood events using social media feeds and plots their location on a map INSIGHT detects fire events using social media feeds and plots their location on a map Citizens can report an event directly from their phone by using the CrowdAlert app INSIGHT detects an emergency event and seeks further information through the CrowdAlert app INSIGHT FP7 318225 Using INSIGHT DCC personnel identifies traffic incidents faster In addition DCC can identify events that could not identify with conventional means Automated visual alert on map quickly showing operators where congestion is occurring INSIGHT visually alerts the TMC to areas of congestion that are impacting public transport INSIGHT visually alerts the TMC to flood related events that the would not normally have a view of INSIGHT visually alerts the TMC to fire related events that they would not normally have a view of Incidents reported by citizens using the CrowdAlert app are displayed on a map in the TMC This feature complements the existing methods as well as contributing to knowledge
15. 25 the system All selected participants were given an in depth explanation about the functionalities of the prototype in order to prepare the evaluation exercise Blackout scenario Q INSIGHT services Sensors Crisis Situation Room Closed me Communities 15 participants 4 Control room Crowd 4 experts Social Data Figure 13 General concept setup of the evaluation exercise Sensor data from mobile networks and social media was available to the INSIGHT prototype No crowdsourced data was available During the evaluation exercise field experts in the control room overlooked the scenario and steered particular information as a cloud of experts via the crowdsourcing capacities of the INSIGHT system They also analyzed the real data stream and steered additional information from other organizations in the field e g police fire brigades rescue services using the administration area of the INSIGHT system designed to assist exercises and the evaluation phase In the crisis situation room the information officers were in charge of the INSIGHT system to collect distribute and refine information on the blackout situation In total the INSIGHT system run on six computers one for each control room one for each exercise room and one for an INSIGHT member accompanying the exercise During the exercise a staff of INSIGHT members Fraunhofer BBK assisted all participants and ensured a smooth process Additionally they collected first feedb
16. FP7 318225 D6 2 Implementation of Scenarios Validation and Assessment Report Lead Partner Fraunhofer UoA Contributing Partners IBM TUD Technion DCC BBK Status Final Scheduled Delivery Date 31 08 2015 INSIGHT FP7 318225 Executive Summary This document describes the evaluation of the INSIGHT system in the two real word use cases city wide nation wide and scenarios Two prototypes instantiations of the IN SIGHT system based on different requirements and data sources were launched and tested during real exercises at Dublin City Council DCC and at the Federal Office of Civil Protec tion and Disaster Assistance BBK The evaluation involved field professionals and training personnel and was operated as an implementation of the two scenarios Extensive feedback was collected during the preparation of the evaluation and conclusively through a question naire and interviews with key participants DCC employees BBK officers The evaluation helped the consortium not only to test the system in a realistic setting but also improve various aspects ofthe system and its methodologies A wide range of criteria were recorded discussed and considered for improvingthe system Close collaboration throughout the pro cess was necessary and supported many direct uptakes and continuous improvements Functionality how useful the system is and Usability how easy the system is to use were the main points of this evaluation INSIGHT in te
17. Group A was way ahead in treating the scenario In general Both groups described INSIGHT as a very helpful tool The Twitter ISA could provide lot of valuable real time information The evaluation shows that the benefits of INSIGHT are actively visible Below a summary of how the two groups utilized the INSIGHT platform is presented Group A e The group decided to usea projector and share the INSIGHT output on a wall in their control center e The group decided to assign its own INSIGHT operator His task was to collect all provided information of the Insight Dashboard and present the result during the team conferences e The group used INSIGHT as a communication platform and provided multiple Tweet messages to inform the public over Twitter Group B e Initially relied entirely on INSIGHT strong loss of relevance because of missing internal information from data sources such as police local authorities situation managers e Did not digital adapt all possibilities did not use a projector e INSIGHT became a source of relevant information during team conferences aside from other sources of official information e Was not used for early detection because not all events have been marked as potentially dangerous improvement for INSIGHT possible 3 7 2 Usage and Usability Evaluation Another goal of the evaluation within the BBK Use Case was to examine the usability and acceptance of the INSIGHT system and of the graphical user in
18. HT system offers to the personnel of Dublin City Council and the citizens of Dublin More specifically we provide information on how some certain tasks are being accomplished in DCC and then compare it with the corresponding INSIGHT functionality and what advantages this offers to the people of DCC D6 2 Version 1 0 31 08 2015 Table 1 Advantages of INSIGHT integration into DCC s workflow Conventional Approach INSIGHT Approach INSIGHT advantage Identify Traffic Incidents in the Streets Dublin Recognize individual congested sensors of Identify Buses that delayed Be about floods Be about fires Citizen are informed informed inform the city about any type emergency Citizens of are informed about any type emergency of Using primarily CCTV amp SCATS TMC operators actively seek areas requiring intervention It is quite a manually intensive task SCATS data needs to be manually checked at each junction Overall congestion is monitored on CCTV with no specific focus on buses The TMC use public information regarding weather alerts and have limited access to flood reports The TMC would not actively monitor alerts for fire events Citizens can call emergency services TMC AA Roadwatch LiveDrive as well as tweeting and e mail DCC inform citizens about emergency events through Twitter radio station AA roadwatch amp DCC website D6 2 Version 1 0 3
19. ION Nach Meldung von den Stadtwerken Duisburg besteht seit 15 55 Uhr ein Stromausfall im Duisburger Stadtgebiet Das komplette Duisburger Stadtgebiet ist vom Stromausfall betroffen Wie es zu dem Ausfall kommen konnte wird zur Zeit intern ermittelt LAGEMELDUNG Unfall BAB A40 nach Sturmschaden im Gerust in der Baustelle auf der RHEINBRUCKE Vollsperrung der BAB 40 in Fahrtrichtung MOERS in Vorbereitung Ruckstau bisher 2 km INFORMATION Alle stadtischen Krankenhauser melden die Aufnahme des Notstrombetriebes Eingeschrankte Patientenaufnahme m glich lediglich Kernbereiche sind voll funktionst chtig LAGEFORTSCHREIBUNG STROMAUSFALL Nach Rucksprache mit den Stadtwerken Duisburg besteht seit 15 45 Uhr nach Ausfall beider 220kV Zuleitungen der RWE ein Stromausfall im gesamten Duisburger Stadtgebiet LAGEERSTMELDUNG BAHN Information durch DB Betriebszentrale WEST Rund 1300 gestrandete Fahrgaste am HBF DUISBURG Auch Busbahnhof vor dem Duisburger HBF uberfullt Unter den Time Source Rs TUE Krisenstab IKILI LE SWDU IKS LIE Leitstelle IRETZTE Leitstelle IKS VALE Leitstelle IRJVA GE EA S d IKS yeI Leitstelle D6 2 Version 1 0 31 08 2015 Fahrg sten viele ltere Mitb rger NOTRUFMELDUNG DIALYSEZENTRUM WALSUM Poststra e 13 Aldenrade kann die Behandlung durch den Stromausfall nicht mehr 16 INSIGHT FP7 318225 aufrechterhalten Notstrom f r 1 5 Stunden sichergestellt 12 Patienten unmittelbar bet
20. Traffic Usage Assessment I used the following layers Usage Assessment Be le lue I used the following columns from the O events list 9 j i a Comments Questions D6 2 Version 1 0 31 08 2015 12 INSIGHT FP7 318225 Statistics 20 29 30 39 40 49 Nationalit Any questions Contact INSIGHT bbk bund de D6 2 Version 1 0 31 08 2015 13
21. a situation In addition they thought Twitter was useful for the Public Relations Section The INSIGHT system has proven its scalability functionality and operation in a real world scenario There is room for optimization mobile network events annotation event detection on unstructured data etc D6 2 Version 1 0 31 08 2015 24 INSIGHT FP7 318225 D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 Bene ee 4 APPENDIX A City Case Questionnaire NSIG HT i INSIGHT URL http 195 134 71 40 8000 html visualization CrowdAlert App lay google com store apps details id gr aueb boutsis crowdalert a Note All users that are about to complete the following questionnaire or part of it should first be introduced to the INSIGHT platform and its features 4 1 Goals of the Evaluation To assess the utility and usability of the INSIGHT platform for the people working in DCC 4 2 Evaluation Period The INSIGHT evaluation will start at and finish at e g a two days period 4 3 Evaluation of the Twitter ISA Functionality The Twitter ISA identifies tweets with content relevant to traffic or flood events The Goal is to check whether the Twitter ISA is able to return relevant content Task Offline evaluation UoA will provide DCC with the set of identified tweets at the end ofthe evaluation period in an Excel file so that DCC can check their relevancy and compare with the Radio Station tweets of
22. acks and recorded important information on usage usability and issues by observation At the end of the exercise all participants were asked to re assess and evaluate the exercise in general and information and communication aspects in particular The persons who worked with the prototype additionally filled out an evaluation form which was accompanied by an interview The evaluation scenario is described in the following section in more details 3 4 Scenario INSIGHT required a fully data driven emergency evaluation scenario This is the first time such a scenario had been developed The INSIGHT scenario build around a massive blackout scenario affecting large areas of Western Europe due to extreme weather conditions over Europe In order to demonstrate the scalability of INSIGHT from traffic to large scale events the scenario experts from the AKNZ and INSIGHT members decided to establish a crisis response and management scenario around the city of Duisburg 0 5M citizens live in Duisburg as of 2012 The city of Duisburg had been selected because of its geographical location in the center of the Rhine Metropole region with almost 10 Million citizens This ensured gt operability of the scenario The participants took the role of the operations staff of the city Duisburg and had the following tasks 1 Coordination of the work of the two TEL operational command post 2 Creation of a situation report 3 Advise to the Lord Mayor 4 Ensu
23. an educational institution for civil safety precaution with a national and international network within the framework of a strategic educational alliance The duties of the AKNZ include e Education and training for civil protection executives and teachers and for staff involved in matters of civil safety precaution e Seminars exercises and other events in the interest of the country s overall crisis management and of civil military cooperation Training events on behalf of the federal ministry Operation of a virtual learning platform Evaluation of mass casualty incidents at home and abroad Evaluation of national and international analyses publications and documentations Making available scientific support for research projects as well as the evaluation and implementation thereof Conduction of studies and investigations e Contributing towards the policy developments of the responsible top executive portfolio holders in the federal authorities assistance in federal and regional government commissions as well as in EU committees Topics of the education and training include e Civil safety precaution as an objective in the overall interest of the country Crisis management in the overall interest of the country administration and economy in particular Communication in the event of risk and crisis Leadership and staff training Risk analysis risk management Disaster medicine health care Self protection self help Civil mi
24. area The airport of Meknes is temporarily closed Spain Meanwhile storm Mary classified as a hurricane has reached the south east coast of Spain Its intensity has increased to wind speeds of 140 km h Mary is accompanied by heavy rain fall the temperature drops to 5 C Due to the weather condition smaller brooks and rivers overflow the banks some streets are flooded In some areas telecommunication systems and power supply fail Above all the cities of Granada Alicante Valencia and Barcelona are affected The airports of Almeria Murcia Alicante Valencia Reus and Barcelona stop their air traffic indefinitely Hundreds of flights are cancelled France Hurricane Mary gains in intensity On the border to France it reaches a maximum speed of 155 km h Besides the damage which has affected Spain the hurricane massively damages the infrastructure Belgium Germany Via Belgium and Germany Mary reaches the maximum speed of 170 km h First offshoots of the hurricane have reached the west of the Federal Republic of Germany In Baden Wurttemberg Rhineland Palatinate Saarland and Hesse massive storm losses occur Development of the special situation French and Spanish airports stop their air traffic indefinitely public transportation collapses Among others railway lines are damaged or blocked by uprooted trees and pylon Due to the heavy precipitation many city quarters are flooded numerous roof
25. area that offered additional steering capabilities and monitoring functions The AKNZ organizes the annual Summer Academy During the summer academy highly experienced experts in the field of disaster management recovery and safety meet at the AKNZ for expert exchange and international training seminars Professionals and participants from around the world apply for participation in this two week event The attendees participating atthe BBK Summer Academy have a background in civil protection and disaster management Part of the summer academy are two main table top exercises that last 1 to 2 days The INSIGHT prototype was deployed at the second table top exercise The INSIGHT system was embedded into the control room as well as the situation room The existing information technology and communication environment of the exercise rooms enabled a highly control testing environment Passive feedback was received via camera observation and monitoring by experts 3 3 Evaluation Setup The prototype was tested and evaluated on July 30th during the BBK Summer Academy at the Academy for Crisis Management Emergency Planning and Civil Protection AKNZ The test took place during a table top exercise as part of the international summer academy 2015 In close collaboration with the organizing experts of the academy and scenario experts the decision was made to run an extreme weather combined with a blackout scenario as the main emergency combined with
26. e C9 Novelty Was the event already captured by other means in DCC or has the same task already been received yes no e C10 Importance Was the question important for describing the event more accu rately yes no e C11 Other comments C8 Relevancy C9 Novelty C10 Importance C11 Comments D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 T4 The inspector should adjust the values in the Settings menu and report the following for each incoming crowdsourcing task C12 Distance Was the task located within the defined distance among the user and the task yes no C13 Number of Tasks Did the amount of received tasks exceeded the defined daily limit yes no C14 Mute Did you receive a task while being in mute mode yes no C12 Distance C13 Number of Tasks C14 Mute 4 7 User Interface and Visualization Functionality The Interface component displays the events found by INSIGHT to the end user The goal of its evaluation is to ensure that it provides a pleasant way to view the events in Dublin city even in a laptop display Evaluation Windows 30 minutes x 3 times duringthex 2 days ofthe evaluation Task During the evaluation of the system the traffic inspectors should interact with the INSIGHT interface The main goal is to understand if the interface provides a pleasant user experience Instructions The interface provides 4 layers or visualization 1 Zi ISA Anomalies RT Events Events
27. elt that more clarity was needed over exact location of events or more clarity on the questions being asked of users Due to the way the Insight system is designed it aggregates several events in adjacent geographic locations to issue crowdsourcing tasks and as a result CrowdAlert receives the approximate location of the greater area where the events take place rather than their exact location We are working toward supplying the exact location of the events when propagating questions to the users CrowdAlert has been downloaded and is currently actively being utilized by 64 individual users since it has been uploaded on Google Play and by 46 users since it went public to Google Play In the following figures we present several statistics about the number of users actively using the app as well as information related to the Android versions they use and their devices As can be observed all users have an Android version higher than 4 0 which is the minimum required version for the app and the majority of the users more than 55 have Android 5 0 or higher Figure 6 Active Devices D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 anwowte FT FTE D Anand 4 t D Andres 4 9 Figure 7 Installs by Android Version CURRENT INSTALL ES Er DE DEVICE A N rag ary 84 A E kerag Giry 4 U Geog n 5 am E terag abus AS E terag Giry 54 E berag Giy e D geryg Guy H D He HAAN GT D germ en He E peryg Gary Hi E Hn Fi
28. ents a heat map function colorized the affected area based on number and severity of events see Figure 15 e Timeline shows the stream of events and the occurrence of relevant events including event details see Figure 16 e Ontology uses the INSIGHT ontology to allow special filtering and organization of events e g into car accident and natural disaster events or blackout events see Figure 17 e Event list holds all event information to provide as much information as possible to the human experts without overloading detected events marked as critical semantically enriched are loaded into a special section of the event list this allows to focus on relevant events e Event details hold information on a particular event such as duration severity message text for social data context e Navigation to assist jumping between places or searching for locations e Layers are preset map layers for streets traffic using google traffic information events D6 2 Version 1 0 31 08 2015 18 INSIGHT FP7 318225 Figure 15 Map Zooming and shifting the map reduced or expanded the selected events Timeline 2015 07 30 09 00 WEIGHT DURATION 3 4 L d d q Figure 16 Timeline Only events with a starting time inside the selected time range are shown Ontology Natural Disaster Geophysical Disaster Biological Disaster Meteorological Disaster Hydrological Disaster Climatological Disaster
29. event identification traffic estimation and prediction etc These applications involve the effective and efficient analysis of data streams coming from different sources SCATS sensors BUS GPS traces and even social media Twitter Allthese techniques are presented in published papers that can be found in conference proceedings or as journal articles For a full list of publications please see D7 2 We have provided summaries of those research efforts in deliverables D3 1 D3 2 D3 3 D4 1 D4 2 organized in Work Packages and Tasks Some of the above methodologies were implemented in INSIGHT s Intelligent Sensor Agents ISAs The ISAs have a data mining machine learning core but had to be trained and tuned on real DCC data and for the tasks that DCC was interested in For more details on the data and requirements of this use case please see Deliverable D5 1 D5 2 provides information on how these technologies were adopted into the INSIGHT ISAs 2 PartA City Wide Dublin City Council In the DCC use case personnel in the Traffic Management Centre were requested to evaluate the INSIGHT system to ascertain if the system could assist them in their everyday tasks The Traffic Management Centre TMC were eager to test the system as the opportunity to improve reaction times to events and to increase the amount of automated alerts was welcomed The overall reaction to the INSIGHT system was a positive one Objectives The purpose of the evaluatio
30. fic Figure 10 Users geographical distribution and type of reported events 2 3 Usability Overall users found the system and application user friendly and easy to navigate Refresh rate of the page was generally viewed as a smooth experience Users did not find the map overloaded with icons as the tabs are easy to turn on off as per user requirements The following figure presents a summary of the user answers to the questions related to the usability evaluation The yes answer was always the positive answer The questionnaire included 10 questions related to usability that the reader can find in Appendix A Usability Evaluation m Answered Yes Results v1 v2 V3 V4 V5 V6 v7 V8 v9 V10 Question Figure 11 10 usability attributes were evaluated V1 V10 Percentage of yeses is actually the percentage of users that were happy with that feature For all attributes that the results were not good the consortium took corrective actions D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 Some indicative comments positive and negative are the following Yes it s easy to navigate inside the map the pan and zoom in and out is very useful and makes navigation easy I find the refresh rate better in Chrome 44 0 2403 155 m than in IE 11 No i had to change to zoom level on the browser to see the layers clearly so they don t overlap but then if i need to see something clearly I need to increase the zo
31. fic in Dublin In those time windows people involved in the evaluation were invited to utilize the system and complete a number of tasks In each time window participants were organized in groups based on their role in the department and completed only a part of the evaluation For example people from the Traffic Management Centre Team worked on SCATS related events Participants from the Bus Priority Team evaluated the bus ISA and Live Bus layer of the system while the radio station monitored social media The following personnel were invited to participate in the evaluation of the system in the TMC 1 Traffic Management Personnel 12 LiveDrive Radio Station 5 _I Traffic ITS Officers 10 The following were invited to download and use the CrowdAlert application General Environment amp Transportation Personnel 229 via departmental e mail group LI IT Department Personnel 100 via departmental e mail group LI General Public via LiveDrive radio program LiveDrive traffic programme had an independent survey commissioned with over 81000 listeners tuned in the Dublin Metropolitan Area Twitter 13 9K followers amp blog post LJ DubLinked via Twitter and and website J Other LinkedIn posts tweets and re tweets ee FA a Ark is le PR ee Figure 2 The INSIGHT system during the evaluation D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 A user manual was created to assist users using the INSIGHT syste
32. gure 8 Installs by device Users had the chance to use test and rate our application Although only 4 users have provided feedback for the app they have given us a 5 star rating RATINGS amp REVIEWS OVERVIEW 0 00 k x xx xx Figure App Ratings at Google Store We also present results related to the Crowdsourcing tasks that have been sent to the users The following figures illustrate the location of the users that have provided responses to different tasks and we also present the corresponding answers As can be observed users from all over Dublin are using CrowdAlert and have provided responses to tasks and as the figures illustrate users participate especially from suburbs where no other sensors exist Moreover 41 9 of the questions issued by CrowdAlert regarding an event have been validated and 47 4 have been rejected Such a small difference is due to events that are difficult to be confidently captured without relying on the human factor D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 Crowdsourcing Tasks Figure 9 Geographical distribution of responses and percentage of validated events Finally we present the location as well as the type of reports that have been retrieved from the users Users mostly report either Traffic events or provide us with feedback that there is no ongoing event at that point User Reports Accident Constructions Earthquake Flood Hazard Major Event No Event Traf
33. he city D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 Examples An event occurred duringthe evaluation is displayed in the following figure The participants confirmed the correctness of information displayed by INSIGHT In addition they mentioned that INSIGHT is easier to operate manage and can aid in combining information sources Figure 4 Comparison of the Dublin Bus System left with INSIGHT right INSIGHT advantages are that a it displays information on an easy to use map b presents only relevant information delayed buses c the user can easily investigate information related to the particular bus and print plots of bus history c users can adjust the system as needed by tuning parameters d it is able to present multiple layers of information e it can be accessed from multiple and remote locations 2 2 4 SCATS Anomaly Detection The SCATS anomaly detection proved very useful in allowing users to detect an singular anomalies related to a junction The anomaly detection alerts from LE INSIGHT indicated individual lanes that had a high degree of saturation which would not be immediately apparent using the SCATS system Being able to use the INSIGHT map along with SCATS and CCTV proved useful to diagnose and confirm anomalies occurred at the location highlighted by the INSIGHT anomaly layer Examples A traffic incident occurred during the evaluation The DublinService announced the incident as follows Ex
34. he official announcement Unfortunately this was the only situation where Group B got aware of the situation before the official announcement Group A had a couple of situations where they got aware of the situation be fore the official announcement also because they used it more intensively than Group B gt AA A Es don LT as Figure 20 INSIGHT exercise at AKNZ Figure 21 INSIGHT interface on the big screen of the exercise room D6 2 Version 1 0 31 08 2015 23 INSIGHT FP7 318225 3 9 Conclusions The feedback of the professional experts as well as the participants was very positive They thought the INSIGHT system was very useful They also said that scenario was not perfectly fitting the goal The INSIGHT system fitted in the blackout scenario and showed that even during a blackout it was still possible to gain information from tweets and mobile phone usage Furthermore all the hypotheses tested positively which means that it was possible to use the INSIGHT system during the whole evaluation process It was possible for the participants to utilize the provided information and that the situational awareness of the participants increased Like any other tool the INSIGHT system requires some affinity to computers or more time Integration of more data sources is crucial Especially more official channels The tweets however they thought were very useful and resulted in the feeling that they became faster aware of
35. ing Framework at the AKNZ urnes 12 EVA OS CU da E 13 a4 CCE aa E EE ner 14 DS el 10 3 0 910 6 LS LES Denner ene rece te ee ere ee ee 17 OO OS AIO ee tenes re eae ee eee eee 18 3 7 Feedback collection and evaluation results sn 20 3 7 1 Evaluation Results side es eine 20 3 7 2 Usage anid Usability Evaluation A 21 3 7 3 PS AC 1 OS o a ae a ana 22 58 IlyDorBeses PSN ee nn ne 23 33 COCO euere 24 4 APPENDIX A City Case Questionnaire sisi 1 4 1 Goals of the Evaluation rennes 1 42 NeW AWA OR PCI E dan ee cn 1 4 3 Evaluat onof the Twitter BA uiid 1 4 4 Evaluation of the BUS ISA rennes 1 4 5 Evaluation OF the sCATS BA aa 3 4 6 Evaluation of the Crowdsourcing Component sn 5 4 7 User Interface and Visualization ussssssssssnesesesnnnnesnnnnnnnnnnnnnnnnnnennnnnnnnnnennnnnnnnnnnnnnnnnnennnnnnnnnnen 7 4 8 Overall Evaluation of the System inner 8 5 Appendix B Nation Wide Case Questionnaire nn 9 D7 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 D7 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 1 Introduction and Objectives The goal ofthe INSIGHT projectis to radically advance our ability of coping with emergency situations by developing innovative technologies and systems that will put new capabilities in the hands on disaster planners and city personnel to improve emergency planning and response During the three years of the project the consortium developed a number of methodologies for anomaly detection
36. litary cooperation Provision and safeguarding in areas of critical infrastructure Economy business Energy supply Transport Communications Food Water supply e Crisis management disaster relief on an international level EU NATO UN 3 2 Testing Framework at the AKNZ Learning how to manage emergency situations in digital societies requires special training facilities and experts with particular expertise The AKNZ as Germany s primary training and education center in civil protection and disaster management offers the facilities staff and equipment for most realistic table top exercises This provides an ideal physical framework for the evaluation of the INSIGHT prototype D6 2 Version 1 0 31 08 2015 12 INSIGHT FP7 318225 Experienced scenario architects at the AKNZ supported the INSIGHT staff in designing and building a realistic test scenario It was a novelty to design a fully data driven table top exercise that suit the requirements of the AKNZ e Operability a scenario must be feasible and actable e Consistency a scenario must be logically linked and complete e Flexibility a scenario must allow individual adaptation during the exercise to reflect decisions taken e Relevance a scenario must be relevant in terms of the overall goal The unique setup needed for the evaluation of INSIGHT had to fulfill all these requirements Therefore the INSIGHT toolset had been greatly extended by an administration
37. lower level events e g car accident Our intention was that the participants would learn how to manage the consequences of a large scale blackout under severe conditions using novel information sources for situation awareness and decision support including mobile network sensor data as new source of overview data social media and crowdsourced data The target area was set to a large metropole in a densely populated area A city that fulfilled the expert requirements was Duisburg Prerequisite for selecting the blackout scenario in Duisburg has been the availability of real world data Therefore Twitter as well as mobile network sensor data had been analyzed to record blackouts in Germany since the start of the project in 2012 To prepare the evaluation the prototype was introduced during a lecture in advance of the exercise The lecture included live demonstrations of the system as well as a thorough introduction to digitalization in disaster response The general setup of the evaluation is depicted in Figure 12 The participants were split into two groups of 15 members This parallel setup was chosen to ensure a maximum of reliability and significance of the evaluation results Each group setup has been an instance of the general setup consisting of a control staff of experts participants and INSIGHT staff members Each group selected one or more persons who would be responsible to operate D6 2 Version 1 0 31 08 2015 13 INSIGHT FP7 3182
38. m The manual can found here http insisht ict eu CrowdAlertApp pdf The INSIGHT System has also its own manual http insisht ict eu user guide pdf Dublin City Council also produced a video series to assist the Traffic Management Centre in using the different features available in the INSIGHT system These videos can be viewed on our INSIGHT Youtube Channel https www youtube com channel UCORvM9dn5fz6ZQoEtg gIzg All activities v gt INSIGHT Project uploaded a video CrowdAlertApp by INSIGHT Project x INSIGHT Project uploaded a video Jay ago INSIGHT 11 AdditionalLayers by INSIGHT Project 1 day ago 11 views Figure 3 INSIGHT S youtube channel We suggested the personnel involved in the evaluation to either watch the videos or read the INSIGHT user guide Most users stated that they could understand the main features of INSIGHT faster by watching the video instead of reading the manual The consortium intends to use the videos as a dissemination and an education tool After the completion of the evaluation process the participants were asked to answer the questionnaire presented in Appendix A 2 2 Functionality In this section we provide the evaluation of INSIGHT main capabilities More specifically we investigate how helpful and effective these features are in a DCC s workflow b Dublin s citizens 2 2 1 Overall INSIGHT benefits to DCC The following table provides an overview of what the INSIG
39. m intensively at the beginning until they realized that they did not receive all information via INSIGHT Other information via fax email or phone was initially ignored The attention dropped significantly after this had been noticed and the focus went towards internal information sources 2nd finding The information aggregation by system is ideally for gaining a quick overview of the situation But it is important to have digital or classical means to share this information Having only a laptop is hindering the uptake of information by the entire group as it is not visible Only one group decided for themselves to define an INSIGHT Operator and present the INSIGHT dashboard using a projector INSIGHT was more accepted than expected by group A D6 2 Version 1 0 31 08 2015 20 INSIGHT FP7 318225 Zrd finding The capabilities to retrieve information from data streams produced by the public also encourages to send information back to the public One group used the system as a channel to send information to the public via Twitter They also used the crowdsourcing capabilities to coordinate the voluntaries by sending them designated messages and task descriptions 4th finding Group A installed an INSIGHT officer Continuously monitoring the INSIGHT event stream enabled them to become aware of certain situations way before officially recognized This allowed this group to react upon this knowledge and gain a temporal advantage At the end
40. n is to test the INSIGHT system in real circumstances and evaluate all of its crucial elements 1 The three ISAs Twitter Bus SCATs 2 The mobile application CrowdAlert 3 Event detection capabilities 4 Visualization Interface 5 Other secondary features e g plots of historical data The evaluation was set up in such way that we evaluate the above features in terms of a functionality and b usability Simply put functionality relates to how the INSIGHT system aids DCC employees achieving more than when using their conventional workflow Usability relates to how easy to use and intuitive the system was Note that the majority 95 of participants were not familiar with the system A Training Step preceded the evaluation by providing the participants a user guide and a set of video tutorials 2 1 Evaluation Protocol Relevant DCC personnel were invited to participate in the evaluation ofthe INSIGHT system Using the Traffic Management Centre and the adjoining Local Incident Room as a central hub allowed staff to evaluate the system all together seeking clarification by asking questions and attending to concerns that may have arisen The evaluation was operated in two 2 hour time windows steps and in two different days The time windows were LI Day 1 08 00 10 00 16 00 18 00 and D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 LI Day2 09 00 11 00 15 00 17 00 These are typically periods of high volume of traf
41. nel useful If no please comment on information you would like to have on this panel Yes No V7 Do you find the information presented when the user clicks to an icon enough for the description of an anomaly or an event If no please comment about extra details you would like to have Yes Comment V8 Do you think that the data layers offered RT Events ISA Anomalies are enough If not what is the layer you think the system is missing Yes Comment V9 Do you find the settings panel enough in order to customize your experience If no please include any settings suggestions that you may have Yes Comment V10 Do you find the icons used in the visualization system exploratory for the event or the anomaly they depict e g A bus icon for the Bus ISA etc If no please provide alternatives Yes Comment 4 8 Overall Evaluation of the System Task Please provide your overall assessment of the usability and utility of the INSIGHT platform D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 5 Appendix B Nation Wide Case Questionnaire Evaluation AKNZ FR Bundesamt AN f r Bev lkerungsschutz Please leave some time to und Katastrophenhilfe answer the following questions Thank you for your feedback INSIGHT feedback collection You will help us to get some information about the use of the INSIGHT system Visualization and Usability fully mostly partially do not completely agree agree disagree agree
42. om level again to read text Yes but by default it has a dash on the ID fields so if you are searching you would always have to delete this 2 4 Conclusions In general this feedback loop was very useful since INSIGHT was tested in real circumstances The participants were very happy with the system and the mobile application They declared that INSIGHT is a very useful asset and that it would help improving their workflow The consortium received many comments and took actions in order to fix all issues inconsistencies and bugs D6 2 Version 1 0 31 08 2015 10 INSIGHT FP7 318225 3 PartB Nation Wide Use Case BBK AKNZ The second part of the INSIGHT testing scheme aims to evaluate the nation wide usability and demonstrate the scalability of the INSIGHT system towards nation wide emergency situations During the past 3 years the BBK as one of the two end users provided continuous input and valuable guidelines for adjusting the system for the needs of professionals in the case of large scale emergencies This includes detection and data visualization as well as many usability features The main challenges in testing the second use case are twofold Plannability it would require an emergency situation of nation wide magnitude Liability the staff of the BBK has to react upon risks and individual threats and public needs as soon as they become known As this is possible for lower level events like accidents or traffic jam
43. pect Delays amp Diversions to routes 46A 63 amp 75 due to an incident on York Road Dun Laoghaire See www dublinbus ie for updates INSIGHT identified the event on the Live Bus and the Live SCATS layer as seen in the following Figures Once the notification was received from Dublin Bus it was clear on INSIGHT that there was traffic issues experienced on York Road with the Live SCATS anomalies and then roads around York Road having alerts for Liver Bus due to the diversions put in place D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 Sensor 1394 Detector 6 Saturation 190 Traffic Flow 8 Lat 53 2962Lon 6 14214469920241 31d 1394 6 Vehicle 44015 Lineld 7 Delay 726 Congestion false Speed 9 At Stop false Direction 0 Generate Plots ay he Villy Figure 5 Live SCATS and Live Bus reporting on the same incident 2 2 5 Round Table Events The degree of saturation DS threshold for SCATS alerts may need to be raised While events being raised by the system are correct and there is a higher than normal DS the congestion is below the threshold that would require intervention by traffic control operators in the TMC DCC talked to the responsible partners and decided to increase the corresponding threshold 2 2 6 CrowdAlert The crowdsourcing application was very easy to use The buttons make it very easy to report an event and view it immediately on the map display With relation we to users replying to alerts it was f
44. ring information about the situation to KGS crisis staff and 5 Processing the damage situation This task D6 2 Version 1 0 31 08 2015 14 INSIGHT FP7 318225 driven setup ensured gt relevance as it illuminates the overall objectives for the crisis managers The scenario was steered by the experts in the control room to ensure gt flexibility The experts influenced the course of action of each scenario individually to the reactions taken by the crisis management group In order to embed the scenario and logically link all following events a scenario storyboard was produced This ensured gt consistency of the scenario as well as a clear orientation on the real world data This included the following introduction to the scenario For days the whole of north and central Germany has been under the influence of a low pressure area which brings wet and cold weather with occasional heavy precipitation and high wind speeds Morocco On 11 November 2008 meteorological institutes issue more and more reports about occasional storms west of North Africa and across Morocco which are expected to move in the direction of Western Europe Meteorologists hardly notice that the storm increases in speed and intensity On 12 November it arrives in the Moroccan mainland at 115km h The City of Meknes 500 000 inhabitants is mainly affected and must look after about 30 000 directly affected people who live in the city centre and the surrounding
45. rms of the underlying Big Data architecture is evaluated in Deliverable D2 3 D7 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 Document Information Full title Intelligent Synthesis and Real time Response using Massive Streaming of Heterogeneous Data Project URL URL http www http wwwinsight icteu OO ict eu EU Project officer Ms Alina Lupu Deliverable Implementation of Scenarios Validation and Assessment 31 08 2015 es Immo Distribution Public Type Authoring Fraunhofer UoA Partner Contact Person Hendrik Stange Ioannis Katakis Sylvia Steenhoek Stephen Lynch Dimitrios Gunopulos hendrik stange iais fraunhofer de katak di uoa gr stephen lynch dublincity ie dg di uoa gr insight bbk bund de D7 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 D7 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 Table of Contents 1 HMHilr duetlonand OCCU VS a einigen 1 2 Part A City Wide Dublin City Council sin 1 21 PR GO PE OO ee ent 1 Do WACO Ay atest a le a ee nee 3 2 2 1 Overall INSIGHT benefits to DCC s ssesssressrssrrssrensnensnnnrnesnrnsnronnnsnrnsnnnsnnonnnnnrennrnnnnsnnnna 3 222 Twitter Event Delete on ze ee 5 2 2 3 BUS Anomaly Detection uses 5 2 2 4 SCATS Anomaly Detection 32 2 een 6 2 2 5 Round Fable Evers nenne ae ee 7 2 2 6 COR euere rue 7 ZI SAD I Ve ee a ae unsre 9 24 CO A OO en een net eee 10 3 PartB Nation Wide Use Case BBK AKNZ mures 11 31 TheAKNZ nenne tie 12 3 2 Test
46. roffen Transport ins Krankenhaus dringend erforderlich IKIE YA EA Sud ANFORDERUNG Lokale berschwemmung durch Starkniederschl ge Kanalisation U Bahn Sch chte im Bereich U Bahnhof K NIG HEINRICH PLATZ und STEINSCHE GASSE Altstadt Ben tigen weitere Pumpen IR EYE Leitstelle LAGEMELDUNG Mobilfunknetzbetreiber melden einen Teilausfall der Mobilfunknetze seit 16 30 Uhr Nach Wiederanfahren des Stroms k nnte das Mobilfunknetz innerhalb von 2 Stunden wieder mit voller Kapazit t betrieben werden IEJILIEN Leitstelle NOTRUFMELDUNG Unfall am Containerterminal Hafen Absturz eines Containers w hrend Anliftvorgang eines Containerentladungskrans ALTE RUHRORTER STR EIKE EA Sud LAGEFORTSCHREIBUNG BAB 40 Die Bergung an der Unfallstelle wird sich voraussichtlich um zwei Stunden verz gern und um ca 00 00 Uhr abgeschlossen sein Verkehr muss auf der BAB zur ckgef hrt werden IRVA NIE Leitstelle NOTRUFMELDUNG Brand in Rheinhausenhalle Beethovenstrasse Rheinhausen Mitte Charity Gala Dolphin Care betroffen Anscheinend ist auch Schirmherr Dr Bertrand Luxemburgischer Botschafter in Deutschland anwesend IEYAETE Leitstelle INFORMATION Der Direktor Wolfram Ullrich der Universit t Duisburg Essen meldet dass sich 600 Participanten an der Fakult t f r Maschinenbau und Energietechnik Lotharstr 1 21 47057 Duisburg als freiwillige Helfer eingefunden haben Diese st nden zur Hilfeleistung bereit Er fragt an ob und
47. rs based on a consolidation of all feedbacks and observations made by the professional experts of the BBK as the experienced end user H1 Participants used the prototype during the entire exercise This hypothesis tested positively Both groups used the prototype as an information source during the entire exercise However in Group B only one participant monitored the prototype where in Group the prototype was visible on the wall in a way the whole group was able to see it Furthermore Group A used the prototype more at the beginning of the exercise than at the end of the exercise but Group B explored the features of the prototype more during the exercise and even started to use the crowdsourcing function of the prototype H2 Participants implemented the information given by the prototype This hypothesis tested positively In both groups the information was used to inform the rest of the group and the chief of staff about the current situation However halfway Group B did not value the information as much as they used to do at the beginning of the exercise Group A valued the information even more during the exercise and started to feed the prototype with their information by using the crowdsourcing function H3 Participants became earlier aware of the situation because of the prototype This hypothesis tested positively Group B got aware of a flooded area because of a tweet that popped up in the prototype this awareness came before t
48. s 5 22919 51 422105 6 759537 Vielen Dank Bitte nicht l schen duration 120 lat lon 51 411060 6 883121 weight Kassel i fr 7 1 ve olautzen OT G any gt ARS M a h mn lte y 1 ze lat lon details 7 Twit 6 r J rec gt b ulda lt 3 2015 51422105 Vielen Dank Bitte nicht l schen 9 6 759537 5 514110607 Stromaustal in der Altstadt yon M lheim Ruhr Biode Frankfurt 6 583121 jJavarbeser haben mal wieder ein Kabel gekappt lt ich wollte w r Wiesb deng KS T Ma Leo T a P i 30 07 2015 51 238455 in D sseldorf Die Monitore bleiben Mannheim 3 1 32 6 814350 en 30072015 51440907 ADerwesten Stromausfall in essen holsterhausen zer Men See en N hd 60352 7 016018 stobbestr nieberdingstr wisst ihr etwas danke pp risruhe LA udejovice m m RN Da Kralmlon 4 30 072015 51 446658 je Stadtwerke_DU kan ded L J 5 6 3 6727975 sowas nicht machen wenn normale Mens a L i Passau J ite und da noch a Uh E 51 440007 Figure 14 The INSIGHT user interface dashboard for the second use case In this use case the INSIGHT interface was created to visualize a huge amount of events INSIGHT assists the operator to focus on the relevant subsample of events There are three different navigation filters to select the relevant events e Map visualized the events on a map using a dynamic clustering method which uses the zoom level to organize and cluster ev
49. s are unroofed entire wood land D6 2 Version 1 0 31 08 2015 15 INSIGHT FP7 318225 destroyed The airports of Cologne Bonn Dortmund D sseldorf M nster Osnabruck Bremen Hannover and Hamburg are closed indefinitely Arriving international fights are diverted to the airports in the south and east of the country The exercise starts when a severe storm hits Duisburg Within 30 minutes there is an announcement that there is a blackout throughout the entire city and the mobile phone network is only partially functioning This announcement is followed by an announcement about a car accident at the BAB 40 Rhine bridge which causes a massive traffic jam blocking a major highway from and to Duisburg Aside the blackout and traffic jam the participants need to solve problems caused by collapsed trees blocked streets flooded areas and blocked railway and subway stations The scenario storyboard also contained a logically linked and temporally ordered list of events Social and mobile network data has been selected from previous eventto reflect the emergency situation It is the task of the INSIGHT system to detect and visualized events that correspond to the described situation The following list of situation events are an excerpt of the complete list of events Content AUFTRAG bernehmen Sie die im Zuge des Unwetters n tigen Gefahrenabwehrmafsnahmen als Einsatzleitung Fuhrungsstab in der kreisfreien Stadt Duisburg INFORMAT
50. s it is relatively hard to do for emergencies affecting more than 100 000 people at the same time To account for these challenges and to broaden the scope of the testing under controlled conditions as well as to structure the evaluation of INSIGHT the decision was made to involve the central educational institution of the federal state AKNZ for disaster management in Germany The INSIGHT project was granted to get evaluated during the international summer academy of the BBK The summer academy is a world wide renowned training event and expert exchange on international level A key success measure is that all international professionals operate under most realistic circumstances The BBK takes the greatest efforts to make the trainings and exercises as close to reality as possible Objectives The purpose of the implementation and evaluation phase is to put the INSIGHT system into operation and evaluate all of its crucial functionalities 1 Data stream processing and analysis ISA Twitter Mobile Phone 2 Crowdsourcing functions closed group approach 3 Event recognition capabilities 4 Information provisioning 5 Operating the system The evaluation was set up in such way that we evaluate the objectives mentioned with respect to a functionality and b usability and c transferability Functionality relates to the systems capabilities and that the system works as designed this includes the main goal of INSIGHT to put professionals during
51. screen of INSIGHT and for every BUS anomaly delay marked by this icon note down the following information from the popup window that looks like this View the Bus Stops Generate Plots Date B 24 201 O0 Comments D6 2 Version 1 0 31 08 2015 INSIGHT FP7 318225 Please close the gsindow by clicking the x mark after 5 finish working with an anomaly The Approve Reject and Don t Know icons are used in order to send immediate feedback to the INSIGHT database about the event Important notes 1 The x mark refers to the panel the bus information are presented and not to the browser x mark 2 Please evaluate only the red bus icons If blue bus icons appear on the screen probably the Live Bus layer is open too In this case please disable the Live bus layer The required information is the following e B1 Anomaly ID AID e B2 Is the delay reported by INSIGHT actually a delay Relevancy Please provide the significance of the delay 1 No Delays 5 Heavy Delays e B3 Was the delay anomaly captured by other conventional means from DCC cam eras SCATS etc Novelty no scats cctv etc e B4 Was there also a traffic problem at that segment of the city it might be the case that a bus delays does not co occur with traffic incidents Traffic no yes e B5 Other comments B1 AID B2 Relevancy B3 Novelty B4 Traffic 4 5 Evaluation of the SCATS ISA Functionality
52. terface the INSIGHT user frontend Both groups had the same technical equipment and received the same introduction and no training on the system The simplistic design of the frontend allowed an easy navigation through the software Both groups had no issues to directly work with the system No assistance had been given during the exercise The following usage usability impacts have been monitored during the exercise Group A e INSIGHT was fully integrated into the control center steering room e Designated system operator in the controlling center e Additional critical events and Tweets were embedded during the training D6 2 Version 1 0 31 08 2015 21 INSIGHT FP7 318225 e No introduction or explanation of the INSIGHT system was given on the day of the exercise Group B e INSIGHT only presents the ISA output e No designated INSIGHT operator e No additional events or Tweets were generated e No introduction or explanation of INSIGHT was given on the day of the exercise 3 7 3 Feedback Collection After the eight hour training an open discussion with both groups was initiated Figure 18 shows the result of the discussion after the evaluation was finished INSIGHT plays a major role in information management A CARY r SHU Mey Figure 19 Feedback on Information Management after the evaluation scenario was finished Some original statement of the participants organized in groups are given below Group A Twitter information
53. tion in the events O 0 0 0 O list I clicked the refresh button regularly O O O O O I often changed the selection of the columns O O O O O I often used the lat lon function to locate an O 0 O 0 O event I changed the number of events on one page 0 0 0 O 0 regularly I often scrolled down on the events list O O O O O I often created new events on the map O O O O O I prioritised the events based on their weight I selected all the types of ontology during the O entire exercise I changed the types of ontology regularly I had a good overview of the situation O O O O O shown on the map There was too much information on the 0 0 0 O 0 map The peaks and dots on the timeline were 0 0 0 0 0 useful The feature to adjust the time frame was 0 0 0 O 0 useful The event markers were useful O 0 0 i D6 2 Version 1 0 31 08 2015 11 INSIGHT FP7 318225 The heat map was useful O O O O O The areas layer was useful O O O O O The traffic layer was useful O O O O O The navigator function was useful O O O O O It was useful to save locations O O O O O It was useful for the decision making to O O O O O see the location of the event It was useful for the decision making to O O O O O see the duration of the event It was useful for the decision making to O O O O O see the source of the event It was useful for the decision making to O O O O O see the type of the event The selection of the different ontologies 0 O 0 O 0 was useful Event
54. uch as information from within hospitals or other cultural facilities are hard to detect based on public data Automated alerting also works based on the severity level of events or the number of events in an area There multiple ways to inform the user heat map highlighting events adding to list of critical events special icons The results with respect to scalability are The goal was to scale from local events such as car accidents and traffic jams to emergency situations affecting gt 100 000 people at the same time With the Duisburg scenario local events traffic jam had been successfully integrated with large scale situations blackout Apparently combining events on different scales does not negatively affect readability nor usability of the system Key findings are 1st finding If INSIGHT is fully integrated into the information distribution channels the acceptance level of the entire system is greatly enhanced This became evident as one group also used the INSIGHT system to also integrate external information from police health or emergency organizations local authorities into the system Although this information did not directly come from mobile network or Twitter it indicated that having one central system for key information from public data streams and official information strengthens the relevance of the system and thereby increases the attention paid to the system in general Interestingly Group B was using the syste

INSIGHT

Contents

Download Pdf Manuals

Related Search

Related Contents