Integrated and Evaluated VIDI system & System Manual
Contents
1. 18 5 1 DATAJACOUISITION zs aeri EE d erue rure er dard koa Oca GE s UK a ages ORION de 18 5 2 DATA AGGREGATION AND AUGM ENTATION cene nennen nnn 18 5 3 ANALYTIC MODULES ze ERE a eate Rated o DEED Te 18 5 4 CLIENT SIDE AND CLIENT SERVER COM MUNICATION 19 6 id did ec E 20 APPENDIX A SOURCE CODE ORGANIZATION nnne 21 VIDI page 3 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 VIDI page 4 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 EXECUTIVE SUMMARY This document describes the VIDI toolbar i e the software component of the project The toolbar was developed following the VIDI software architecture proposed in D2 1 It includes a data acquisition component database access and Web crawling a basic data handling component data aggregation and data augmentation and analytic modules providing notifications browsing suggestions topical atlas and topical timeline The docume2. Events Regio Properies Properies Natural Feature lt 71 unusal false SG EMAIL testovidi eu Person Jeko Lx Complex Events similar Add Add Operator Node PAR NOT 218 INEPADiscussionForum Organization 10 5 Properies Gq organization Eu Parlamen Action Node Add Figure 8 Example of a notification pattern for the INEPA discussion forum VIDI page 15 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 The example in figure 8 shows a pattern that describes a notification pattern within the INEPA discussion forum where two members of European Parliament from Slovenia are mentioned in the European Parliament context VIDI page 16 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 4 LIVENETLIFE LiveNetLife is an existing software package which offers automatic connection establishment and real time chat between users who are browsing completely independent but topically related web pages As it was considered relevant to boosting e participation plans were made in VIDI deliverable D1 1 to adapt it and include it in the VIDI p
3. Mestsk ast Ko ice Sidis X D Evropske volitve 2009 X http flocal tedist 27 2F X Dr avljanski forum Evrops X C Jezik Language Sloven ina O Seznam forumov skrbiteza ohranjanje okolja ez SPLETNA ANKET v lRegistriraj Q Iskanje D Prijava Danes je 10 01 2010 16 42 Selected items Aktualno Kako ob utimo vpliv Neodgovorjene teme Aktivne teme Romane Jordan Cizelj ko bomo premagali podnebne spremembe Napisal a Ako a Kopenhagen Upanje Va a stali a in predlogi evropski poslanki dr Romani 29 12 2009 07 51 Jordan Cizelj podnebno konferenco Kopenhagnu Ln in okoljske politike parlamentu scum Moderatorsko poro ilo naj EU pozitivno diskriminira pri zaposlovanju Napisal a Drago Stanovnik a spletnega posveta a i predlogi evropskemu poslancu Jelku 02 12 2009 09 26 glede uvajanja pozitivne diskriminacije za iskalce prve zaposlitve in skrbi za trajno socialno varnost v EU Status Connecting bomo premagal podnebne spremembe Zadnji video iri akujemo od Evropskega parlamenta Napisal a Gostja 3 pri akovanja in vsebinski predlogi s katerimi bi 12 11 2009 14 50 morali evropski poslanci ke ukvarjati v mandatu b utimo vpliv EP na na e ivljenj
4. eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 3 USER MANUAL 3 1 Getting Started To use the VIDI toolbar navigate to a VIDI supported forum In the scope of this project the following forums have been supported e Evropske volitve Slovene http www evropske volitve si e MC KoSice Sidlisko Tahanovce Slovak http mutah tahanovce sk 8080 mutah web sk forums jsp id 50049 e Political part of index hu Hungarian http forum index hu Topic showTopicList t 9111313 If the forum owner has installed the toolbar forum wide it will appear unobtrusively hidden in the left border of the page as shown in Figure 3 Clicking on the blue handle expands the toolbar making it ready for use If the toolbar does not appear automatically not even in the hidden state the forum owner has most likely not installed it Please follow instructions in Section 2 to create a bookmark button in your own browser Clicking the bookmark button will make the toolbar appear in expanded state Dr avljanski forum Evropske razprave Prva stran Mozilla Firefox BAX Ele Edit View History Bookmarks Tools m e S 5 htto vw evropske volitve si SB Goode P G Ri Most visited Inspectorwidget vidi C vidi 85 X Disable Cookies v fp CSS
5. zotero Figure 5 Browsing suggestions the relevant links are marked by orange icons their size proportional to link relevance 3 4 Using the Topical Atlas To see an atlas of all posts in a chosen subpart of the forum as illustrated in Figure 6 first indicate your interest by selecting several threads as described in section 3 2 Then press the Atlas button on the VIDI toolbar The atlas appears in the middle of the window Calculating all the data needed to display the atlas can take several minutes so please be patient To discard the atlas click anywhere outside the popup area VIDI page 12 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 Dr avljanski forum Evropske razprave Prva stran Mozilla Firefox Jee File Edit View History Bookmarks Tools Help lt gt 0 htto inww evronske voitve si Me ee Ri Most visited vidi vidiss 7 VIDI toolbar X disable 8 Cookies P CSS v Forms g Images 4 Information E Miscellaneous Outine Resize D Tools View Source Fi Options OGLDF PK summary X ff OGLDF 0 606 0 07 X Mestsk ast Ko ice X Evropske volitve 200 X http foc Lst 27 2F_X D Dr avljanski forum X expand textbox to fi
6. X a v Jezik Language Sloven ina O Seznam forumov skr c ET kolja em SPLETNA vORegistriraj se Q Iskanje D Prijava Aktualno Selected items Kako ob utimo vpliv Kaj pri akujemo od E Odgovori kandidatov Komentiramo rezultat Status Ready Zadnji video Dr avljanski forum Maribor najava 7 12 09 podnebn Odzivi Focus area Post titles Number of keywords ON OFF i IL Mia IV Find Next Previous Highlight all C Match case downloads MouseLogzp docAtasDataoy udoferkazp Cear Transferring data from localhost M sor zotero Figure 6 The topical atlas of a part of the forum Similar posts are displayed close together forming topical clusters The atlas chart comprises points each representing a forum post and keywords each roughly describing the topic of its immediate neighborhood To get further information about parts of the chart move your mouse over it The lightly blue shaded area under the mouse cursor is the focus area posts within the focus area are summarized by a list of keywords that appears next to your mouse cursor To reduce the need to scan the whole chart with the focus area some keywords are given in advance Those appear in green in the background The area of the ch
7. movies do not perform any serious computation data is computed on the server side the movies merely display it and allow some user interaction flash docAtlas Source code for the topical atlas flash movie flash canyonFlow Source code for the topical timeline flash movie ForumExplorerBar The GWT project for the client side of the VIDI toolbar ForumExplorerBar src Java sources which then get compiled to javascript by GWT Also some xml configuration files for the project ForumExplorerBar war Additional resources css for the toolbar and some more xml config files Most resources are in server gfx though Note the js file this is a hacked version of what GWT produces with instructions on how to reapply the hack to future GWT outputs The hack enables to use the toolbar as a bookmarklet something which is not possible by default server Server side of the toolbar Implemented in python with the heavy computation offloaded to C libraries server glib JSI s C standard template libraries and text mining libraries server clib A small project which produces a dll to be used by python Uses glib for algorithmics and Boost Python for exposing functions and classes server gfx Images referenced by the toolbar server json A python library for handling JSON data server canyonFlow docAtlas Data py Scripts for generating XML inputs for the flash movies These act as simpl
8. v Forms g Images v d Information v E Miscellaneous v G Outine v Resize D Tools Ty view Source Fi Options 4 Profesionaln 7 404 NotFo Problem http lt req2 C IST WORLD http erviet C Cookies Goo Evropske voli Mestsk cast X C Ca v i i ES KAKO BOMO PREMAGALI PODNEBNE SPREMEMBE e Dr avljanski forum E se amp B 4 parlamenta ELEKTRONSKO PARTICIPACIJO EParticipation O projektu Arhiv Domov Evropske razprave Forum Novice Poro ila Video Registracija Prijava Povezave vEIRegistriraj se Q Iskanje Prijava SPLETNA ANKETA anes je 30 12 2009 00 44 Aktualno vvv a bomo premagali podnebne spremembe 7 52 Napisal a Ako G i Va a stali a in predlogi evropski poslanki dr Romani 29 12 2009 07 51 jordan Cizelj za podnebno konferenco v Kopenhagnu in okoljske politike v Evropskem parlamentu Ali naj EU pozitivno diskriminira pri zaposlovanju Napisal a Drago Stanovnik a osla acinu 02 12 2009 09 26 cije iskalce prve trajno socialno varnost v EU vvv VIDI pri akujemo od Evropskega parlamenta a pri akovanja in vsebinski predlogi s katerimi bi se morali evropski poslanci ke ukvarjati v mandatu 2009 2014 ob utimo vpliv na na e Zivljenje Na
9. The javascript in question injects the toolbar s HTML code in the existing page making the toolbar appear The rest of this section describes the installation process for each of the two methods 2 1 Server Side Deployment The forum owner only needs to insert the following code in the head section of forum s HTML template script type text javascript id gwt vidi src http vidi ijs si forumexplorerbar forumexplorerbar noca che js gt lt script gt Nothing more is required The toolbar will appear in hidden state see Figure 3 for each user visiting the page 2 2 Client Side Deployment If the user wishes to use the VIDI toolbar on a forum that does not include the above snippet of code in their pages he should create a bookmark button pointing to the following URL Google Web Toolkit http code google com webtoolkit http en wikipedia org wiki Bookmarklet VIDI page 8 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 javascript function var 20e document createElement sc ript e id gwt vidi e src http vidi ijs si forumexp lorerbar forumexplorerbar nocache js bookmarklet 1 docum ent body appendChild e O For convenience further instructions on how to create a bookmark button and a copy paste ready version of the above URL are available
10. also section 3 1 3 2 Canyon Flow Visualization in deliverable D2 1 VIDI page 7 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 2 DEPLOYMENT MODES AND INSTALLATION Deliverable D2 1 envisioned the VIDI toolbar to be deployed as an Internet Explorer plugin The upside of this approach is that it requires no involvement from forum owners The downside is that the users have to run an installer in order to use the toolbar which limits the dissemination potential of the toolbar Another shortcoming of this approach is that it cannot support browsers other than Internet Explorer We have reconsidered the idea and consequently the toolbar has instead been developed using the GWT platform This means that the client side of the toolbar is written in javascript and can be deployed in two ways e reference to the relevant javascript file can be inserted in the HTML template of the forum pages by the forum owner making the toolbar available to all visitors without any involvement on their part e Asan alternative if the forum owner is not willing to include the toolbar the user can still inject the relevant javascript into the page using bookmarklets Both methods are independent of the browser as long as it has sufficiently strong support for javacript which is the case with all major modern browsers
11. apisal a Ako G a in predlogi evropski poslanki dr Romani 29 12 2009 07 51 lj za podnebno konferenco v Kopenhagnu v Evropskem parlamentu pozitivno diskriminira pri zaposlovanju Napisal a Drago Stanovnik a 02 12 2009 09 26 premagal zaposlitve in skrbi za trajno Sean varnost v EU podnebne spremembe AKTUALNE RAZPRAVEIME Zadnji video ees pri akujemo od Evropskega parlamenta Gostja G pri akovanja in vsebinski predlogi s katerimi bi 2 SE Pare 2009 14 50 evropski poslanci ke ukvarjati v mandatu 009 2014 ob utimo vpliv EP na na e Zivljenje 1 90 Napisal a gema G Va e izku nje mnenja rezultatih dela 12 11 2009 14 evropskih poslancev k ter u inkih sprejetih odlo itev v Evropskem parlamentu Mj ER Odgovori kandidatov k za EP na va a vpra anja 4 16 Napisal a Simon Delakorda G Odgovori kandidatov in kandidatk na volitvah v 05 06 2009 22 27 Dr avljanski forum Evropski parlament vpra anja ki ste jih postavili na spletnem Dr avljanskem forumu evropske volitve 2009 7 12 09 podnebn Komentiramo rezultate evropskih volitev 2009 Napisal a Irena Sovdat Va i odzivi komentarji rezultate 1 14 03 07 2009 08 05 glasovanja in udele bo na volitvah za Evropski Odzi
12. art of which they are representative is marked with a light shade of green Hovering the cursor over a forum post shows its subject as it appears on the forum e g Re new anti smoking law Clicking on a post navigates the browser to the corresponding thread Settings Just under the atlas chart there are several settings you can adjust The and buttons increase and decrease the size of the focus area respectively Depending on the browser you use you may also be able to adjust the focus area size by scrolling the mouse wheel Using the Number of keywords slider you can adjust the number of keywords with which the documents within the focus area are described VIDI page 13 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 Clicking the ON OFF button switches the display of forum posts titles on and off By default titles are hidden and only appear when you move the mouse over a post in order to reduce visual clutter If you choose to display the titles permanently their size can be adjusted with the Font size slider just next to the Subjects button 3 5 Using the Topical Timeline To see a timeline of topical evolution of a chosen subpart of the forum as illustrated in Figure 6 first indicate your interest by selecting several threads as described in section 3 2 Then press the Ti
13. azprav 33 391 Napisal a Jernej Prodnik re Mitja Tukaj so arhivirane kon ane spletne razprave in spletna 17 01 2008 00 17 SB Mouselogzp docatlasData py Ljudo erska zip_ Transferring data from localhost M zotero Figure 7 Topical timeline Each colored stripe represents a topic its description in the form of keywords is given below the graph The thickness of each stripe corresponds to how much a topic was talked about at a given moment The timeline appears in the middle of the window To discard the timeline click anywhere outside the popup area The main part of the visualization is the graph in its upper half Each of the colored areas of the graph represents a topic on the forum The thickness of the colored stripe shows how active this topic was through time The actual dates are given just above the graph at the very top of the visualization VIDI page 14 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 Hovering over a topic shows a tooltip with keywords describing the topic At the same time keywords for all the topics are displayed below the graph in a color coded legend Additionally when hovering over a topic small red numbers are displayed at the top of the graph for each time slot These are an absolute indicator of the top
14. ce Napisal a Irena S 03 07 2009 08 C 14 N Ostale razprave o evropskih temah in EU 14 22 Napisal a Dunja Pros Sor za informacije in razprave ki e niso objavljene Ky veder I SP EJ S ue Vet SOJA Wear ee aye POGONE U Date 7 1 Nae 3 3 Obtaining Browsing Suggestions To obtain suggestions on which forum threads might be of interest to you first indicate your interest by selecting several threads as described in the previous section Then click the first button on the VIDI panel Suggestions After a few seconds links on the current page identified by the system as relevant to you will be marked with an orange icon see Figure 5 The bigger the icon the higher is the probability that the link is truly relevant To clear the suggestions reload the page VIDI page 11 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 forum razprave Prva stran 1 Jee Edit View History Bookmarks Tools p gt S 0 httos nmw evropske voitve si TM e 216 Most visited vidi C vidiss 7 VIDI toolbar X disable cookies CSS v Forms v g Images v A Information E Miscellaneous v Outine Resize D Tools Source Fi Options vOD OGLDF PK Summary for 0 X ff OGLDF 0 606 0 070 13 X
15. e Napisal a Sonja Lokar 2 sakdanje izku nje in mnenja o rezultatih dela 12 11 2009 14 38 Pskih poslancev k ter u inkih sprejetih odlo itev v Napisal a Simon Delakorda G Bovori kandidatov in kandidatk na volitvah v 05 06 2009 22 2 Dr avljanski forum Vropski parlament na vpra anja ki ste jih postavili anai E Dr skem fi opske volitve 2009 tnem Dr avljanskem forumu evropske volitve 200 rezultate evropskih volitev 2009 Napisal a Irena Sovdat G komentarji na rezultate 03 07 2009 08 05 one na volitvah za Evropski Odzivi 5 Evropske volitve Napisal a Jure a Romana Jordan Cizelj 3 mnenja in razlogi zakaj se udele iti oz s 15 06 2009 17 19 Tanja Fajan 3 deleziti volitev v Evropski parlament Jeko 8 le razprave o evropskih temah in EU Napisal a Dunja Obersnel za informacije in razprave ki e niso objavljene Kveder 3 na forumu Registrirani uporab ahko odpirajo 07 11 2009 09 52 svoje teme razprav o Evropski uniji iiv tem in prispevkov Napisal a Simon Delakorda G iv tem in prispevkov ki niso vsebinsko povezani z odprtimi 29 009 14 19 razpravami oz niso v skladu s pravili in na eli spletnega komuniciranja na forumu X Find pvpon Next T previous E Highlight all Match case amp Downloacs NB MouseLogzp 2 docAtlasData py__ ljudo erska zip ear R 3 B
16. e web services with their own parameter syntax server crossdomain xml This file must be here to allow flash movies from accessing the py web services from the previous bullet point VIDI page 21 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 server db_triggers py Database triggers for maintaining up to date word statistics The triggers are written in python for postgres server index py A web service mini framework accompanying services for handling arbitrary function calls from the toolbar In the end there are only two services present notably one for computing browsing suggestions server structure py A library for structuring the database computing and extracting named entities updating statistics etc Also contains many standalone functions for outstanding maintenance work on the database e g complete recomputation of some statistic server sync Scripts for obtaining data from forums either via SQL connections or by crawling and caching it in the local database Uses structure py VIDI page 22 of 22
17. eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 TITLE Deliverable D2 2 Prototype Document Type WP Task Document ID Version Date Status Report on prototype including user manual WP2 VIDI 02 20091231 D2 2 1 0 31 12 09 Final Organisation Authors Contributors Responsible partners JSI Mitja Trampu Marko Grobelnik Dunja Mladeni c Bla Fortuna Bla Novak Nenad Stojanovi Sinan Sen Distribution PARTNERS Purpose of Document VIDI D2 2 Integrated and Evaluated VIDI system 8 System Manual Document History 17 12 2009 outline of the deliverable JSI 20 12 2009 First version of the deliverable JSI 22 12 2009 Overall revision JSI 28 12 2009 Adding section on Notifications FZI 31 12 2008 Final VIDI page 1 of 22 eParticipation Workprogramme Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 Integrated and Evaluated VIDI system amp System Manual VIDI Deliverable 2 2 31 December 2009 VIDI page 2 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 Table of contents jbxeldeela
18. ed description of the functionalities is available in VIDI Deliverable D2 1 Dr avljanski forum Evropske razprave Prva stran Mozilla Firefox Dee Edit View History Bookmarks Tools gt 49 Jhtto www evropske volitve si Mv ee Most visited vidi vidi 85 VIDI toolbar X Disable 8 Cookies i CSS v 8 Forms g Images v 1 Information E Miscellaneous v Outine Resize D Tools View Source v Fi Options VOD DL OGLDF PK Summary for O X 0 L X Mestsk ast Ko ice Sidis X Evropske volitve 2009 X http focal tedist 27 2F X Dr avljanski forum Evrops X Cf 9 SPREMEMBE Dr avljanski forum 2 858 G amp a vanje z evropsko Evropskega parlamenta ELEXTRONSKO um ano Jordan Cizelj O PARTICIPACIJO gt Prijava O projektu Povezave Selected items av Evrops u mov Evropske razprave Forum Novice Video Registracija Jezik Language Sloven ina v C Seznam forumov Atlas f teme Aktivne teme Timeline Status Ready yORegistriraj se Q Iskanje D Prijava Danes je 10 01 2010 16 42 Aktualno 20 12 2000 Zapis dr Jordan Cizelj nhagen Upanje bomo premagali podnebne spremembe N
19. http vidi ijs si install html Once the bookmark button has been created the user can use the VIDI toolbar by visiting a VIDI supported forum of interest and pressing that button anski forum pske razprave Prva stran oO c4 B Most visited Dvd D Disable Cookies KAKO BOMO PREMAGALI PODNEBNE SPREMEMBE Drzavijanski forum spletno posvetovanje z evropsko Evropskega parlamenta poslanko Romano Jordan Cizelj za konferenco v Kopenhagnu in Se pentiag Selected items pn delo v Evropskem parlamentu 17 november 2009 15 januar 2010 UE Ali gre na Evropske Domov Evropske razprave Forum Novice Poro ila Video Registracija Jezik Language Sloven ina M Seznam forumov Status Ready Neodgovorjene teme Aktivne teme PRISPEVKOV a Kako bomo premagali podnebne spremembe 5 52 1 1 Va a stali a in predlogi evropski poslanki dr Romani Jordan Cizelj podnebno konferenco v Kopenhagnu in okoljske politike v Evropskem parlamentu Ali naj EU pozitivno diskriminira pri zaposlovanju M Va i predlogi evropskemu poslancu Jelku Kacinu glede uvajanja pozitivne diskriminacije za iskalce prve zaposlitve in skrbi za trajno socialno varnost v EU 2 gt gt gt gt gt Figure 2 Client side deployment with a bookmark button example in Firefox VIDI page 9 of 22
20. ic s popularity they represent the actual number of posts from the given time period talking about the given topic The topics are automatically determined by semantically clustering the selected posts in a hierarchical fashion To avoid over segmentation all the posts are initially split into just two topics If you wish to delve deeper into a topic click its stripe in the graph If subtopics are available the topic will split into two Click on the black arc on the right to merge the subtopics once again Click on the vertical arrows in the left margin to temporarily hide all other topics and expand the selected topic along with its subtopics over the whole graph Click on the light grey arrow in the upper left corner to show the remaining topics again 3 6 Defining Notification Patterns VIDI System supports an email based user notification about user relevant situation within a discussion forum In order to notify the user he she must model the situation of interest and register in the VIDI System For this reason VIDI notification pattern user interface Ul provides discussion forum specific categories such that the user can select the relevant categories configure and connect them to each other using the provided operator nodes In order to define a notification pattern the user can use the drag and drop functionality provided by the UI Pattern Input Auto Connect UID 21B1A52A 3C50 43F2 872E D5F59ACD965A et Most 2
21. iscussion forums without worrying about to miss important changes See also section 3 6 for defining notification patterns 1 2 Browsing Suggestions The toolbar can given a list of topics and threads of interest for the user suggest further topics and threads on the current page that are similar to the selected ones and therefore potentially also interesting For a more technical description consult Section 3 1 2 Link highlighting in deliverable D2 1 1 3 Topical Atlas As a means of visually structuring a large subset of the forum VIDI toolbar can plot all posts from a selection of topics and threads in a two dimensional space Each post is represented by a point and the points are arranged in such a way that the proximity of two points is roughly proportional to the similarity of the two corresponding posts This way clusters of points naturally form representing subtopics of the part of the forum that is being analyzed For each cluster the user is able to determine the keywords and with that the topic See also section 3 1 3 1 Semantic Space Visualization in deliverable D2 1 1 4 Topical Timeline To determine the popularity of topics through time the VIDI toolbar can automatically determine the relevant topics in a given subset of the forum and plot the number of posts on each of the topics as a function of time The number of identified topics and consequently their specificity can be adjusted by the user See
22. meline button on the VIDI toolbar eo Dr avljanski forum Evropske razprave Prva stran Mozilla Firefox BAX Bile Edt View History Bookmarks Tools 5 c s e O htto f vww evropske volitve si v 8B Pe f Most visited 7 vidi 7 vidi 85 7 VIDI toolbar Disable cookies v 7 CSS v Forms v g Images v 1 Information v E Miscellaneous v G Outine v Resize D Tools v 4 View Source v Fi Options 4 OGLDF PK Summary ff OGLDF 0 606 0 07 7 Mestsk ast Ko ice A Problem loading Problem loading Dr avljanski foru Expand text box to fi YouTube Jonathan CF Jezik Language Sloven ina v O Seznam forumov yORegistriraj se Q Iskanje D Prijava 01 2010 18 25 Selected items Ali gre Evropske Status Ready PRITOZEVALI PARLAMENT JAVNI DRZAVNEM DRZAVLIANSTVU Click to see subtopics POMEMBNO GLAS EVROPSKI GORICI NOVE Dr avljanski forum Maribor najava 7 12 09 podnebn PRITOZEVALI PARLAMENT JAVNI DRZAVNEM Odzivi DR AVLJANSTVU Romana Jordan Cizelj 3 Tanja Faj 3 GRE PRIHODNOSTI VELIKO ODLOCAMO PARLAMENT Lojze SC 64 Arhiv posvetovanj in r
23. nt targets three groups of readers e Software end users The document includes a user s manual describing the use and functionality of the VIDI toolbar e Forum owners One of the two possible modes of deployment for the VIDI toolbar is to install it on the server hosting the forum The document provides instructions to this end e Developers A technical overview of the architecture and algorithms is given in Section 5 In Appendix A we describe the structure and organization of the source files comprising the VIDI toolbar Further information is provided in the forms of comments within the source code Note that accompanying CD contains the source code of the presented software as well as the installation instructions VIDI page 5 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 1 VIDI TOOLBAR FUNCTIONALITY As shown in Figure 1 the VIDI toolbar is used in parallel with the web forum of interest It is composed of a selection panel in the upper part of the toolbar and three action buttons further down as indicated by the two arrows in Figure 1 Using the selection panel the user can select the part of the forum she might be interested in Once the selection has been made the action buttons provide access to the main VIDI toolbar functionalities described in the following subsections A more detail
24. ome established preprocessing steps are performed on all forum posts once they are stored in the database HTML cleanup tokenization lemmatization stopword removal frequent n gram extraction We adapted an existing lemmatizer Jur i et al 2007 Slovene and Hungarian stemming rules were provided by the authors Slovak ones were trained on the Slovak National Corpus available at http korpus juls savba sk We also track the most frequent surface form for each lemma Additionally we perform named entity extraction and consolidation After preprocessing and named entity extraction all distinct terms are enumerated and a sparse vector of term frequencies is stored for each post To speed up processing of analytic modules we also store sparse TF vectors for all forum threads and topics and update them upon post insertions We also cache some other basic statistics e g document frequencies for all terms average post length etc Some of the caches refresh in real time some have to be refreshed by periodically running appropriate scripts 5 3 Analytic Modules The analytic modules providing input for browsing suggestions topical atlas and topical timeline GUIs access the database directly The computationally intensive parts are written in and exposed to python with Boost Python The python wrapper performs some additional formatting and exposes the functions as web services Database is accessed with either native drivers p
25. ork around cross domain scripting restrictions VIDI page 19 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 6 REFERENCES Fortuna B Grobelnik M and Mladenic D Visualization of Text Document Corpus Informatica Journal 29 2005 pp 270 277 Gr ar M 2009 D2 1 Architecture of the VIDI Integrated System and Test Scenarios VIDI Project Report Jur i M Mozeti I Lavra N 2007 Learning Ripple Down Rules for Efficient Lemmatization Proceedings of the 10th International Multiconference Information Society IS 2007 Vol A pp 206 209 Ljubljana Stojanovi N amp Gr ar M 2009 D1 1 As Is Analysis and Tool Selection VIDI Project Report VIDI page 20 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 APPENDIX A SOURCE CODE ORGANIZATION This section is highly technical in nature It explains the basic folder structure of the various software components comprising VIDI To make the code package self contained this part of documentation is given ina README TXT file accompanying the source code and is merely repeated here in its original form README txt This file Sketches the directory structure flash Flash movies The
26. pdakbeee P EXECUTIVE fe 1 VIDI TOOLBAR FUNCTIONALITY nnne a i a 1 1 REAL TIME NOTIFICATION Sinnet nr Seer i Pd ere t va eee ee Rd e a E ae C e E D E Ye un 1 2 BROWSING SUGGESTIONS dices dci ca der ERE Dk e ere Eri E ER ERR bed EE 1 3 TOPICAL ATLAS 3 itio et reti erbe Ib cn a ve ava ada Pau e ERR stad PUE ERE Ce uL d ated 14 TOPICAL TIMELINE beri de abe E C e b th at bh bee Pb ee oat dae 2 DEPLOYMENT MODES AND INSTALLATION nnne 2 1 SERVER SIDE DEPLOYM ENT CE e Poe Leod ed Feb re d ERES Ev Ee EPA deus 2 2 CLIENT SIDE DEPLOYMENT iig FyumeE H 10 31 GETTING STARTED t e SERE Ted RD i oi anc REL Po eO Rd cc dte 10 3 2 EXPRESSING AREAS OF INTEREST eene nnne eene nnda 11 3 3 OBTAINING BROWSING SUGGESTIONS enne 11 3 4 USING THE TOPICAL ATLAS ri rr eset detect ridere ce rod tee a tetas AM As 12 3 5 USING THE TOPICAL TIM ELINE Tea ete Dee Pen de ceat ete dee tex Pea vnda 14 3 6 DEFINING NOTIFICATION PATTERNS cccccscccccccccssssssecccesesesscsuecceuuccesesuueececsss rennen nnn 15 4 HIVENETLIFE aia ai 17 5 TECHNICAL BACKGROUND
27. pisal a Sonja Lokar ur mnenja o rezultatih dela 1 90 12 11 2009 14 38 inkih sprejetih odlo itev v guenvortk va a vpra anja A Napisal a Simon Delakorda amp Wi Odgov andidatk na volitvah v 4 16 65 06 2009 22 27 Evropski parlament na vpra anja K ih postavili j spletnem Dr avljanskem forumu evropske volitve 2009 isal a 2 11 2009 14 50 3 oi Zadnji video a Komentiramo rezultate evropskih volitev 2009 Napisal a Irena Sovdat a JEN wa AA na sana name Figure 3 The toolbar in its hidden state VIDI page 10 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 3 2 Expressing Areas of Interest In addition to the toolbar there is another way in which the VIDI platform makes itself seen on the web page Small icons appear next to each link that points to a discussion topic or a discussion thread as illustrated in Figure 4 Click an icon to express interest in the corresponding forum section The sections selected in this way are listed in the yellowish selection panel at the top of VIDI toolbar Click an icon again to deselect the corresponding forum section Note In deliverable D2 1 an automatic implicit detection of user s browsing interests was foreseen as opposed to the explicit selection panel icon
28. roject The plans have been followed through and LiveNetLife is now deployed on the Slovene use case web site www evropske volitve si and could possibly be added to the remaining use cases as well Client side deployment for LiveNetLife will not be offered within the scope of VIDI since LiveNetLife is being developed independently of the project at this stage it does not offer open access to its services to a potentially uncontrollable number of users VIDI page 17 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 5 TECHNICAL BACKGROUND The software architecture largely follows the plans proposed in VIDI deliverable D2 1 the main components are therefore only briefly outlined in this section Consult Appendix A for a list of actual source code files corresponding to architectural components described here 5 1 Data Acquisition A local copy of all data from all supported forums is stored in an SQL database For the Hungarian use case Bla Novak has written a specialized web crawler to obtain the data For the Slovene and Slovak use cases data is obtained with direct SQL access to the respective databases only some additional DB schema translation is needed Both the web crawler and the SQL crawler are run periodically to keep the local copy of the data fresh 5 2 Data Aggregation and Augmentation S
29. s combination However it has later been determined that users prefer tighter control over specifying their current interest as it may not have much to do with their past interests especially for casual visitors to the site Kaj pri akujemo od Evropskega parlamenta Og Va a pri akovanja in vsebinski predlogi s katerir bi se morali evropski poslanci ke ukvarjati v mandz 2009 2014 Kako ob utimo vpliv EP na na e ivljenje 9l Va e vsakdanje izku amp nje in mnenja o rezultatih del evropskih poslancev k ter u inkih sprejetih odlo itev Evropskem parlamentu VE Odgovori kandidatov k za EP na va a vpra anja Odgovori kandidatov in kandidatk na volitvah v Evropski parlament na vpra anja ki ste jih postavili n spletnem Drzavijanskem forumu evropske volitve 207 Napisal a Gostja 12 11 2009 14 5 Napisal a Son ja L 12 11 2009 Napisal a Simon 05 06 2009 22 2 Komentiramo rezultate evropskih volitev 2009 amp Va i odzivi mnenja in komentarji na rezultate glasovanja in udelezbo na volitvah za Evropski parlament 2009 Ali gres na Evropske volitve E Napisal a Jure amp 118 sen sides mnenja in razlog i zak se udele iti oz se ne 15 06 2009 17 1 udele iti volitev v Evropski parlament ki bodo potekale junija 2009 Figure 4 A close up of the web page showing the small VIDI icons with which the user can express interest in the topic s of choi
30. vi parlament 2009 gres na Evropske volitve 2 11g Napisal a Jure G x Va a mnenja in razlogi zakaj se udele iti se ne 15 06 2009 17 19 udele ti volitev v Evropski parlament ki bodo potekale iko junija 2009 Je X Find Previous E Highlight al al C Match case f Downioa s MouseLog zp docAtiasData py L Ljudo erskazp Cear Done M zotero Figure 1 The toolbar deployed on INEPA s web forum The selection panel and action buttons are marked with arrows VIDI page 6 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 1 1 Real time Notifications A discussion forum usually contains a lot of discussions and plenty of discussion topics In some cases the user is only interested in a couple of discussion topics and wants to be alerted if certain discussion topics become more important or new facts are posted within new topics In order to inform the user about these kinds of information the VIDI toolbar offers a user driven real time notification functionality Once the user has specified his interest the notification system starts observing the discussion forum and as soon as the situation of interest happens to inform the user about the situation Using this functionality the user is able to be informed about important changes within d
31. ython or via ODBC For browsing suggestions ranking is performed based on cosine similarity of TF IDF vectors of the query threads and target threads VIDI page 18 of 22 eParticipation Workprogramme VIDI Visualising the impact of the legislation by analysing public Discussions using statistical means Project Reference No EP 07 01 014 For the topical timeline hierarchical bisecting k means with TF IDF cosine distance is used on posts sparse TF vectors For the topical atlas the high dimensional space defined by the sparse TF vectors is projected onto several hundred dimensions using LSI latent semantic indexing and from there onto two dimensions using MDS multidimensional scaling To speed up the process the projection is determined by only observing the distances between up to several hundred clusters of documents 5 4 Client Side and Client Server Communication The client side of the software is written in Java and snippets of Javascript using the GWT Google Web Toolkit platform The visualizations topical atlas topical timeline are written in Flash with ActionScript 2 Javascript the toolbar and the Flash visualizations communicate using flashvars from Javascript and Flash s Externallnterface class to Javascript both directions possible Flash communicates with the server using custom formatted GET HTTP requests Javascript communicates with the server using JSONP JSON with padding callbacks to w
Download Pdf Manuals
Related Search