D6.2. User interface description and user manual of prototype 3
Contents
1. Scene saved jueves 26 de junio de 2008 9 29 05 Scene saved jueves 26 de junio de 2008 9 33 20 SS lt Random 400 wh a lt 1 1 gt amp Pallets Groups Mapping fn Statistics amp Lifestyle Fig 41 History a scene is recovered 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 40 43 8 CONCLUSION The interface of prototype 3 and the working procedure has been described The prototype 3 is available with the following functionalities of search random search search by image sample tuning old relevance feedback search by text search by mixed text and image The importance of this prototype 3 lies on the fact that it will allow to receive the feedback from the users after the tests It will be possible to test the new user interface and ergonomics elements to evaluate the quality of the new grabbed database with the new sectors distribution to evaluate the performance of the improved text search engine and accuracy of retrieved images to evaluate the performance of the improved image search engine and accuracy of retrieved images to evaluate the performance of the improved mixed text and image search and accuracy of retrieved images to identify possible improvements for final TRENDS software The integration of the different servers under the user interface running in the client s side was carried out and valid2. amp Pallets Groups Mapping j _ Statistics amp Lifestyle t Fig 12 Search by image sample drag and drop of a previously retrieved image 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 20 43 Search gt Advanced Search gt Sectors selections as many as wanted All is by default and None for reset Select the image by clicking it drag and drop the sample image in the images frame or using its contextual menu and select search with As it can be shown in the picture the user can change the image sample Just right click and select Delete Select the descriptors by means of which the search will be done By default all the descriptors are selected It is necessary to click on them to deselect The available descriptors are o Colour images with similarities by colour are searched o Shape images with similarities by shape are searched o Texture images with similarities by texture are searched Click Search After this action the retrieved results appear in the sphere Fl TRENDS Software EEK Simple Advanced History K A Q Search ac Keywords MI Image e e a rr Tuning Sectors Options amp Pallets Groups Mapping A Statistics amp Lifestyle Fig 13 Retrieved results of a query by image sample As it has been indicated before it is possible to c
3. Fig 3 Interface for advanced search functionality 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 13 43 As it can be appreciated the following elements appear on the left Simple search only search by text or random search no keywords over all sectors can be done The system retrieves any images without considering keywords images or any other parameter It is a highly inspirational procedure Al TRENDS Software Ee Simple Advanced History Search mm SC GO amp Pallets Groups Mapping PA Statistics amp Lifestyle Fig 4 Example of simple search by text or random Keywords it allows introducing the keywords for the search by text Although at this moment only keywords are possible to be inserted in the user interface of prototype 3 there is the possibility of using complementary information o Suggested words as long as the user is writing the closer words appear when 4 characters have been typed For instance fashion will be proposed if the user types fash o Associated words proposed words to disambiguate by context if any This is linked to the term pertimmizers The first version of suggested words is available in the first version of prototype 3 delivered in June 2008 M30 In the future the system will have a filtered and improved version Image it is the place to drag and drop the selected images for the search coming either from
4. 23 Fig 17 Results retrieved after tuning 222485580808 vans semen lawns ie van atlases des teba dentine deccaveneer 24 Fig 18 Introducing keywords and image sample in a mixed search 25 Fig 19 Context menu of every image ax e a2 ees ere deca se prete ENER E esse ren sacandxcpavadesaec ee 26 Fig 20 Information about the properties Of every image 27 Fig 21 Zoom in of the images in a sphere 28 Fig 22 Zoom out of the images in a sphere 28 Fig 23 Sphere with its context MENU needed CR OT 29 Fig 24 Zoom in of the SONG G eege gege ged eg 30 Fig ee 31 Fig 26 Zoom out of the working space by using the ZOOM slide DAT 32 Fig 27 Zoom in of the working space by using the ZOOM slide bar 32 Fig 20 An sphereis MAINO EE 33 Fig 29 Display possibilities accessed by the context menu of a sphere 33 Fig 30 Display modes mosaic is EE 34 Fig RE Ee EE 34 Fig 32 Display mode big and Een ET 35 Fig 33 Display mode MOSSY dsssnceneneererenenene ennen nn erne nn net rat not e le ee lasse Une 35 Fig 34 Set directory in the personal sphere to store images 36 Fig 35 browsing in the PC to set directory in the personal sphere to store images 36 Fig 36 Sub spheres in the personal sphere for stora0ge 37 Fig 37 Way of hiding the sub spheres in the personal spher
5. g Leg La C7 t 4 cae EES Y mHE June amp Pallets mm Groups m Mapping i Statistics amp Lifestyle Fig 30 Display modes mosaic is default e Single Ml TRENDS Software Simple Advanced History K 2 Q Search Keywords gt Image gt Tuning gt Orbost gt test new name 996 amp Pallets mm Groups Mapping i Statistics amp Lifestyle Fig 31 Display mode single 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 34 43 e Big and small car fast 261 test new name 996 amp Pallets Groups i Mapping 7 Statistics amp Lifestyle Fig 33 Display mode messy 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 35 43 7 4 STORAGE OF IMAGES IN FOLDERS This is a new functionality not available in prototype 2 The possibility of storing the interesting images found in the retrieval process in an ordered way and managed by the user has been requested from the beginning of the project by the designers Here it is Every image retrieved by the search engines can be moved to the personal or collective sphere Now and differently from the final version of prototype 2 the personal and collective spheres are linked with real directories on the user computer In this way drag amp drop an image i
6. It is possible to access the sphere contextual menu by right clicking on it Ca car 996 Minimize Maximize Restore Group Ungroup Mapping Slideshow Display mode Set directory Fig 23 Sphere with its context menu 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 29 43 The available options are Minimize the sphere is minimized and it is placed in the bottom left area of the interface Maximize the sphere is maximized and it is placed in the centre of the interface Restore after being hidden it recovers its initial look and position Slideshow the images contained in the sphere will be displayed in slides Display mode the images contained in the sphere can be displayed in four different ways this is mosaic by default big and small single and messy This will be visually described in next section Set directory only for personal and collective spheres the directory for storage of the images in the user s computer can be set first access to the executable or modified later access Close this sphere and its results are removed e Zoom and zoom of a sphere Wheel up and down with the mouse once a sphere is selected Fl TRENDS Software CS x E d Advanced History E A Q Search gt Keywords car 996 Pallets Groups 9 Mapping 3 Statistics am
7. the new interface of prototype 3 that has been developed in T6 2 The resulting deliverable D6 2 is headlined User Interface description and User Manual of Prototype 3 This deliverable describes the user interface developed for prototype 3 it explains the procedure for installation and execution of the software the functionalities for search and the general available functionalities at this stage of the project 1 3 STRUCTURE OF THE REPORT The following document D6 2 will be organised into the following parts General introduction 2 Overview of prototype 3 the purpose and main features of the prototype 3 are described as long as the elements that are integrated the communication protocols and integration steps 3 User interface development environment the programming tools used for the development of the user interface are enumerated Procedure for execution it describes the steps for running the interface Description of the interface and available functionality the interface is described Working procedure search functionality it is explained how to use the search functionality of the prototype 3 7 Working procedure general available functionality it is explained how to use the options and functions available in the interface Conclusions 9 List of figures and tables The understanding of the sections by all partners is of great importance because they constitute the reference of the user interface of prototype 3 and t
8. Lifestyle Fig 8 Search by text from the Advanced panel 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 17 43 In both Simple and Advanced search the system proposes some suggestions when the user types a word The suggestion is proposed whenever 4 or more letters are typed A list of the available indexed words is displayed to the user The word is selected with enter or double clicking over it from the list M TRENDS Software BER Simple Advanced History K A L Search house housing housky housse houston house hous housgard houshiary housin housner houssa housten hous4u2 housalb3 houscleaning houshiary houssa1 houstexanslogoi Pallets Groups Mapping 3 Statistics Ex Lifestyle Fig 9 Search by text with the suggested words proposed when hous has been typed The results for a query by text are retrieved when search is pressed 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 18 43 Pi TRENDS Software E GE Simple Advanced History x K Ak seach acc Keywords v L r more gt car 162259 M Image gt Tuning gt a Sectors gt Options gt lt 1 1691 gt amp Pallets mm Groups a Mapping ff Statistics amp Lifestyle Fig 10 Result
9. bar 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 32 43 e The name of the sphere can be modified Al TRENDS Software HEL Simple Advanced History x K A L Search ast Keywords gt M Image gt rr Tuning gt ia Sectors gt Es me SODE ne sl my Du Pallets Groups aa Mapping 3 Statistics amp Lifestyle Fig 28 A sphere is named 7 3 DISPLAY FUNCTIONALITIES It is possible to display the retrieved results in different ways Right click over a sphere to make the context menu appear and select Display mode test new name 996 a em Ld rc J D ad A AER ERR gt Minimize Maximize Restore yl Dy Di Group Ungroup Mapping Slideshow Display mode Big and small Mosaic Set directory Close LS Single Fig 29 Display possibilities accessed by the context menu of a sphere 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 33 43 There are four display modes available e Mosaic it is the default one Pl TRENDS Software HEL Simple Advanced History K 2 QX Search gt Keywords gt E Image gt gt Tuning gt Sectors gt Ca Options gt test new name 996
10. the personal or collective sphere not available in this P3 version from previous results or from random search Tuning this is the old named Relevance Feedback whose aim is to refine the search this is the system can learn from the information provided by the user He can classify the images as good or bad depending on his expectancies At least one positive 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 14 43 image has to be selected This way the system can learn what the user is looking for and what is not useful In this iteration the system tries to fit better with the user s objectives Sectors users have the possibility of choosing the sectors they want to explore The database of the TRENDS system is organized according to these sectors identified in T2 2 This way focused search is possible By default all sectors are active The user can select them one by one by clicking on it activate all click All or deactivate all click None Nevertheless it is important to take into account that no search can be done if there is no sector selected The version 3 of database contains 25 sectors Options in order to tune the search it is possible to select the category in size of the image being possible to choose among three options small medium large Search once the parameters are defined it is needed to click Search After a few iterations the us
11. CUTION kkr ankr r narnia nEn E tanane nanenane nene 9 5 DESCRIPTION OF THE INTERFACE AND AVAILABLE FUNCTIONALITY sise 9 6 WORKING PROCEDURE SEARCH FUNCTIONALITY ccccccsceseceeeeeseceesseueeeeseueeeueaaeeeeneaes 12 6 1 RANDOM SEARCHE stenge te een ee ne A Red As NE Non de Are Melee 15 6 2 SEARCH BY TEXT OUER 16 6 3 SEARCH BY IMAGE 19 6 4 TUNING RELEVANCE FEEDBACK ocincetiicscsicansistinas tes ivivcdsvintetels asia CEA 22 6 5 MIXED SEARCH BY TEXT AND IMAGE cccceccceeeecccseeeeceseeeeesesseeaaeseeeaeueessaneueeaes 24 7 WORKING PROCEDURE GENERAL AVAILABLE FUNCTIONALITY 0ccccccsseecceseeeceeeeeeeeeenes 26 7 1 HANDLING OF AN IMAGE cccccccececccsceeccesceeeceeseeeuceesseausesseeaaueeeeauaueeaeanaueeaeanaueeses 26 7 2 HANDLING OF A SPHERE ssiiiscceecccccececseceenneseenennseeceenececneneeeceenaeeeeeenseues 29 7 3 DISPLAY EUNGCTIONALUTIES rta nkene ttina an nenennrnnnnn nne 33 7 4 STORAGE OF IMAGES IN FOLDERS iiieeiicceesseeceeeseeceeneseecenneeeeennsses 36 4 54 SLIDESHOW fac or Pelee tna Eed EE 38 06 jHISTORY ee Se E at A ORLA EE 39 Bx GONGEUSION WEE 41 9 LIST OF FIGURES AND TABLES ccccccceceeeeceeeeececeeeeeueceeeeeueceeeueuseseueeuseseuuauseveueaaueeenanes 42 TO GROSSARY EE 43 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 2 43 1 GENERAL INTRODUCTION This report corresponds to the deliverable of WP6 D6 2 User interface description and user manual o
12. LT Pa SE FPE LE mms S Options gt Zu By Owl 2 1 16262 gt amp Pallets mm Groups E Mapping Statistics amp Lifestyle Fig 17 Results retrieved after tuning 6 5 MIXED SEARCH BY TEXT AND IMAGE The final version of prototype 2 already included this functionality It consists of the possibility of launching searches combining both queries this is keywords and image samples The interface delivers the whole query to the Request Management Server RMS that distributes the query to the proper search engine the keywords will go to the Text Search Engine TSE and the image sample will go to the Image Search Engine ISE Once both retrieve the best matching results the MTISC merges the results provided in a newly ranked list according to a fusion algorithm specifically designed What the user can see in the interface are the results ranked after this fusion In the next figure it can be appreciated how both queries are inserted 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 24 43 Fl TRENDS Software Simple Advanced History x K A Q Search e st Keywords v E more gt Image Vv more gt Ta D O Tuning gt as Sectors iv advertising architecture art car cinema communication design fashion food furniture hifi household interior_design lan
13. Trends Research ENabler for Design Specifications FP6 IST 2005 27916 Deliverable Security Leading partner Carole Bouchard Approved by Carole Bouchard 30 06 08 D6 2 User interface description and user manual of prototype 3 This document describes the user interface of prototype 3 and how to use it Acronym TRENDS gt O SERAM PERTIMM INRIA List of ROBOTIKER participants CRF FIAT SB STILE BERTONE UNIVLEEDS UNIVERSITY OF LEEDS CU CARDIFF UNIVERSITY Coordinator SERAM organization SOCIETE D ETUDES ET DE RECHERCHES DE L ECOLE g NATIONALE SUPERIEURE D ARTS ET METIERS E mail contact j person carole bouchard amp paris ensam fr Project Website www TRENDSproject org STREP Specific Targeted Research Project Contract number FP6 IST 27916 Start Date 1 January 2006 Duration 29 months 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 1 43 0 INDEX 1 GENERAL INTRODUCTION nus reueeucscceeececeenuneeecnnneeecuenceecennceeceneneeeceneneeeeenaneceenansues 3 1 1 WORK PACKAGE 6 OBJECTIVES USER INTERFACE DEVELOPMENT scccccccccnnn 3 1 2 DESCRIPTION OF WORK TASK TG 4 1 3 STRUCTURE OF THE REPORT c ccccsceccccseeeceseuececeeeuueeceseeueuceseueuaeeeueuueeeeeeseeeueanss 5 2 OVERVIEW OF PROTOTPE utri nn ank nnen E EnA Arno AAEEen EEEE anane n nna 6 3 USER INTERFACE DEVELOPMENT ENVIRONMENT asiisiesccecccsceeeeneseeceenseeceenseeenennues 8 4 PROCEDURE FOR ESE
14. antic adjectives are specific words frequently used by the corporation of designers in order to describe visual low level features according to feelings or emotions in front of a specific visual stimulus They often use metaphoric references in other sectors For example the front face of a sport vehicle can be described as aggressive like a snake face Semantic mapping The semantic mapping is a planar representation of a set of models in a reference sector according to two reference axis limited by semantic adjectives and their antonyms The semantic mapping gives a semantic description of the different models in the 2D space Server A server is a software or a computer that provides services to other softwares or computers often remotely over a computer network connection TCP IP The TCP IP protocol is a communication protocol widely used for transmitting messages and data between clients and servers It is the protocol used for Internet data transmissions XML XML stands for eXtensible Markup Language It is a World Wide Web Consortium recommended markup language that supports a wide variety of applications It is made of XML tags organized in a hierarchical tree and written in a specific format 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 43 43
15. ated in prototype 2 The communication protocols and overall architecture of the system is totally kept in this prototype 3 in relation to prototype 2 The improvement has been done inside every identified module 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 41 43 9 LIST OF FIGURES AND TABLES List of figures Fig 1 General user interface of TRENDS prototype 3 9 Fig 2 Sphere for the visualization of the retrieved results 11 Fig 3 Interface for the advanced search functionality 13 Fig 4 Example of simple search by text or random 14 Fig Do NOON Sealen EE 15 Fig 6 Retrieved results in a random search AAA 16 Fig 7 Search by text from the Simple panel 17 Fig 8 Search by text from the Advanced panel renerne nn 18 Fig 9 Search by text with the suggested words proposed when hous has been typed 18 Fig 10 Results of a search EE 19 Fig 11 Advanced search MON nds en nee et SALE 20 Fig 12 Search by image sample drag and drop of a previously retrieved image 20 Fig 13 Retrieved results of a query by image sample 21 Fig 14 Positive examples in Tuning functionality 22 Fig 15 Positive examples added to the box in tuning functionality 23 Fig 16 Negative examples added to the box in tuning functionality
16. d drop all the images that fulfilled his expectations inside 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 22 43 Pl TRENDS Software SG Simple Advanced History K A QX Search x e x Keywords gt gt v M Image O Tuning s ES FS oe Options gt amp Pallets mm Groups Kl Mapping Wes Statistics amp Lifestyle Fig 15 Positive examples added to the box in Tuning functionality In the case the user wants to indicate the images that do not fit at all with his expectations he must click on more A negative box will appear where to drag and drop the not fitting images K TRENDS Software FEEL Simple Advanced History x K A QX search 4 Keywords gt M Image gt ef Tuning v less V GIE Sectors gt Options gt Pallets o Groups 9 Mapping j _ Statistics amp Lifestyle Fig 16 Negative examples added to the box in Tuning functionality 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 23 43 After this selection the user must click on search The sphere with the tuned results will appear in a new sphere M TRENDS Software HER Simple Advanced History K A Q Search ast Keywords gt Image gt gt Tuning ait E FE de LE
17. d in the selected sphere according to their associated semantic adjectives Only simulation is available Statistics it contains the interface for the clustering by text or by colours in every sector depending on the user s requirements Only simulation is available Lifestyle it represents the information about the sources of the database Only simulation is available The user finds two spheres in the upper right corner of the user interface Personal sphere to store the images the user has found and is interested in for his personal use Collaborative sphere to store the images the user has found and wants to share with the colleagues The prototype 3 is available with the following functionalities of search random search search by image sample tuning old relevance feedback search by text search by mixed text and image sample The general functionalities available are Drag and drop an image for the query by image or to store it in Personal Sphere or Collaborative Sphere Contextual menu of a sphere minimize maximize and restore group ungroup mapping slideshow display mode set directory personal and collective and close Contextual menu of an image cut copy delete go to web source set as favourite print save as search with refresh and properties Properties of every image name size date web source sector and semantic description Storage of selected images in personal and collecti
18. dscape luxury music nature packaging photography product design science fiction sport transportation virtual all none Options gt amp Pallets mm Groups EI Mapping i Statistics amp Lifestyle J MALE Fig 18 Introducing keywords and image sample in a mixed search In this version of the prototype 3 and as it was generated in prototype 2 a log file is generated after every search In this LogFile txt the information about the search is available actions times and possible errors are there registered This information is really useful for the obtaining of quantitative results such as the response time the number of errors the queries nature etc 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 25 43 7 WORKING PROCEDURE GENERAL AVAILABLE FUNCTIONALITY 7 1 HANDLING OF AN IMAGE Once the results are retrieved the user has the choice to carry out the following actions e Context menu for every image by selecting it and right click Delete Go to web source Set as favourite Print Save as Search with Refresh Properties Fig 19 Context menu of every image The options available for every image context menu by right clicking are Cut the selected image is cut Copy the selected image is copied Delete the selected image is deleted Goto web source the user can directly open the web where the image comes from Set as favourite it is st
19. e BER Search Images dl amp Pallets wm Groups Mapping fj Statistics amp Lifestyle Fig 36 Sub spheres in the personal sphere for storage It is possible to hide all the subfolders in the interface for that it is needed to click on the Ungroup option of the context menu and the main sphere will appear 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 37 43 Minimize Maximize Restore Group Ungroup Mapping Slideshow Display mode Set directory Close Fig 37 Way of hiding the sub spheres in the personal sphere for storage 7 5 SLIDESHOW It is possible to visualize the results available in a sphere in a slideshow This functionality is accessed by the context menu of the sphere or by double clicking in one image retrieved in results sohere Whenever the user clicks of slideshow the sequential display of the images appears The user can modify the delay in seconds between the images a TRENDS Software Baa Search Images Fi Slideshow Random 400 amp Pallets Groups Mapping fa Statistics amp Lifestyle Fig 38 Slideshow 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 38 43 The user can move to previous and next image with the two arrows buttons He can move forward automatically using the play button and configure the delay time among the images 7 6 HISTORY This functionality
20. e for siora0ge 38 FIO 38 QUOSSNOW E 38 Fig 39 Save the scene for history purposes 39 Fig 40 History how to delete a scene previously store 40 Fig 41 History a scene is FECOVETED EE 40 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 42 43 10 GLOSSARY Client A client is a piece of software that accesses services from another piece of software a server often remotely over a computer network connection Cooccurrences Expressions frequently found with the query HTTP Acronym for Hyper Text Transfer Protocol Protocol used by the web it allows hyperlinks to work for every object of the web Ontology Ontologies are representations of knowledge via concept networks often hierarchical trees Relevance feedback One of the methods used in CBIR to deal with semantic gap It uses machine learning techniques to infer from the interactive information provided by the user what is the target class of images Relevance feedback is also used as an interactive browsing method and for the mass annotation of large quantities of images Sectors of influence Sectors of influence are all the industrial artistic or natural sectors that are used for the transfer of some formal and functional attributes into the reference sector For instance the biomorphism is a key sector of influence for car design Semantic adjectives Sem
21. e xml generated for history requests responses and feedbacks and the xmlNumber file for indexing the files Launch executable Trends exe The configuration file contains the following elements Results per request number of retrieved results every request FSE IP and Port IP address and port to access the Fusion Search Engine in Pertimm Timeout waiting time for responses ROOT_MIRROR where a copy of the images of databases is accessible ROOT THUMBNAIL where the thumbnails of database are ROOT_XML where the files with metadata are placed directories of personal and collaborative sphere 5 DESCRIPTION OF THE INTERFACE AND AVAILABLE FUNCTIONALITY Once the user launches the executable the general view of the user interface appears NI TRENDS Software EK y w Search Images preien LT A Pallets amp Groups Mapping 7 Statistics SP Lifestyle Fig 1 General user interface of TRENDS prototype 3 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 9 43 The user finds different options in the menu bar Search images it gathers the search related functionalities Pallets it represents the harmony component generation functionality colours and textures Not available yet Groups it represents the interface for clustering images Only simulation is available Mapping it represents the interface for the generation of the proposed 2 axis representation of the images containe
22. er can carry out with tuning or others It is needed to click red cross to start a new one this indicates the end of search The parameters of the search can be cleared by clicking in the black circle 6 1 RANDOM SEARCH The following actions have to be carried out Click Search neither keywords or images are added All sectors are selected by default Fl TRENDS Software HEJ simple Advanced History x K IQD 2 1 i Wa more gt e amp Pallets mm Groups Mapping fe Statistics Ee Lifestyle Fig 5 Random search Once it has been performed the initial retrieved results will be shown in the interface 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 15 43 M TRENDS Software Simple Advanced History K I QOX search Random 400 Elm Fis FEM gt y ay oy amp Pallets mm Groups Mapping i Statistics amp Lifestyle Fig 6 Retrieved results in a random search The sphere with the results retrieved by random search will be displayed alone in the workspace every image can be moved to personal or collective spheres As it has been explained in the previous section the images can be zoomed in or out by means of the scroll bar on the left in the bottom the number of the current page can be shown over the total of available pages The user can move forward or backward easily The scroll bar on the bottom allows display
23. f prototype 3 This is the first version of the deliverable that corresponds to the stage of the prototype for the testing sessions to be held during July in users sites 1 1 WORK PACKAGE 6 OBJECTIVES USER INTERFACE DEVELOPMENT The objective of WP6 is to elaborate the user interface for the complete system This interface is a collection of different interfaces that present different functionalities such as the mixed image and text search pallets generation ambience boards generation mapping and clustering The WP6 is fully related to WP1 and WP2 which constitute the input for this WP WP1 gathers the requirements of the users regarding the functionalities of the system The WP2 gathers the system technical layout and the integration of the elements developed in the other WPs interface included The output of WP6 will be evaluated by the end users in WP7 WP6 is split up into three tasks T6 1 Interface development of prototype 2 the interface of prototype 2 is developed Prototype 2 is the first interactive software whose aim is to evaluate the first version of the interface and its functionalities It is constituted by the user interface an executable that works on the user s computer and two servers the text search engine and the image search engine The prototype 2 is available with the following functionalities of search random search search by image sample search by relevance feedback search by text 16 2 Inter
24. face development of prototype 3 The prototype 3 represents the almost final version of the TRENDS system and will contain the following elements the new highly innovative user interface the mixed text and image search component the relevance feedback final component the grabbing component 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 3 43 the clustering and mapping functionalities the harmony generation component the statistics the lifestyle the history the display modes and slideshow functionality At the moment of writing this first version of the deliverable D6 2 the following functionalities are available the new highly innovative user interface and related functionalities the mixed text and image search component the relevance feedback final component 16 3 Final software tuning The final software is the prototype 3 with the improvements detected by the users after the last evaluation tests It will contain all the functionalities defined for TRENDS system This deliverable D6 2 is the output of the T6 2 and its aim is to describe the user interface of prototype 3 1 2 DESCRIPTION OF WORK TASK T6 2 This report is related to the task T6 2 Interface development of prototype 3 After the definition of the sectors of influence from an initial sociological and design trends database T2 1 and the definition of a procedure for the identification of the
25. g all Microsoft Visual C is the unquestionable IDE Integrated Development Environment In prototype 3 Microsoft Visual Studio version 2008 has been used over NET 3 5 Microsoft Expression Blend is the new professional design tool that allows using a wide library of existing graphical elements and creating customised controls with a variety of effects according to the user s creativity It can be easily integrated into Visual Studio environment to incorporate functionality to the graphical element The reasons that lead the developer team to the use of these tools were the following ones The fact that it is C C based guarantees the fast response and interaction with the user Microsoft Expression Blend presents graphic libraries wide enough for the building of the advance interface It provides with professional finish C was used for the development of the communication protocols and other functions of prototype 2 that are kept in this version It allows importing DLL for algorithm execution It allows generating installation kits easily transportable to the users computers 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 8 43 4 PROCEDURE FOR EXECUTION It is necessary to follow these steps before running the application NET Framework 3 5 installed Copy the files in the desired directory o Trends exe o Trends exe config o xml directory with its subfolders you will find there th
26. ge gt P Tuning gt oS gt _ options gt lt 1 11 gt Pallets e Groups m Mapping 2 Statistics amp Lifestyle Fig 21 Zoom in of the images in a sphere Al TRENDS Software EEK simple Advanced History K Wow sens a gt ee gt 0 Tuning gt Ws Sectors gt go d ex 659 Em E Ei SES meer Fk EA en 22 AN EE Salo amp Pallets em Groups Mapping 3 Statistics amp Lifestyle Fig 22 Zoom out of the images in a sphere 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 28 43 7 2 HANDLING OF A SPHERE The results retrieved by every search are stored in a sphere The information shown in every sphere is number of total results For shuffle search it is established 400 but any search will have a different number of total results These results are ranked by score similarity to the query number of results visualized By default 96 of the retrieved results are displayed in mosaic mode but it depends on the zoom and on the display mode established by the user The following actions are possible Click the arrow on the right to display the rest of available results in the next page Click the arrow on the left to display the rest of available results in the previous page Several functionalities are available to handle these spheres
27. ges Prototype 3 is constituted by the user interface a light executable 1 7 MB that runs in the user s computer and two servers the text search engine TSE and the image search engine ISE the Request Management Server RMS that addresses the queries to the proper server and gathers the results coming from both of them is placed physically in the TSE server The prototype 3 is available with the following functionalities of search random search search by image sample search by relevance feedback search by text and search by mixed text and image Semantic developments such as ontology tags are present in the text search engine but still not visible from the user interface The basis of the communications protocols and formats for the exchanges between the modules are defined to match the requirements of the functionalities implemented The general functionalities available in the user interface for handling of data and results are wider than in the previous version This is also a big advantage In this prototype the database index is built by steps integrating indexation from Pertimm images validation from INRIA and ontology tags from CU but there is not automated process to link these actions 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 6 43 This prototype has been implemented with two distant servers the image search engine located at INRIA and the text search engine located at PERTIMM A cop
28. he TSE verifies the format of the request is correct o the TSE sends back the response to the UI o the Ul receives the response o the UI verifies the format of the response is correct o the Ul shows the images retrieved in the planned workspace 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 7 43 For further details in the system architecture it would be better to refer to D2 5 and Metadel3 where all the modules links and aims of every module and links are fully specified The results of the tests on prototype 3 in the pilot testing June 08 and with the end users July 08 will be described in the report D7 2 as output of T7 2 3 USER INTERFACE DEVELOPMENT ENVIRONMENT The user interface of prototype 3 has been developed with the integration of the following technologies Windows Presentation Foundation WPF WPF is a graphical subsystem in NET Framework 3 0 formerly called WinFX and is directly related to logical part XAML The development environments are Microsoft Visual Studio 2008 Microsoft Expression Blend and C C is an object oriented programming language developed and standardised by Microsoft as part of its platform NET which was later approved as a standard by the ECMA and ISO It syntaxes results from C C and uses the model of objects of platform NET C is a programming language designed to generate programs over this platform There are several compilers for C language amon
29. he working procedure The evaluation of the quality of the retrieved results and ergonomics specifications depends deeply on the understanding of the available functionalities 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 5 43 2 OVERVIEW OF PROTOTYPE 3 Prototype 3 is the improved software after the prototype 2 development first interactive version of the software and its evaluation by the end users Some of the existing differences between prototype 2 and prototype 3 are exposed next it contains the improved database the second version of the database contained about 500 000 images while this new version contains about 1 900 000 images which is four times bigger It is not necessary to change the resolution of the screen to 1280 x 1024 as in prototype 2 The interface auto sizes It is not necessary to change the font s size to big fonts 120 ppp The interface auto sizes The interaction with the interface is faster than in prototype 2 The feedback provided by the system to the user has been encouraged The general aspect of the interface of prototype 3 has changed according to the tests and feedback retrieved by the end users during the different test sessions The general functionality of the interface for the handling of data and results is wider than in prototype 2 Many options are possible that will make it easier the search the display and the storage of the retrieved ima
30. ing the rest of the results available of the current page On the left the different options for handling the sphere are available it can be minimized maximised restored and closed The zoom in the sphere can be applied by using the mouse wheel the mouse being over it 6 2 SEARCH BY TEXT QUERY The search by text can be launched from Simple or Advanced search panels and requires the following actions From Simple search Insert the desired keywords It is possible to insert as many keywords as wanted 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 16 43 Al TRENDS Software SG Simple Advanced History K A Qy Search fast car more gt Pallets Groups Mapping jj _ Statistics amp Lifestyle Fig 7 Search by text from the Simple panel From Advanced search Click under Keywords insert as many keywords as wanted Sectors selections as many as wanted All is possible and None for reset Fl TRENDS Software SER Simple Advanced History K J Qe search 7 a Keywords car fast more gt M Image gt Tuning gt Sectors v car cinema communication design fashion food furniture hifi household interior_design landscape luxury music nature packaging photography product design science fiction sport i amp Pallets Groups Je Mapping jj Statistics amp
31. is also new and was not available in prototype 2 It was requested by the users that expressed the desire of having a way of tracking their searches Every time the user wants to store the whole workspace this is the scene he has to right click on the workspace to make the related context menu option appear Fl TRENDS Software BALE Simple Advanced History Scene saved jueves 26 de junio de 2008 9 29 05 Scene saved jueves 26 de junio de 2008 9 33 20 Save to history Ez Random 400 red car 485 1 E Gs amp Pallets mm Groups Mapping FN Statistics amp Lifestyle Fig 39 Save the scene for history purposes Every trace of the history can be removed if it is not desired to keep it there The user has to right click on the specific element and select the delete option 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 39 43 NI TRENDS Software BOE Simple Advanced History SS Ste SES EET Seier Scene saved jueves 26 de junio de 2008 9 Seet red car 485 LG d ea amp Pallets Groups mg Mapping fj Statistics amp Lifestyle Fig 40 History how to delete a scene previously stored All the information contained is saved properly with the date and time of storage To recover that information the user has to double click on the desire stored scene a TRENDS Software Simple Advanced History
32. lick end of search red cross in search window in case the user wants to manage the images in the workspace If he prefers the 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 21 43 user can go on with another search by image or tuning as many times as desired Prototype 3 only admits one image as input sample every query This first version of prototype 3 does not admit images that are not present in the database this is images stored previously in the user computer sketches or modified images It will be available in later versions of the prototype 6 4 TUNING RELEVANCE FEEDBACK Once a query by image is done it is possible to tune the results by means of Tuning option When Tuning is open the Image window is closed since conceptually both of them cannot be performed at the same time The user will indicate to the system which results are useful or which are useless according to their expectations For that it is necessary to drag and drop the images in the proper place in the interface The interface is as follows Fl TRENDS Software Simple Advanced History K A Q search ua Sectors Options Pallets Groups Je Mapping f Statistics amp Lifestyle Fig 14 Positive examples in Tuning functionality According to the user s approach initially a positive box is opened The user should drag an
33. mages The following 96 images are displayed Display options it is possible to visualize the results in a different manner o Mosaic by default in row column o Single only one image is displayed in front every time while the others remain below in small size o Big and small proportional distribution where big and small images are displayed symmetrically o Messy disorganized way of visualizing the images with a certain angle These functionalities of the interface and others are better to be described along with the prototype working way It will be thoroughly detailed in next sections 6 WORKING PROCEDURE SEARCH FUNCTIONALITY In this prototype 3 it is possible to carry out the following searches random search search by text default and except options search by image sample search by mixed text and image sample tuning old relevance feedback In this first version of prototype 3 it is not possible to use personal images stored in the computer so for image search it is mandatory to carry out previously a shuffle search This way several images will be retrieved and one of them will be used as sample image In later versions of prototype 3 it will be possible to upload local images or sketches to the search engine In next figure the interface for search functionality is shown 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 12 43 Select sectors All Nore
34. nto one of these two folders is similar to the fact of copying an image into a specific directory The user can even select this directory by clicking on Set directory in the context menu of the personal and collective sphere as it is shown in next figure Minimize Maximize Restore Group Ungroup Mapping Slideshow Display mode Set directory Close Fig 34 Set directory in the personal sphere to store images Buscar carpeta amp O Prueba controles curvi amp ED Prueba dll mapping CV ED Roadmap P3 Sphere Collective amp ED versiones enviadas a we7 a ED wre e weg Crear nueva carpeta Aceptar Cancelar Fig 35 Browsing in the PC to set directory in the personal sphere to store images 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 36 43 This directory can be changed as many times as required Moreover it is possible to classify the images into as many subfolders as desired For that it is needed to double click on every folder that is contained in the personal or collective spheres It can be handled in the same way as the rest of spheres and even removed if desired by the user This way the user can store and order the images in the way he prefers for his personal use personal sphere or to share with his colleagues collective sphere The directories and folders can be really generated by the user by means of Windows options Ml TRENDS Softwar
35. ored as a favourite image not available in this first version Print the image is printed if any available printer not available in this first version Save as the image can be saved and renamed Search with the search by image is launched and that image becomes the sample query Refresh the image url is accessed again to reload the image 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 26 43 Properties the following information is shown name size date web source sector and semantic description o Name how the image file is named o Size its dimension in Kbytes o Date if available EXIF date o Web source source where the image was grabbed o Sector sector in the database where the image is stored o Semantic description the semantic adjectives and coocurrences are described not available in this first version e Properties Name A ke Master Car Design jpa Size Date Web source http www carbodydesign com events 2005 08 15 master car design audi Master Car Design jpg Sector Semantic description Fig 20 Information about the properties of every image e Zoom and zoom for an image Use the scroll bar on the left to zoom in and out the images 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 27 43 Pl TRENDS Software DER Simple Advanced History K Wow sors SC gt 8 ma
36. p Lifestyle Fig 24 Zoom in of the sphere 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 30 43 For Zoom Select the sphere and press Shift left click as many times as wanted fi TRENDS Software Simple Advanced History K A Le Search Keywords car 996 amp Pallets Groups m Mapping gt Statistics amp Lifestyle Fig 25 Zoom out of the sphere The zoom slide bar in the bottom right of the interface can also be used to zoom in and out the whole workspace 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 31 43 Fl TRENDS Software Simple Advanced History K A QX Search s ast Keywords gt E Image Tuning s Sectors eg Options amp Pallets mm Groups Mapping jj Statistics amp Lifestyle Fig 26 Zoom out of the working space by using the zoom slide bar Fj TRENDS Software simple Advanced History R J QY sear ac Keywords 12 Tuning x e gt E Image gt gt gt gt Options Ga car 996 keim a lt 1 1 amp Pallets Groups in Mapping f Statistics amp Lifestyle Fig 27 Zoom in of the working space by using the zoom slide
37. s of a search by text The interface manages properly the available functionalities every time For instance when a search by text is launched tuning cannot be unfolded at the same time In the same way when tuning is selected no other panel can be unfolded but sectors 6 3 SEARCH BY IMAGE It can be carried out when an image is available by any means In the first version of prototype 3 the only way of obtaining an initial image is by previous search or stored In personal or collaborative spheres It is possible to drag and drop any result image as sample for a query To make a query by image sample it is needed to click on Advanced Search tab 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 19 43 Pl TRENDS Software So Simple Advanced History K A Qy Search 5 ac Keywords gt a Image gt l r Tuning gt ia Sectors gt Options gt Ransom 402 a Pallets Groups Mapping f Statistics A Lifestyle A IR is Fig 11 Advanced search menu The user should click on Image and eventually in Sectors to select them The following interface appears M TRENDS Software FEER Simple Advanced History K A Qy Search Random 400 SEB a set BA s m 4 ney P Tuni gt ba 0 Sram Sectors gt ot Sa 2 wo age 2 FER
38. ve s sphere in the interface that means real directories in the user computer 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 10 43 History of the searches in the session maintained after the user has exited the application Handling and movement of spheres over the workspace The display functionalities are mosaic single big and small messy All these functionalities will be thoroughly explained in next sections It was proposed an innovative way of displaying the results by spheres The information shown there is Number of total images available Random 400 Options Displayed BS es results l Minimize Maximize Restore Close Page shown over Zoom of the images the total Scrolling the k number of results of the available shown page pages Fig 2 Sphere for the visualization of the retrieved results The following elements are identified Displayed results the results retrieved by the search engines are ordered by ranking 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 11 43 according to the parameters of the search Those fitting better the criteria are higher ranked and retrieved first Number of displayed images of the total available once a search is launched the system identifies how Many results are available They are ranked In the sphere only a certain number is displayed Next page of i
39. websites T2 2 the definition of a procedure for the mono sector mappings ambiences and pallets definition and statistics module T2 3 the definition of the first design elements for the graphic interface T2 4 the definition of the software architecture of the TRENDS system 2 5 the prototype 2 was developed by month 18 for the first tests with the end users in order to achieve the final TRENDS software The functionalities gathered in T1 4 have been translated into options available from the interface The first version of the graphic interface is D2 4 The interface of prototype 2 was developed in 16 1 Prototype 2 was the first interactive software coming from the design of the interface in ppt format In this first interactive version the priority was to create a tool that allowed validating the main functionalities of the software and to verify that the integration among the different software modules was 2008 TRENDS Consortium Members D6 2 User interface description P3 30 06 08 4 43 possible It was fully achieved The prototype 2 was evaluated along the different and periodic test sessions with the end users This lead to the identification of a series of weak points related to both dimensions this is the quality of retrieved results and the functionalities performance and the interface itself As output and according to the users proposals a new interface was designed that would gather all the improved elements This is
40. y of the database stands on both servers which is acceptable as this prototype has no update functionality for the database Only the user interface stands on the client machine The user interface is independent from the servers It communicates with the servers by means of XML files XML protocol is used for data exchange The communication protocol has been defined among the servers and the interface XML for the query by text by image random by mixed text and image XML for the advanced options of search suggestions ontologies pertimmizers XML for the retrieval of results XML for the tuning old relevance feedback XML for mapping XML for pallets XML for clustering XML for text and image statistics HTTP POST for delivery of XML files with the requests and responses It has been necessary to perform the integration with every server Integration with Image Search Engine ISE INRIA Robotiker It was verified that o the user interface sends a request to the ISE o the ISE receives that request o the ISE verifies the format of the request is correct o the ISE sends back the response to the UI o the Ul receives the response o the UI verifies the format of the response is correct o the Ul shows the images retrieved in the planned workspace Integration with Text Search Engine TSE PERTIMM Robotiker It was verified that o the user interface sends a request to the TSE o the TSE receives that request o t
Download Pdf Manuals
Related Search