Home

Dokument_1.

1. 13 1 5 GENERATION AND MEASUREMENT OF EEG eeeeeeeeeeeeeenneeeeeeeeeeeeeeeeenneeeeeeeeees 14 1 5 1 Generation in the brain 2 2n 22 nennen 14 1 5 2 Electrode POSING an es 15 1 5 3 Montage and measurement modes oooooooWo oo 16 2 THE OPENVIBE PLATFORM nungen 19 2 1 OVERVIEW ee re 19 2 2 INSTALLATION AND COMPATIBILITY nee een 20 2 USER MODES scene ee ee en 20 24 THE ARCHITECTURE ne aan bob 21 25 PLUGINS ne KN ne abi 22 26 TOOLS ko Konon ik 23 2 7 EXISTING SCENARIONS AND TUTORIALS 0ccceccecceeececcecececceccaeececetecaueceeeaeeceeeass 24 3 EEG DATA ACQUISITION AND PREPROCESSING coooo W Woo 25 OS HARDWARE was chins ana Sintec han haan Sta Sn es ha en Slat hea ea stn 25 3 2 PREPARATION AND INTERFACING oooooo o o o o ooW mna 27 3 2 1 Dealing with Impedance ooooo o W oom Wanna 27 3 2 2 Connecting the Server and the Designer oooooooooo 28 3 2 3 Channel Selection and Naming ooooooo Woo 29 3 2 4 Sampling Frequency oooooo o WWW mma 29 3 3 ARLIFAGTSIN THE EEG sma 30 6 6 5 Denok nata 30 3 32 EIO EA A E er er Cone ee 31 3 3 3 Avera gN Sa NN Bu Erna 32 3 3 4 Signal to Noise Ratio nennen ea 33 34 EVOKED POTENTIALS nee LP aici eet anc 34 3 4 1 Definition and MOA S 3s sisse cis scene aaa an uan 34 34 2 NOISE and RE Sa na Bana Naa abi 35 4 EXPERIMENTS AND RESULTS AS 37 41 THE O00 SPELLER ae einnehmen 37 4 1 1 Experiment in B
2. 2010 24 W v Drongelen Signal Processing for Neuroscientists Amsterdam Elsevier 2008 25 P Husar Biosignalverarbeitung Berling Heidelberg Springer Verlag 2010 26 S M Dunn A Constantinides and P V Moghe Numerical Methods in Biomedical Engineering London Elsevier Academic Press 2006 27 M Werner Digitale Signalverarbeitung mit MATLAB Fifth ed Wiesbaden Vieweg Teubner Springer Fachmedien 2012 28 B Rivet A Souloumiac V Attina and G Gibert xDAWN Algorthm to Enhance Evoked Potentials Application to Brain Computer Interfaces GIPSA lab Grenoble Institute of Technology Grenoble 2009 29 M Vinther Improving Evoked Potential Extraction Logicnet Orsted 2002 30 E Donchin M K Spencer and R Wijesinghe The Mental Prosthesis Assessing the speed of a P300 Based Brain Computer Interface IEEE 58 Transactions on Rehabilitation Engineering vol 8 no 2 pp 174 179 June 2000 31 H Cecotti and A Gr ser Time Delay Neuro Network with Fourier Transform for Multiple Channel Detection of Steady State Visual Evoked Potentials for Brain Computer Interfaces Lausanne 2008 32 F Bie mann New Methods for the P300 Visual Speller Dept Empirical Inferences MPI for Biological Cybernetics T bingen 2006 59 60 List of Figures Figure 1 1 Illustration of a simple Brain Computer Interface 7 3 Figure 1 2 A block diagram o
3. BCI SNR GUI VR fMRI NIRS Analog Digital Converter Electroencephalogram Electrocardiogram Electromyogram Electrocardiogram Central Nervous System Discrete Fourier Transformation Fast Fourier Transformation Linear Discriminant Analysis Evoked Potentials Visual Auditory Evoked Potentials Steady State Visual Evoked Potentials Brain Computer Interfaces Signal to Noise Ratio Graphical User Interface Virtual Reality Functional Magnetic Resonance Imaging Near Infrared Spectroscopy Table of Contents ACKNOWLEDGEMENTS une III PE SUA aan ee IV LIST OF ABBREVIATIONS naeh V 1 INTRODUCTION HUMAN COMPUTER INTERACTION nn 1 TI OVERVIEW ee ea Na eat 1 1 2 AIM AND SCOPE GE THE THESE sancscvesconwtneuensaitaieseunwsnauaweaittanennieunnsouainneniensonunsenialtuesiong 2 1 3 THE BCI FRAMEWORK aa aa ba Sama 2 13 1 PringipleSrihe BGI see ee ama min kana 2 1 3 2 Integrating Biofeedback ooooo W Woo 3 1 3 3 History of the BC see ea 5 1 3 4 Other Methods of Signal ACgUISItION ooooW oo 5 1 3 5 Classification of BE Sc ana maa dmna 6 1 4 PHYSIOLOGY OF THE PEG ci bina banana ama ha 7 14 1 BAGH QR OUING anne een 7 14 2 Electrical Activity in the Brain 8 1 4 3 Neuron ACINNY co ba BN Baen 9 1 4 4 Action Potentials serene ste a 10 1 4 5 Characteristic Frequency Bands of the EEG u 11 1 4 6 Geography of the EEG Waves Mapping Function to Region
4. 00 16 1172 16 0547 62 50 19 1113 19 0547 56 60 Mean delay 60 12 ms Standard deviation 2 24 ms Latency 60 12 2 24 ms Table 6 Unipolar AEP recording on same computer client same as server Signal time s Stimulus time s Delay ms 4 1016 4 0547 46 9 7 1011 7 0547 46 4 10 1220 10 0547 67 3 16 1000 16 0547 45 3 19 1001 19 0547 45 4 51 We ignore the runaway value in row 3 to achieve Mean delay 46 00 ms Standard deviation 0 79 ms Latency 46 0 79 ms We can infer that the unipolar signal is at least 14 ms faster on a same computer recording as compared to the network setup Without analyzing hardware and software performance in these experiments we can generally conclude from the results of tables 2 3 4 and 5 that signal acquisition will always have a time delay It is also clear that unipolar recording delivers the signal faster than the bipolar recording and on the whole network acquisition is slower than acquisition on the same computer The smaller standard deviation values also indicate that the timing of the signals and stimuli is more exact compared to network acquisition How latency affects performance of a particular BCI application like the visual speller and what magnitudes would be admissible is not yet clear Eliminating latency does not guarantee good performance however According to Cecotti amp Graser 31 reducing BCI latency typicall
5. and fed into the one hand into the preprocessing pipeline and xDAWN spatial filter which has been trained in step 2 on the other hand nothing is done to the raw signal Both signals are then fed into separate stimulus based epoching boxes set at duration of 600 ms each The boxes synchronize the ERPs with the target event and the rest of the signal with the non target event The averages of these two blocks are computed every second aggregated and used to train the linear discriminant algorithm of the classifier The classifier should now be 39 able to detect a p300 target waves and discriminate it from the background non target waves Step 4 Using the Speller online Having trained the spatial filter and the classifier a new signal was recorded for an online session It was preprocessed in the same pipeline as in step 2 The output of the xDAWN filter was passed to 12 separate but identical pipelines one for each repetition of row column step 3 above At the end of the pipeline is a classifier LDA which after calibration passes the output to a voting classifier configured default to reject non target input and pass the target character to the P300 speller visualization for rendering on the grid A target character is intensified in green Figure 4 2 below This would be the character the algorithms have predicted as intended input from the user and is displayed as the result for each of the 10trials Figure 4 2 A typical ou
6. figure 3 2 the client s and server can run on the same machine or on different machines on a network We use both interchangeably li Acquisition Client Acquisition Server x Acquisition Client all i Acquisition Client Network Figure 3 2 Client server setup in OpenViBE 23 After starting the acquisition client the acquisition driver matching the hardware type has to be selected in the drop down list We chose TMSi Refa32B matching our amplifier figure 3 3 The TMSi Refa32B driver is still rather unstable and crushed on several occasions 28 Fr OpenVIBE acquisition server 3 26 Einstellungen lt Verbinden f gt Wedergebe Sample count per sent block x X Device drift 0 00 ms tolerance is set to 2 00 ms Driver TMSi Refa32B unstable f7 Driver Properties Connection port 1024 0 host connected Figure 3 3 Acquisition server interface Hardware driver number of electrodes and their names sampling frequency and other settings can be done here 3 2 3 Channel Selection and Naming To get rid of unwanted noise in the signal it is important that the number of channels be set to the number actually used in the acquisition The default number is 32 Unused electrodes must also be decoupled from the amplifier Appropriately naming the channels Menus Driver properties and setting on the acquisition server
7. targets It is odd ball because the events on which it depends can be classified into two categories where one of them the target event is rarely presented Also the task to be performed depends on these two events The rare event elicits an ERP with a P300 component Since the events are mixed determining wanted results is purely statistical and is dependent on a classifier algorithm On average nearly 20 of all flashes will contain the target character and the other 80 will not 5 Theoretically by determining which rows and columns elicited an ERP and further determining their intersection cell the computer is able to predict the character the user was focusing on However the recorded EEG contains elicited ERPs as well as other brain activities noise including artifacts due to involuntary blinking decreasing the signal to noise ratio SNR of the P300 waves According to Donchin 30 increasing classification accuracy is done by averaging epochs over repeated trials Many repetitions decrease the number of characters minute the user can spell however The xDAWN algorithm employs a Bayesian linear discriminant analysis to synchronize target stimuli with the evoked potentials and uses spatial filtering to enhance their power thereby increasing efficiency of classification More on this in Rivet et al 28 37 Figure 4 1 The alphanumeric grid and visual stimulus the subject focuses on a character 23 4 1 2 Method A black box
8. to enhance a time locked signal component like an evoked potential in noisy measurements Signal averaging in the time domain spatial assumes that Measured signal and noise are uncorrelated The timing of the signal is unknown A consistent signal component exists during measurement en Zt ae Noise in the signal is purely random with zero mean For a measurement x consisting of signal s and noise n over N trials x k sj k nj k 4 represents the kth sample point in the jth trial The mean of the N sampled trials is given by N N s AI 1 on FY lt ss Ol 5 j l j l 32 To reduce noise we have to choose N large enough such that x k y gt s k n From equation 3 5 we can derive the variance of o 6 Var x Indicating that the estimate of s in the average x improves with a factor of a 24 3 3 4 Signal to Noise Ratio It is generally impossible to make a noise free signal measurement if the measurement chain is free of errors some will be introduced by the instruments used amplifiers electrodes etc The aim of filtering is to make the noise component comparably small relative to the signal component Signal to noise ratio of a measurement rms signal 7 SNR 10 logy red JB 20 logio ms noise rms noise is the comparison of their power or amplitudes where in discrete time series power is the mean squared amplitude ms zlr N x 8 i 1 And amplitude is the root of the mean
9. 8 13 F L Nicolas Alonso and J Gomez Gil Brain Computer Interfaces a Review Sensors pp 1211 1279 2012 14 S Sanei and J A Chambers EEG Signal Processing West Sussex Chischester Southern Gate John Wiley amp Sons Ltd The Atrium 2007 57 15 E Niedermeyer and F L Da Silva Electroencephalography Fifth ed Philadelhia Pennsylvannia Lippincott Williams and Wilkins 2005 16 E Basar EEG Brain Dynamics Amsterdam Elsevier North Holland Biomedical Press 1980 17 Natural Health School A Home Study Program in Herbalism Nutrition and Natural Health 16 07 2012 Online Available www naturalhealthschool com 18 H H Dr Mueller Practice in Clinical and Health Psychology 2012 Online Available http www drmueller healthpsychology com 19 M Teplan Fundamentals of EEG Measurement Measurement Sicience Review vol 2 2002 20 BCI2000 BCI 2000 July 2012 Online Available http www bci2000 org 21 J Malmivuo Bioelectromagentism WebStat 12 Febraury 2010 Online Available http www bem fi Accessed 20 July 2012 22 C Brunner BCI Software Platforms in BC Software Platforms Graz Austria 2005 23 Y Rennard F Lotte V Delannoy G Gibert E Maby and M Congendo OpenViBE An Open Source Software Platform to Design Use and Test Brain Computer Interfaces in Real and Virtual Environments Presence Teleoperators amp Virtual Environments
10. AWN spatial filter did not narrow down the space to enhance the accuracy of the classifier algorithm Accuracy of the classifier depends heavily on the signal to noise ratio of the evoked potentials so if the filters do not perform optimally the classifiers will also falter According to BieBmann 82 shortening inter stimulus interval ISI does not adversely affect the accuracy of the classifier We used an ISI of 150ms default setting of which 50ms was no flash duration Slowing down the flash duration three fold to achieve ISI of 450 ms improved the classifier performance but not that of the speller with the two subjects as can be seen from the results see 4 1 4 Latency in the acquired signal would seem to suggest that the evoked potentials are not synchronized with the stimuli therefore leading to improper classification But the P300 wave has a latency of about 600ms and the highest recorded latency on our system was about 63ms which would shift the center of the wave from 300 to about 360 400 ms which is within range On evidence of the visualization experiments in 4 2 1 we can conclude that the hardware and the software platform together can be used to acquire a good EEG signal It also has to be noted that visualization of the EPs in OpenViBE could not be achieved hence the migration to MATLAB On the flipside OpenViBE will soon fully support MATLAB scripting making it possible to replace some of the boxes in the pipelines wi
11. Brain Waves EEG Tracings Beta en A Mn a MN Mp wy 13 30 Hz N 1 a N A Alpha a AV Nu WwW WWW MV MW 8 13 Hz mma Ww NV env Miya un N N A A Pee NA N m 0 d v WV J Nan f V A Y U en 4 8 Hz V V y J A j t z ES Delta 5 Vv j 3 0 5 4Hz Figure 1 6 Typical encephalographic rhythms and their frequencies 18 Alpha a rhythm 8 13 Hz This rhythm is most prominent in normal subjects who are relaxed and awake the activity is suppressed when the eyes are open The largest amplitude of this rhythm can be measured in the occipital region of the brain Beta B rhythm 14 30 Hz This is a fast rhythm with low amplitude associated with an activated cortex and which can be observed for example during certain sleep stages The beta rhythm is typically observed in the frontal and central regions of the scalp 12 Delta 5 rhythm lt 4 Hz This rhythm is typically encountered during deep sleep and has large amplitude thus low frequency It is usually not observed in the wake normal adult but is indicative of cerebral damage or brain disease encephalopathy Gamma y rhythm gt 30 Hz The gamma rhythm is related to a state of active information processing of the cortex Using an electrode located over the sensorimotor area this rhythm can be observed during finger movements Theta 6 rhythm 4 7 Hz This rhythm mainly occurs during drowsiness and i
12. RUPRECHT KARLS UNIVERSIT T HEIDELBERG Studiengang Medizinische Informatik BACHELOR THESIS Brain Computer Interfaces OpenViBE as a Platform for a P300 Speller Herbert S Kisakye Matrikelnummer 172588 E Mail hkisakye stud hs heilbronn de Heilbronn August 6 2012 Supervisors Prof Dr Rolf Bendl Dipl Inform Med C Maier ACKNOWLEDGEMENTS This thesis is based on research did in the Bio Signal Laboratory at the Heilbronn University of Applied Sciences in the summer of 2012 Without the help of different individuals would never have been able to achieve much am greatly indebted to Dipl Inform Christoph Maier for his readiness to take time out of his busy schedule working on his Doctorate to answer questions and give me invaluable guidance would like to thank Prof Dr Rolf Bend immensely for pointing me in this direction of a relatively new yet exciting field Of course this work would never have come to fruition without the people mostly fellow students at the university that sat through many hours of repetitive sometimes monotonous experiments and those that helped with a few technical aspects Thank you all so much Lastly a special thank you to all my friends and family for the emotional support You kept me going Herbert Sunday Kisakye Abstract Aside from hardware a major component of a Brain Computer Interface is the software that provides the tools for translating raw ac
13. T temporal P posterior O occipital 19 The numbering represents left lobe odd and right lobe even Figure 1 8 Original 10 20 elecrode placement system with 19 electrodes B the extended one has 70 C Adopted from 20 15 The actiCAP used in the experiments here follows the 10 20 system see 3 1 Hardware however it follows an extended 32 active electrode configuration with added Reference and Ground 1 5 3 Montage and measurement modes For all EEG acquisition in the experiments a variation of the montage shown in figure 1 9 was used used electrodes are marked in yellow see 4 1 2 4 2 1 and 4 2 2 Figure 1 9 Positions of electrodes used for acquisition in the experiments 20 Besides the international 10 20 system many other electrode placement systems exist for recording electric potentials on the scalp Measurement of evoked potentials is usually done using either unipolar or bipolar placement Bipolar measures the potential difference between two electrodes while unipolar measures the potential of an electrode referenced to a neutral electrode or to the average of all electrodes 21 These two techniques are depicted in figure 1 10 below Figure 1 10 Illustration of A Bipolar and B Unipolar modes of measurement Source http www bem fi book index htm 17 2 The OpenViBE Platform 2 1 Overview OpenViBE is a free and open source software platform for designing testing and using B
14. a monitor as the stimulus Figure 4 5 The subject was instructed to focus on the center as the squares alternately changed color between black and white There was 44 minimal ambient light with the interstimulation interval set to 2 seconds in the LUA script Using the actiCap the subject s EEG was recorded with electrodes Pz O4 Oz placed in the region of the visual cortex Figure 1 7 Cz was used as a reference electrode The recorded signal was analyzed in MATLAB for the transient P300 peaks anna 3d Figure 4 5 Visual stimulus constantly timed pattern reversal as seen by subject According to S rmno amp Laguna 5 a normal subject s VEP waveform is described by a small positive peak a larger positive peak occurring about 75 ms after stimulus N75 and a large positive peak about 100 ms after stimulus P100 The duration of the VEP may extend beyond 300 ms In figures 4 6 and 4 7 we observe all these three characteristics which is proof that the hardware used and the software platform are capable of reproducing an expected response potential 45 Figure 4 6 Red is average over background EEG blue is average over rare potential Figure 4 7 Characteristic P300 peak in 200 400 ms range 46 4 2 2 Auditory Evoked Potentials In this experiment the subject wore a set of stereo headphones With his eyes closed he was instructed to concentrate and mentally register
15. al stability 1 2 Aim and scope of the thesis The Heilbronn University of Applied Sciences Hochschule Heilbronn has recently acquired equipment that can be used for recording the Electroencephalogram EEG in the Bio Signals Laboratory These devices come with a fairly high price tag The idea was to take advantage of the availability of non proprietary open source software for real time processing of brain signals integrate it with the acquisition device and build a functional laboratory based BCI application the P300 visual speller Currently there are not fewer than 10 such open source BCI platforms Among them OpenViBE was chosen basing on the functionality and usability analysis See chapter 2 and the OpenViBE website 6 1 3 The BCI Framework 1 3 1 Principle of the BCI A typical BCI system involves acquisition of electrical signals from the brain s neuron activity and transforming them into commands to control an application The user s brain activity is recorded using a single or multiple electrodes attached to the scalp for the EEG or directly onto the cortex by surgical means for the ECoG Another method is to use depth electrodes implanted in the brain The most common BCls are EEG based as they are not only cheaper but also easier to set up The acquired EEG signal is digitized by an analog to digital converter and fed into a computer for processing Different characteristics can be observed in this ongoing oscill
16. an be subtracted out by the amplifier at the acquisition level While impedance or resistance can be minimized and wires shortened it is harder to eliminate noise caused by heating in the amplifier for example 3 3 2 Filtering Filters are applied in the frequency domain to eliminate unwanted components or features from incoming signals and to minimize artifacts The attenuation of unwanted components might be partial or complete depending on the choice of filter settings The aim of filtering should be to improve signal quality gain by minimizing background noise or interference Filters do have one drawback however they are usually associated with some loss of information and used wrongly they could lead 31 to total loss of the signal Two filtering techniques are employed spatial and temporal filtering Spatial filters combine data from two or more recording locations to derive features of a particular characteristic e g frequency Temporal filters employ a combination of frequency and amplitude restriction methods like band pass filtering frequency and Fourier analysis amplitude 7 Some common filters in EEG signal processing are e Low pass filter high frequencies are attenuated e High pass filter low frequencies are attenuated e Band pass filter passes in a given frequency range only e Notch filter rejects just one specific frequency example of band pass 3 3 3 Averaging Averaging is done
17. artition 422 180 sec algorithm Classifier trainer Finished with partition 422 180 sec algorithm Classifier trainer gt Finished with partition 422 180 sec algorithm Classifier trainer gt Finished with partition 422 180 sec algorithm Classifier trainer gt Finished with partition 422 180 sec algorithm Classifier trainer gt Finished with partition 422 180 sec algorithm Classifier trainer Finished with partition 422 180 sec algorithm Classifier trainer Finished with partition 422 180 sec algorithm Classifier trainer gt Finished with partition 20 performance 86 1111 422 180 sec algorithm Classifier trainer gt Finished with partition 20 performance 81 9444 422 180 sec gorithm Classifier trainer Classifier performance on whole set is 84 0278 sigma 4 35899 20 performance 83 3333 20 performance 77 7778 20 performance 79 1667 20 performance 83 3333 20 performance 91 6667 20 performance 80 5556 20 performance 90 2778 20 performance 83 3333 20 performance 79 1667 SSSSSSSSSISN Figure 4 4 Console Information from classifier performances of subject B 75 49 and subject C 84 03 with the inter repetition duration of 450 milliseconds 43 Clearly there was a marked improvement in the classifier performance 9 6 and 8 7 for B and C respectively However they both had a 0 10 score in the online spelling session This is counterintuitive because the classifi
18. as AEPs and VEPs Exogenous systems do not require extensive training of the user compared to endogenous ones and are easier to set up Endogenous BCls often require the use of neural feedback to enable the user to learn to generate specific wave patterns like slow cortical potentials or sensorimotor rhythms Their advantage is that severely affected persons can learn to operate them 13 Synchronous vs Asynchronous Asynchronous BCIs analyze an EEG patterns in a predefined time window leaving out anything that comes before or after The user is only supposed to send commands during epochs determined by the system Synchronous BCls employ detection of specific events through continuous analysis of the user s EEG It offers a more natural interaction for the user 1 4 Physiology of the EEG 1 4 1 Background Recorded work on neurophysiology goes back nearly 200 years to Carlo Matteucci 1811 1868 and Emil Du Bois Reymond 1818 1896 who were the first to register electrical signals emitted from muscle nerves using a galvanometer The discoverer of the existence of human EEG signals however was Hans Berger 1873 1941 a German neurologist and psychiatrist In 1929 he presented a report that included the alpha rhythm 8 13 HZ as the major component of the EEG signals 14 Work by other scientists in Europe and America in the quest to understand the functioning of the brain and the central nervous system the causes of diseases like brain tu
19. at accepts MATLAB code 23 This is still under development by OpenViBE Some boxes also support a scripting language called LUA for configuring the settings These can be used for example to determine when to send stimulations in an application 22 2 6 Tools The acquisition server provides a generic interface to various kinds of acquisition devices e g EEG or EMG systems Server connection to the hardware is dependent on manufacturers specifications some devices will be shipped with a specific SDK while others use a communication TCP IP protocol over a network serial USB connection Others will need proprietary acquisition software The Designer Figure 2 2 is used mainly by the author and helps him to build complete scenarios based on existing software modules using a dedicated graphical language and a simple GUI Generic stream reader InjOut Set SE The Generic Stream Reader box reads an QpenVi6E file from disk and sends its content to the following box The OpenWi E file format can store any stream of the OpenViBE platform into a file The Signal Displaybox displays the content of the file You can browse each box documentation by selecting the box and pressing F1 Figure 2 2 The designer showing a simple scenario The tools enable the graphical design of a BCI system by adding and connecting boxes representing processing modules without writing a single line of code The author has acce
20. atory activity depending on varying degrees of the user s mental or physical state a relaxation EEG is different from the one recorded when the subject is performing a mental activity for example Specific features like amplitudes of evoked 2 potentials are extracted and passed to translation algorithms for training After a training period the algorithms are able to identify these specific features data mining and machine learning in a subsequent ongoing EEG which they can classify and translate into commands which reflect the user s intent to control a simple word processing program a robotic arm control a wheel chair or any such devices Figure 1 1 This process involves little to no muscular activity and can be understood as an interface between the brain and an application BCI SYSTEM DEVICE COMMANDS Figure 1 1 Illustration of a simple Brain Computer Interface 7 BCls can use signal features in the time domain like amplitudes of evoked potentials or those in the frequency domain like mu u and beta range B amplitudes 7 employed in motor imagery BCIs 1 3 2 Integrating Biofeedback Bruce Eugene 8 defines feedback as sensing of the output of a system and transmission of this signal back to the system input in order to influence the future output of the system Through biofeedback the user learns to control those features of the signal that reflect his or her intent by selectively influencing the amplitu
21. de and Imagined movement causes the same brain activity as real movement signals to control hand leg BCls like prosthetics and game control can be realized through thought 3 power of their output According to Birbaumer 9 most of the clinical BCI studies in human patients use feedback of EEG oscillations or of event related potentials ERPs The subject receives visual or auditory online feedback of his or her brain activity and tries to voluntarily modify a particular type of brain wave this is known as self regulation Figure 1 2 Brain Computer Interface Classification Application interface Application Interface Digital Signal Processing Feature extraction Preprocessing Signal acquisition Acquisition Interface Applications such as Spelling program or neuroprosthesis Feedback Figure 1 2 A block diagram of a Brain Computer Interface Adopted from 5 S rnmo et al 5 found that the EEG exhibits variability due to factors such as time of the day fatigue and hormonal level The training phase of the user and that of the algorithms may not be a onetime event but may need to be repeated on a regular basis in order to achieve good performance The user must develop and maintain good correlation between their intent and the features the algorithms extract for use in the BCI application 1 3 3 History of the BCI BCI is a relatively new field Research started several d
22. ecades after Hans Berger first discovered the electroencephalogram in the 1920s In the 1960s applying easy to use noninvasive methods studies were done on the development of neuro feedback in which monkeys would learn to voluntarily increase or decrease the power of their alpha waves the predominant amplitudes in an EEG They monkeys learnt to increase or decrease firing rates of the neurons in the motor cortex and could move a prosthetic arm simply by thinking about it In the 1970s Jacques Vidal published his work on a government sponsored research in bio cybernetics and human computer interactions at the University of California Los Angeles UCLA Brain Research Institute He showed how brain signals could be used to build up a mental prosthesis 10 He is also credited with coining the term Brain Computer Interfaces Lawrence Farwell and Emanuel Donchin developed an EEG based BCI In Talking off the top of your Head published in 1988 11 they outline an algorithm used in the decision making process of a P300 speller to identify target and non target potentials in an ongoing EEG signal This was the first time event related potentials ERPs were used in a BCI application 1 3 4 Other Methods of Signal Acquisition Signal acquisition for BCls can be grouped into two depending on whether a surgical procedure is required to implant electrodes in on the brain to measure its electrical activity or not Non invasive methods requ
23. ell Figure 1 3 Impulses received through the dendrites are transported along the axon to its terminal branches through to the synapse where they are picked up by other connected neurons s Source http springvisualculture 1b blogspot de 1 4 3 Neuron Activitv The central nervous system CNS is generally an interconnection of neurons and glial cells to create a communication network of both chemical and electrical activity The activities in the CNS are mainly related to the synaptic currents transferred between the junctions synapses of axons and dendrites or dendrites and dendrites of cells A negative potential of 60 70 mV may be recorded under the membrane of the cell body compared to the extracellular environment This potential changes with variations in synaptic activities relating to distributions of Na K and CI across the membrane Figure 1 5 If an action potential travels along the fiber which ends in an excitatory synapse an excitatory post synaptic potential EPSP occurs in the following neuron If two action potentials travel along the same fiber over a short distance there will be a summation of EPSPs producing an action potential on the post synaptic neuron providing a certain threshold of membrane potential is reached If the fiber end in an inhibitory synapse then hyper polarization will occur indicating an inhibitory postsynaptic potential IPSP Following the generation of IPSP hyper polarization there i
24. er are both programmers on the other hand the author and the operator do not need any programming skills The Developer OpenViBE comes with a Software Development Toolkit SDK that enables the programmer to design test and add new functionality to the platform These could be on the Kernel or Plug in basis The Application Developer Uses the SDK to create standalone applications using OpenViBE as a library Applications range from visual scenario editors to VR applications that the BCI user can interact with 20 The author non programmer uses the visual scenario editor to arrange existing boxes to form a scenario He configures these scenarios in order to create a compete ready to use BCI system They however need knowledge of the internals of the platform as well as of BCI systems including basic signal processing The Operator Also a non programmer but with knowledge of neurophysiological signals they would be generally a clinician or a practitioner They simply run the pre built scenarios of the author Aided by visualization components they are able to monitor the execution of the BCI The User The user wears the acquisition gadget like the electrode cap and interacts with a BCI application by means of mental activity The application could be a neurofeedback training program a video game in virtual reality a remote operation in augmented reality etc The user seldom directly uses the OpenViBE platform but rather they int
25. er for all the participants is clearly very low and seems to suggest a randomness of result rather than a correct performance of the classifier on target non target discrimination 4 1 4 Problems and Mitigation The failure to achieve satisfactory results could not be directly attributed to any one factor The obvious exclusion here is subject related failure as performance was poor for all of them From hardware to acquisition and settings in the algorithms of different pipelines in the scenarios any of these could be the problem It could well be the software platform itself A first approach to validating the results was to try and adjust the speed of the row column flashes in both the acquisition and online scenarios to find out whether it improved the subjects score on the speller it was suspected that perhaps the rate of repetition at 150 ms was too fast leading to an interference of the P300 response wave and the following stimulus For two of the subjects one male B and the female C the experiment was repeated with the flash duration set to 400 ms Figure 4 3 making an inter repetition delay of 450 ms a 200 slow down The graphics in figure 4 4 show the obtained classifier performances 42 OVTK_Stimulationid_Label_01 OVTK_Stimulationid_Label_07 0 400000 0 050000 0 000000 fe eS RE 14 000000 Figure 4 3 Flash duration in the flashing sequence settings chan
26. er seems to have identified the evoked potentials correctly 4 1 5 Inference While the results above indicate that slowing inter repetition improves on classifier performance a corresponding improvement on the score in the online sessions was not registered It is probable that the output numbers of the classifier are misleading that it has failed to discriminate the P300 components from the ongoing EEG signal therefore there s no clear distinction of target and non target stimuli It has to be noted that evoked potentials are transient in nature and are in order of magnitude 10 times smaller than the ongoing background EEG In this case the EEG would be a noise masking the EPs which the spatial filters have to actively eliminate If the algorithms Spatial filter LDA classifier are working well it could be that the signal is leaking at some unknown point Furthermore the speed of data transfer on the network is unknown and the latency of triggers for the stimuli may not be synchronized 4 2 Visualizing Evoked Potentials We consider the visualization of evoked potentials as validation for the proper functioning of the acquisition apparatus and the software platform We seek to establish existence of the P300 EPs in the ongoing background EEG which potentials are crucial to the success of P300 speller BCI application 4 2 1 Visual Evoked Potentials The experiment was set up using pattern reversal of a chessboard graphic displayed on
27. eract with it 24 The architecture The OpenViBE architectures encapsulated in figure 2 1 is built around a kernel which employs a plug in manager to ensure functional extensibility Dynamic linki8g H gat run timef i I 1 Network I I commu I nication Figure 2 1 The OpenViBE software architecture 23 The plug in manager is able to dynamically load plug in modules e g DLL in Windows or so files in Linux and collect extensions from them This allows for a quick and efficient expansion of functions and algorithms Other managers around the kernel include Scenario manager for creating and configuring scenarios boxes can be added removed and configured to build a pipeline Visualization manager for displaying graphical information It relies on inbuilt libraries to render 2D and 3D graphical information in display windows Player manager for runtime control over scenarios It allows for playback stopping pausing and fast forwarding 2 5 Plug ins Three types of plug ins are used 1 The driver plug in allows addition of acquisition devices to the acquisition server by using the SDKs or a physical connection Plug ins for the algorithms allows developers to create and add new algorithms to the platform and encourages sharing and reuse of existing ones The box plug ins generally rely on algorithm plug ins to create different signal processing functionalities for the boxes Among the boxes is one th
28. f a Brain Computer Interface oooooooo 4 Figure 1 3 Structure of a neuron 2224444400nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn 8 Figure 1 4 The neuron membrane potential changes and current flow during synaptic activation recorded by means of intracellular microelectrodes 14 9 Figure 1 5 Changing the membrane potential of a giant squid by closing the Na channels and opening K channels 14I ooo o on 11 Figure 1 6 Typical encephalographic rhythms and their frequencies 18J 12 Figure 1 7 A functional map of the cerebral Cortex Wo 14 Figure 1 8 Original 10 20 elecrode placement system with 19 electrodes B the extended one has 70 C Adopted from 20 ooooooo W oo WWW 15 Figure 1 9 Positions of electrodes used for acquisition in the experiments 16 Figure 1 10 Illustration of A Bipolar and B Unipolar modes of measurement 17 Figure 2 1 The OpenVIBE software architecture 23 ooooooW 40 21 Figure 2 2 The designer showing a simple scenario oooooW nennen 23 Figure 3 1 Hardware layout for the experiment ooooooooWooWoW oo 26 Figure 3 2 Client server setup in OpenVIBE 23 oooo Woo Wa 28 Figure 3 3 Acquisition server interface Hardware driver number of electrodes and their names sampling frequency and other settings can be done here 29 F
29. ged from 100 to 400 milliseconds 84 7222 73 6111 697 117 sec algorithm Classifier trainer Finished with partition 13 20 performance 697 117 sec algorithm Classifier trainer Finished with partition 14 20 performance 697 117 sec algorithm Classifier trainer Finished with partition 15 20 performance 70 8333 697 117 sec algorithm Classifier trainer Finished with partition 16 20 performance 76 3889 697 117 sec algorithm Classifier trainer gt Finished with partition 17 20 performance 68 0556 697 117 sec algorithm Classifier trainer Finished with partition 18 20 performance 68 0556 697 117 sec algorithm Classifier trainer Finished with partition 19 20 performance 80 5556 697 117 sec algorit Classifier trainer Finished with partition 20 20 performance 79 1667 697 117 sec algorithm Classifier trainer Classifier performance on whole set is 7548614 sigma 4 54901 422 180 sec algorithm Classifier trainer Finished with partition 20 performance 81 9444 422 180 sec algorithm Classifier trainer Finished with partition 20 performance 84 7222 422 180 sec algorithm Classifier trainer Finished with partition 20 performance 84 7222 422 180 sec algorithm Class fier trainer Finished with partition 20 performance 77 7778 422 180 sec algorithm Classifier trainer Finished with partition 422 180 sec algorithm Classifier trainer Finished with p
30. he results as indicated in 4 3 4 5 this experiment was not a success and the aim of the thesis was in part not achieved On the other hand invaluable insight was gained through research into the technical aspect of a brain computer interface This has been documented in this thesis and it is hoped that it will provide a good basis for future works on this topic The reason s for failure to produce satisfactory results in this experiment are yet unknown but a few can be singled out for future reference The speller application involves four separate scenarios see 4 1 2 each with its own characteristic pipeline and settings for boxes and algorithms For the most part we used these unchanged It is likely that a few alterations could lead to better results but it also difficult to assess which part of the pipeline works well and which doesn t There was the question of loss of signal by the amplifier as evidenced by the sine waves in OpenViBE figure 4 9 and further observed during analysis of the signal in MATLAB We did not compare input to output results of the amplifier and where the inbuilt analog and digital filters were working well so it is unclear whether this was a contributing factor We were able to 53 establish the existence of typical transient evoked potentials synchronized with their target stimuli see 4 2 1 however the classifier algorithm in the online scenario did not identify these correctly It is probable that the xD
31. hey were further instructed to mentally count the number of times this character is intensified in a randomly flashing row or column A total of 10 trials number of characters are done in each session Each row column is intensified for a duration of 100 ms 0 1 sec with an interval of 50 ms from one flash to the next There are 12 flashes of each row and 12 of each column on the grid so each character is intensified 24 times during every trial From one trial to the next there s a delay of 4 seconds to allow the user to change focus on the next character In the scenario this is timed at about 30 seconds from the onset of one character to the next translating to ca 5 minutes for the entire session of 10 trials The data was stored in signals folder ov format with a time stamp Step 2 Preprocessing and training The recorded data file in step 1 is read with a generic stream reader A 4 order Butterworth band pass filter 1 0 20 0 Hz was used to filter out unwanted signals components Signal decimation is then used to cut a 1 second block of 16 samples into four blocks of 4 samples each The blocks are sampled again in epochs of 250 ms each Spatial filtering is then done using the xDAWN filter that reduces the signal space to 3 dimensions most significant for detecting a P300 ERP 23 This is the offline training of the filter Step 3 Training the LDA classifier In scenario 3 again the raw data file is read with the stream reader
32. iel rare aaa een 37 2 12 Method sahih ta ha 38 a13 Performante re BSI GA BA ae Bea a AMA a Giaa 41 4 1 4 Problems and Mitigation am omooomco bubukans keausan 42 45 TA i hi ATRRNN 44 4 2 VISUALIZING EVOKED POTENTIALS ai 44 4 2 1 Visual Evoked Polenlalsun a aan 44 4 2 2 Auditory Evoked Potentials 2 cooo ocooo oom mewesksataka 47 4 3 INVESTIGATING LATENG IEG 2 at area 48 5 DISCUSSION OF RESULTS nenne 53 6 OUTLOOK ee Manan aan akan a AES EE 55 BIBLIOGRAPHY sea 57 EISTEOE FIGURES oria a A E T A 61 LIST OP TABLES sans APPENDIX VIII 1 Introduction Human Computer Interaction 1 1 Overview We live by interacting with the physical world around us This interaction normally involves thought and action The brain through its output pathways the peripheral nervous system and muscles controls motor movement This way we are able to move our arms legs or any part of the body as a means of communication or performing day to day tasks if we are healthy However people with severe neuromuscular injuries such as after an accident or those suffering from neurodegenerative diseases like Amyotrophic Lateral Sclerosis ALS brainstem stroke or cerebral palsy soon lose part or all of this voluntary muscular activity tne communication path between brain and effector muscles having been cut off leaving only the cognitive function intact This in medical terms is referred to as the locked in syndrome sufferers are left full
33. igure 4 1 The alphanumeric grid and visual stimulus the subject focuses on a character Bl 38 Figure 4 2 A typical output from the P300 speller application in the online session 40 Figure 4 3 Flash duration in the flashing sequence settings changed from 100 to 400 Gd ta a ee ee ee eier 43 Figure 4 4 Console Information from classifier performances of subject B 75 49 and subject C 84 03 with the inter repetition duration of 450 milliseconds 43 Figure 4 5 Visual stimulus constantly timed pattern reversal as seen by subject 45 Figure 4 6 Red is average over background EEG blue is average over rare potential Na Sa na An PA AA PA na Pn mieten 46 Figure 4 7 Characteristic P300 peak in 200 400 MS TANGE ooo o oWoo 46 Figure 4 8 High pass filtered 1 Hz on Cz Pz Red is target and blue non target 47 Figure 4 9 Band pass filtered 1 70 Hz on Cz Pz Red is target blue non target 48 Figure 4 10 Display of acquired bipolar audio signal in OpenViBE indicating latency ING DISION OM nennen een hama sa bana bni 49 61 List of Tables Table 1 Signal acquisition methods in BCIS oooooWoWoW oom 6 Table 2 Results of the experiment involving 5 Subjects oo oooooooWoWoW oo 41 Table 3 Bipolar AEP recording on network 1 client 1 Server nn 49 Table 4 Bipolar AEP recording on same computer client same as server 50 Table 5 Unipolar AEP
34. ild an application 3 1 Hardware For the experiments in this thesis the following hardware Figure 3 1 was used 1 A 32 Channel active electrode control box from Brain Products that comes with Electrodes ch 1 32 and their connector 2 extra electrodes Ref and Gnd A 32 channel actiCAP standard 2 based on the 10 20 system Active shielding from 50Hz power line interference 2 A TMSi Porti32 Refa Amplifier A D converter from Twente Medical Systems International with the following specifications 24 unipolar recording ports 4 bipolar recording ports Digital ports Sampling frequency of up to 2048 Hz Channel Fibre optic cabel for speedy data transfer to the computer Bluetooth for wireless transfer 3 Two DELL Optiplex 755 Desktop computers with Intel Core 2 Duo CPU 2 66 GHz processor 4 0 GB RAM 32 Bit Windows Vista Service pac 2 Business Both having OpenViBE and the actiCAP software installed 25 er Windows Vista Speller with OpenViBE processing A SANA actiCAP BioSig Network fe Windows Vista 2 gt an lagi with OpenViBE q a at acquisition TMSi Amplifier AEP Figure 3 1 Hardware layout for the experiments When computers are used for recording EEG the acquired analog signal has to be amplified since it is of very low amplitude and digitized for further processing Each channel is sampled repeatedly at a fixed time interval sampling rate and each sample is converted in
35. ire no surgery However not all acquisition procedures involve the use of electrodes some like fMRI measure electrical activity by use of blood oxygenation level dependence BOLD The table below summarizes the different methods http Avww cs ucla edu vidal vidal htm Table 1 Signal acquisition methods in BCIs Method Measured quantity Invasive EEG Electrical potential No ECoG Electrical potential Yes Micro electrodes Electrical potential Yes MEG Magnetic field No fMRI BOLD No NIRS BOLD No Adapted from D Albis 12 1 3 5 Classification of BCIs There s no standard classification for BCls In some literature the acquisition modes are used thus invasive and non invasive BCls Other classifications are Synchronous Asynchronous BCIs Il Cued Spontaneous BCls Ill Dependent Independent BCIs Dependent vs Independent Although it uses brain signals a dependent BCI will still require the user to apply part of his her peripheral system A BCI built on visual stimuli involves the movement of the user s eyes and muscles On the other hand independent BCls rely on signals that can be triggered without involvement of muscle activity with audio stimuli the tasks only involve thinking Endogenous vs Exogenous In endogenous BCI a user spontaneously generates the brain signal used for control while in exogenous BCI the signal is generated as a 6 response to an external stimulus such
36. ise and EPs The recorded EEG contains P300 potentials as well as other brain activities muscular and or ocular artifacts leading to a very low signal to noise ratio SNR of the P300 potential 28 These brain activities are unrelated to the experiment and produce waves with higher frequency bands in comparison to the evoked potentials The frequency content of muscular movement artifacts can usually not be removed by low pass filtering Further the frequencies of the background EEG overlap harmonics with the evoked potentials such that conventional filtering cannot be used to achieve higher SNR 29 To cancel out the noise epochs of the evoked potential have to be averaged see 3 3 3 Sampling more epochs improves SNR 35 36 4 Experiments and Results 4 1 The P300 Speller 4 1 1 Experiment in brief The experiment is based on the odd ball paradigm first developed by Farwell and Donchin 1988 and analyzed in 30 It utilizes the P300 event related potential ERP elicited by instructing a subject to focus on a character contained in a 6x6 grid of alpha numeric characters A Z and 0 9 Figure 4 1 The computer generates flashing sequences of rows and columns at a preset frequency visual stimuli The user is asked to distinguish between a common stimulus non target and a rare stimulus target 28 This can be done for example by mentally counting up each time the target flashes in a row or in a column and ignoring the non
37. ity is a replacement of missing samples by the OpenViBE TMSi acquisition server 48 Figure 4 10 Display of acquired bipolar audio signal in OpenViBE indicating latency and distortion We further analyzed the bipolar recording in matlab Arrival of first epoch 8438 2048 4 1201 seconds Arrival of first stimulation 4 0547 seconds Table 3 Bipolar AEP recording on network 1 client 1 server Signal time s Stimulus time s Delay ms 4 1201 4 0547 65 4 7 1128 7 0547 58 1 10 1196 10 0547 64 9 16 1162 16 0547 61 5 19 1191 19 0547 64 5 49 Mean delay 62 88 ms Standard deviation 3 07 ms Latency 62 88 3 07 ms Table 4 Bipolar AEP recording on same computer client same as server Signal time s Stimulus time s Delay ms 4 1064 4 0547 51 7 7 1079 7 0547 53 2 10 1074 10 0547 52 7 16 1035 16 0547 48 8 19 1074 19 0547 52 7 Mean delay 51 82 ms Standard deviation 1 77 ms Latency 51 82 1 77 ms These results suggest that for the Bipolar recording the network connection slows down the signal by a minimum of 11 milliseconds Moreover the transport across the network between two different computers appears to increase the standard deviation of the latencies 50 Table 5 Unipolar AEP recording on network Signal time s Stimulus time s Delay ms 4 1147 4 0547 60 00 7 1162 7 0547 61 50 10 1147 10 0547 60
38. makes it easier for signal analysis and identification based on a given channel Figure 3 3 3 2 4 Sampling Frequency An EEG signal as acquired by the electrodes from the scalp is an analogue quantity that needs to be digitized for further processing on a computerized system The conversion is performed by means of the multichannel analogue to digital converter ADC The sample rate at which this is done must be enough to represent the change in the analog signal 24 The effective bandwidth of the EEG signal is approximately 100 Hz To satisfy the Nyquist criterion equation 3 a minimum frequency of 200 Hz is recommended for sampling EEG signals in order to avoid distortions due to aliasing Mathematically sampling is equivalent to multiplying the input analog signal with a series of Dirac impulses y t x X SE nT 1 n 29 Where x t is the input signal y t the sampled signal t a Dirac impulse and Ts the period Multiplying the signals in the time domain amounts to a convolution in the frequency domain co P xN 3 a nz ae 2 n 0 There are periodicities in both time and frequency in the sampled signal meaning that there should be symmetry or periodicity in the frequency domain to avoid overlaps 25 26 14 27 By the Nyquist criterion the maximum frequency that can be sampled is half the maximum existing in the time domain Thus Ts lt gt 2 fmax Failure to adhere to thi
39. mors or epilepsy and finding treatment for them has led to modern day methods of recognition diagnosis and treatment used in clinical encephalography 15 This correlates CNS functions as well as dysfunctions and diseases with certain patterns ofthe EEG 1 4 2 Electrical Activity in the Brain Brain tissue is made up of three components 1 neurons nerve cells 2 glial cells located between neurons and 3 extracellular space which is made up of mainly fluid containing other macro molecules Glial cells make up the myelin sheath which protects the axon Neurons are responsible for intracellular and intercellular signaling which they do by generating and transmitting electrochemical impulses as a response to stimuli The human brain contains over 25 billion neurons Typically the neuron consists of dendrites which act as receptors for signals from other neurons a cell body and the ending called an axon The axon is attached to other nerve cells through their dendrites or axon to form a synapse The synapse acts as an interface between nerve cells In the human brain each nerve cell is connected to approximately 10 000 other nerve cells mostly through dendritic connections 16 14 17 terminal Dendrite Axon button 1 Soma cell body Nucleus Myelin sheath Figure 1 3 Structure of a neuron The dendrites together with the cell body and part of the initial piece of the axon are the input surfaces of the c
40. n certain stages of sleep High frequency low amplitude rhythms reflect an active brain associated with alertness or dream sleep while low frequency large amplitude rhythms are associated with drowsiness and non dreaming sleep states 5 14 1 4 6 Geography of the EEG Waves Mapping Function to Region Most of the neural activity is distributed near or around the outer surface of the brain the cerebral cortex It is perhaps the most important part of the CNS and the different regions of the cortex are responsible for processing vital functions such as sensation learning voluntary movement speech and perception figure 1 7 Voluntary movement is primarily controlled by the area of the frontal lobe just anterior to the central sulcus the motor cortex The motor cortex controls tasks requiring considerable muscle control e g speech facial expressions and finger movements Sensory information is processed in various parts of the lobes auditory in the superior parts of the temporal lobe the visual cortex being situated at the posterior part of the occipital lobes and the somatic sensory being located just posterior to the central sulcus of the parietal lobe 13 Parietal lobe concemed with the reception and processing of sensory information from Frontal lobe the body having to do with decision making problem solving and planning Temporal lobe having to do che with memory concemed emotion hearing with
41. nd of real and virtual environments from which it receives data for processing 2 2 Installation and Compatibility OpenViBE can be installed on computers running Windows XP or higher for both 32 and 64 Bit However smooth running on 64 Bit architectures is not guaranteed as some drivers have yet to be included It is advisable to use a 32 Bit system for example if one intends to use the acquisition server for an online application To install the Software on Windows one needs to have the NET framework Microsoft Visual Studio 2008 or 2010 on their machines The easiest way would be to download and run the installer win32 install dependencies exe directly OpenViBE also runs on Linux systems such as Ubuntu and Fedora mainly the current versions There is as yet no compatibility with the Mac OS architecture Users having this platform would have to install a parallel Operating system like Windows or the Linux variants which will probably impede efficiency as they can no longer make use of native UNIX capabilities Before one purchases any acquisition device one has to look under the list of supported hardware to verify compatibility of the acquisition server drivers with OpenViBE The alternative would be to write one s own driver However a good number of devices common on the market are supported 2 3 User Modes The platform has been designed for four different types of users The developer and the application develop
42. oft Research Ltd Cambridge 2008 3 F Walsh BBC News Health 16 May 2012 Online Available http www bbc co uk news health 18092653 Accessed 28 June 2012 4 TedGlobal Director A Headset tha reads your brainswaves Film 2010 5 L S rnmo and P Laguna Biolectrical Signal Processing in Cardiac and Neuronal Applications Massachussetts Elsevier Academic Press 2005 6 Inria Rennes 2012 Open Vibe 04 2012 Online Available http openvibe inria fr 7 J R Wolpaw N Birbaumer D J McFarland G Pfurtscheller and T M Vaughan Brain Computer Interfaces for communication and Control Clinical Neurophysiology no 113 pp 767 791 2002 8 E N Bruce Biomedical Signal Processing and Signal Modeling New York John Wiley amp Sons Inc 2001 9 N Birbaumer and L G Cohen Brain Computer Interfaces Communication and Restoration of Movement in Paralysis The Journal of Physiology 2007 10 M Kurz W Almer and F Landolt Brain Computer Interfaces 2006 11 L A Farwell and E Donchin Talking off the top of your head toward a mental prosthesis utilizing event related brain potentials in Electroencephalography and clinical Neurophysiology Ireland Elsevier Scientific Publishers 1988 pp 510 523 12 T D Albis A Predictive Speller for a Brain Computer Interface based on Motor Imagery Milan Artificial Intelligence and Robotics Laboratory of Politecnico de Milano 200
43. ox as depicted in appendix A1 B To achieve optimum recording the electrode impedance has to be brought down to within 0 25 kQ After fitting the cap and the corresponding electrodes before starting measurement the control box is powered and the Z button is pushed to start the impedance check assisted by the actiCap software The following color codes represent the three different classes of impedance levels e Green Impedance lt 25 kQ optimum for acquisition e Yellow Impedance 25 to 60 kQ e Red Impedance gt 60 kQ By applying a special conducting gel to the scalp using a syringe and needle through the crevices on the electrode heads and gently massaging it in the impedance is brought down to a working minimum Appendix A1 This is indicated by transition of LED light from red through yellow to green It takes a while for the gel to act Depending on the number of electrodes involved this process could take anywhere between 15 30 minutes During signal acquisition Impedance check should be turned off but it is good practice to verify levels at the end of every measurement 27 3 2 2 Connecting the Server and the Designer The purpose of OpenViBE is to get data from the acquisition device through the acquisition server and send it to one or more clients for recording or processing 6 In our experiments the client will be the OpenViBE designer responsible for hosting the BCI application pipelines As depicted in
44. quired brain signals into commands to control an application or a device There s a range of software some proprietary like MATLAB and some free and open source FOSS accessible under the GNU General Public License GNU GPL OpenViBE is one such freely accessible software This thesis carries out a functionality and usability test of the platform looking at its portability architecture and communication protocols To investigate the feasibility of reproducing the P300 xDAWN speller BCI presented by OpenViBE users focused on a character on a 6x6 alphanumeric grid which contained a sequence of random flashes of the rows and columns Visual stimulus is presented to a user every time the character they are focusing on is highlighted in a row or column A TMSi analog to digital converter was used together with a 32 channel active electrode cap actiCAP to record user s Electroencephalogram EEG which was then used in an offline session to train the spatial filter algorithm and the classifier to identify the P300 evoked potentials elicited as a users reaction to an external stimulus In an online session the users tried to spell with the application using the power of their brain signal Aspects of evoked potentials EP both auditory AEP and visual VEP are further investigated as a validation of results of the P300 speller List of Abbreviations ADC EEG ECG EKG EMG ECoG CNS DFT FFT LDA EP V A EP SSVEP
45. rain Computer Interfaces The platform consists of a set of software modules that can be easily and efficiently integrated to develop fully functional BCIs for both real and virtual reality applications 22 23 Developed at the French National Institute for Research in Computer Science and Control INRIA OpenViBE is licensed under the GNU Lesser General Public License version 2 or later The platform is updated every 3 months current version 0 14 03 Modularity and Reusability The platform is a set of software modules devoted to the main aspects of signal processing in BCI applications These include acquisition pre processing processing and visualization of cerebral data as well as enabling interaction with VR displays Thanks to the box concept users can easily add new software modules to suit their needs Different Types of Users Tools were designed for different types of users VR developers clinicians BCI researchers neurologists etc can use the platform depending on their programming skills and knowledge in brain processes Portability OpenViBE operates independently of software targets and hardware devices An abstract level of representation allows it to be run with acquisition hardware such as EEG or MEG Connection with Virtual Reality The software can be integrated with high end VR applications throught the Virtual Reality Peripheral Network VRPN server OpenViBE acts as an external peripheral client to any ki
46. recording on NetWOFk ooooo o WWo oo WWW maa 51 Table 6 Unipolar AEP recording on same computer client same as server 51 62 APPENDIX A 1 Acquisition Hardware The pictures show the main Hardware used for signal acquisition in the experiments A and B show the actiCAP electrode system consisting of the following e In A The cap conducting gel and syringe in picture A e In B 1 splitter box 2 32 active electrodes 3 amp 4 ground and reference electrodes 5 actiCAP connector 6 amplifier adapter 7 actiCAP control box In C The 32 channel TMSi analog to digital converter In D Desktop Computer 8 Picture taken from the actiCAP user Manual by Brain Products Picture taken from www blogspot de 63 A 2 Setting up the experiment and dealing with Impedance Picture E a mounted actiCAP with the electrodes connected to the powered control box The red light of the LED indicates high impedance of skin electrode medium see 3 2 1 Picture F applying conducting gel through opening in the electrode head to reduce impedance When the LED light changes from red through yellow to green the operating impedance of the electrode has been attained 10 Picture taken from actiCAP user manual by Brain Products 64
47. s an overflow of cations ve particles from the nerve cell into the extracellular space or an inflow of anions ve particles into the nerve cell figure 1 4 This flow ultimately causes a change in potential along the cell membrane Primary currents generate secondary currents along the cell membrane in the intra and extracellular space The portion of these currents that flow through the extracellular space is directly responsible for the generation of field potentials These field potentials usually with frequencies of up to 100 Hz are called EEGs if they possess a constant signal average If there are slow drifts in the average signals this results into DC potentials which may mask the EEG 16 14 5 1 4 4 Action Potentials Transmission of information from one neuron to another takes place at the synapse the signal initiated in the soma propagates through the axon as a short pulse called the action potential AP Although the initial signal is electrical it is converted in the presynaptic neuron to a chemical signal neurotransmitter which diffuses across the synaptic gap and is subsequently reconverted to an electrical signal in the postsynaptic neuron APs are initiated by many different types of stimuli sensory nerves respond to many types of stimuli such as chemical light electricity pressure touch and stretching Conversely nerves in the CNS are mostly stimulated by chemical activity in the synapses A stimulus m
48. s condition results in a permanently distorted signal However in a multichannel recording setting the sampling frequency too high will lead to significant increase in memory use over time 3 3 Artifacts in the EEG 3 3 1 Definition Artifacts are unwanted components in a measured quantity that can be attributed to biological and technical processes involved in the measurement chain that are not related to the physiological or pathological aspect of the point of measurement and its immediate surroundings EEG signals must always be scanned for these distortions before they are processed for use in applications The artifact in the recorded EEG may be either subject related or technical Subject related artifacts are unwanted physiological signals that may significantly disturb the EEG Technical artifacts include AC power line noise electrode impedance resistance in electrode 30 wires and other hardware specific noise The most common EEG artifact sources can be classified in as follows Subject related e Artifacts related to minor body movements e EMG e ECG components pulse pacemaker e Eye movements EOG e Sweat on the surface to be measured Technical sources e 50 60 Hz power supply e Impedance fluctuation e DC components due to measurement electrodes e Cable movements and broken wires 19 Physiological artifacts can be minimized by introducing extra electrodes for ECG EOG for example in the measurement These c
49. squared amplitude N 1 2 rms 7 xi 9 i 1 For continuous time series T 1 ms x t dt 10 0 33 T 1 rms By x t dt 11 0 A good measurement is one where SNR is high This can be improved upon by signal processing 24 However if the power of the noise is greater than that of the signal the noise will compete directly with the signal on the channel resulting in reduction of the rate of data transfer or complete inability to read the signal 3 4 Evoked Potentials 3 4 1 Definition and modalities Evoked potentials EPs sometimes called Event Related Potentials ERPs are low amplitude 0 1 to 10 uV transient waveforms that appear in the ongoing background EEG as a exogenous response to an external stimulus the user was given They are electrical responses in the brain cortex or the brainstem to various types of sensory stimulation of nervous tissues 5 therefore they tend to have a latency related to the time of stimulation presentation Their amplitude is the sum of a large number of action potentials APs that are time locked to sensory motor or cognitive events 14 Auditory Evoked Potentials AEP are generated in response to an auditory stimulus usually produced by a short sound wave AEPs give an insight into the propagation of neural information by the acoustic nerve from the ear to the cortex and can be used to diagnose ailments and complications to this pathway including hearing loss Vi
50. ss to a list of existing modules in a panel and can drag and drop them in the scenario window Each module appears as rectangular box with inputs at the top and outputs at the bottom The boxes are connectable through their inputs and outputs Double clicking on a box shows its configuration panel An 23 embedded player engine allows the author to test and debug their scenario in real time 2 7 Existing Scenarions and Tutorials The platform comes with a number of ready made scenarios to provide the uninitiated user with an overview of functionality ranging from signal acquisition analysis visualization and rendering through to complete BCls that explore different aspects including integration with virtual reality Relevant tutorials to some of these applications are also provided There is clear in scenario documentation as well as box documentation that can be accessed online More information about these and new releases can be accessed on the OpenViBE website 6 24 3 EEG Data Acquisition and Preprocessing Acquiring a good EEG signal for use in developing a functional BCI requires knowledge of the physiological processes involved and factors that could influence the quality of signal attained sources of errors and how to deal with them Necessary hardware must however be in place Of paramount importance is the ability to correctly interface the various components in order to acquire the signal process it and bu
51. sual Evoked Potentials VEP There are two common methods for eliciting VEPs in one the stimulus is given as a pattern reversal for example alternating black and white squares of a chess board on a monitor display The other stimulus is given as flashing sequences see 4 3 1 The electrical response elicited by visual stimuli can be recorded from the scalp in the occipital region of the brain Figure 1 7 and can serve as an evaluation for visual pathway functionality and diagnosis of ocular and retinal disorders 34 Somatosensory Evoked Potentials SEP Elicited by electrical stimulation of peripheral nerves from the surface of the body this type of EPs can offer valuable information about nerve conduction and the functionality between selected stimulation points from the spinal cord through to the cerebral cortex It is a method of intraoperative monitoring during spinal cord surgery 5 Measured amplitude of the EPs usually depends on the type and strength of the stimulus electrode position on the scalp including mental state of the user i e varying degrees of attention and wakefulness Commonly used in BCls applications are auditory and visual stimulation where the most common evoked potential used is the P300 EPs can also be elicited as a response to performing cognitive functions such those that require attention and memory processes These responses are termed endogenous since they do not involve physical stimuli 3 4 2 No
52. test was carried out using an existing xDAWN P300 speller in OpenVIBE The experiment sought to reproduce the described results of the BCI using our acquisition hardware This BCI is divided into four scenarios 1 Data acquisition xDAWN spatial filter training P300 classifier training 2 in Online use of the speller The experiment involved 5 able bodied subjects users with no prior experience at the start 4 male and 1 female all above 18 years of age All but the female subject repeated the experiment at least once on different days Subjects were seated comfortably in a quiet with minimum illumination at a distance approx 1m away from the monitor Step1 Acquiring data After a background signal check recording was done using the actiCAP and with 16 Ag AgCl unipolar electrodes FP FP2 F3 Fz Fa C3 Cz Ca P7 P3 Pz P4 Ps O1 Oz O2 whose impedance levels were brought to within 0 25 KQ 38 green section 3 2 1 The cap has no Nasion electrode but offers in its place a Reference electrode The signals were amplified and digitized using a grounded TMSi amplifier see Hardware at the OpenViBE default rate of 512 Hz and a sample count of 32 per block delivering 512 32 16 blocks of data per second We used one computer as the server and the other as the client on a network In this session the subject is asked to focus on the character suggested by the computer highlighted by blue on the 6x6 grid figure 4 1 T
53. th own algorithms 54 6 Outlook Unsatisfactory results recorded in the visual speller notwithstanding this thesis has undoubtedly covered aspects of the BCls that will be of help to others researching this topic in the future Basing on the results of the experiment it can be suggested that realizing an own spatial filter and classifier especially with a high level language like MATLAB which in the future will be fully supported by the OpenVIBE platform with the added advantage of the LUA scripting language could be the way to achieving better Brain computer interfaces are a novel way to give disabled people and locked in patients the means to live a comparably normal life again However the rates of data transfer in current applications is still very low about 25 bits minute according to Walpow 7 therefore they are only capable of basic communication and control functions Visual spellers have been documented to achieve only 2 3 words per minute With ever increasing computing power transfer rates of the BCls will also improve For now word prediction algorithms like those used in phone messaging could be integrated to achieve a higher spelling rate on lower processing power 55 56 Bibliography 1 D S Tan Brain Computer Interfaces Applying our Minds to Human Computer Interaction Springer Verlag London Limited London 2010 2 Microsoft Corporation Being Human Human Computer Interaction In The Year 2020 Micros
54. the occurrence of a rare high frequency ton the target stimulus and ignore the normal low frequency ton the non target 2000Hz was used for the target and 1000 Hz for the non target The subject s EEG was recorded with the same electrode placement as in 4 2 1 above and the signals analyzed in MATLAB Figure 4 8 High pass filtered 1 Hz on Cz Pz Red is target and blue non target A normal AEP shows marked difference in signal properties relating to the various structures located in the auditory pathway Figures 4 8 and 4 9 show the P300 wave as a response to the target stimulus and the normal wave train for the non target 47 Figure 4 9 Band pass filtered 1 70 Hz on Cz Pz Red is target blue non target 4 3 Investigating Latencies We use two sinus wave audio signals 200 Hz target and 100 Hz non target in a scenario with two signal generators and stimulation to record and analyze the time dependencies in OpenViBE 2 minutes of the signal was recorded at a sampling rate of 2048 Hz with two variations 1 Unipolar recording 2 electrodes with 1 ground 2 Bipolar 1 bipolar pin with 1 ground A visual inspection of the acquired signal in OpenViBE Figure 4 10 indicates a delay of about 0 2 seconds of the signal on the stimulation Also malformation of the wave suggests a signal leak in the amplifier or more likely a loss of samples during transmission over the TCP IP protocol on the network Another possibil
55. to its digital equivalent by the analog to digital converter ADC For quick and efficient streaming of these data samples to the computer the TMSi Amplifier uses a bidirectional fiber optic cable instead of a wire connection An inbuilt analog low pass filter tuned to a high sampling frequency is used as an anti aliasing filter during signal acquisition Digital low pass filters are then used to set the bandwidth of the acquired signal by eliminating frequencies outside the operating range 7 Sources 1 Computer graphic from www blogspot de 2 Electrode cap graphic from www brain project org 3 AEP graphic from www clipart com 26 3 2 Preparation and Interfacing 3 2 1 Dealing with Impedance EEG has small amplitude 0 1 100 uV because it undergoes attenuation through fluids bone of the skull and skin before it can be picked up by the electrodes on the scalp It is crucial that the recording gear helps to cleanly capture a quantity rather than distort it High impedance may lead to distortions artifacts which can be difficult and costly to filter from the actual signal The electrodes used with the actiCAP are Ag AgCl disks lt 3 mm diameter that are integrated with a tricolor LED system for showing the impedance of the skin electrode interface Embedded in the electrodes is an operational amplifier that acts as an impedance converter Each electrode is connected with flexible leads to the TMSi amplifier through the router and control b
56. tput from the P300 speller application in the online session After a trial the target character chosen by the classifier algorithms is highlighted in green At the end of the spelling session there are two rows at the bottom of the grid 40 1 Target for the characters suggested by the computer and 2 Result for the characters the algorithms identified as input from the user To the left the console output as classifier performance on each of the used electrodes including the mean and standard deviation 4 1 3 Performance The following performance values were observed for the five subjects in both the offline classifier training and the online trial with the speller Table 2 Results of the experiment involving 5 subjects Average Classifier Number of Characters Performance correct out of ten Subject A 66 52 2 10 B 68 26 1 10 C Female 75 34 1 10 D 77 45 0 10 E 67 69 1 10 The results depicted were all obtained with the default settings in the OpenViBE scenarios for the xDAWN speller The documentation of the scenarios suggested that a performance of the classifier above 70 would be necessary for the user to achieve at least 80 score in the online session Subject A s classifier result is below the required result in comparison to Subject D s 77 5 that is several points above However D achieves no score on the speller to A s 2 10 The average score of 1 10 41 on the spell
57. ust be above a threshold level to cause the neuron to fire an AP Very weak stimuli cause a small local electrical disturbance but do not produce a transmitted AP This results in the neuron acting as an ON OFF switch As soon as the stimulus strength goes above the threshold an action potential appears and travels down the nerve 10 Figure 1 5 Changing the membrane potential of a giant squid by closing the Na channels and opening K channels 14 Not all neurons contribute to the excitation of the postsynaptic neuron inhibitory effects can also take place due to a particular chemical structure associated with certain neurons A postsynaptic neuron thus receives signals which are both excitatory and inhibitory and its output depends on how the input signals are summed together 14 5 The intensity of the input signals is modulated by the firing rates of the action potentials High firing rate in the sensory neurons is associated with considerable pain or in motor neurons with powerful muscle contraction 5 1 4 5 Characteristic Frequency Bands of the EEG The EEG has a typical amplitude of about 2 100 uV and a frequency spectrum of 0 1 60 Hz In healthy adults the amplitudes and frequencies found in the EEG change from one state of a human to another such as wakefulness and sleep There 11 are five major brain waves Figure 1 6 distinguished by their different frequency ranges and the power they contain
58. vision and language Figure 1 7 A functional map of the cerebral cortex 15 Generation and Measurement of EEG 1 5 1 Generation in the brain Collective activity of millions of cortical neurons produces an electrical field which is sufficiently strong to be measured on the scalp The electrical field is mainly generated by currents that flow during synaptic excitation of the dendrites the excitatory postsynaptic potentials In the cerebral cortex this current flow generates a magnetic field measurable by Electromyogram EMG and a secondary electrical field over the scalp measurable by EEG systems The amplitude of the EEG signal is related to the degree of synchrony with which the cortical neurons interact synchronous excitation produces a large amplitude signal on the scalp because the signals from individual neurons will add up in a time coherent fashion while asynchronous produces irregular looking EEG with low amplitude waveforms 14 Source http www wpclipart com 14 1 5 2 Electrode positioning Clinical EEG is measured using the 10 20 International standardized system for electrode placement The montage is with electrodes attached to the scalp at locations defined by certain anatomical reference points the numbers 10 and 20 signify relative distances between different electrode locations on the skull surface 5 Electrode placements Figure 1 8 are labeled according to adjacent brain areas F frontal C central
59. y conscious of their surroundings but with little to no mobility in their bodies making it impossible for them to live without external help Advances in the fields of neuroscience brain imaging technologies and computing have provided us with the opportunity to directly interface the brain with a computer 1 thereby creating a communication and control system that bypasses peripheral nerves and muscles to enable interaction through brain activity alone the brain computer interface BCI BCls can enable locked in patients to still interact with their environment by solely harnessing the power of their brain activity Devices including spelling applications for communication input and navigation on a computer system such as moving a cursor or controlling a robotic arm have been tested with success Real time brainwave activity is beginning to be used to control digital movies turn on music and switch lights on and off and to control virtual objects like an avatar 2 3 4 5 Messages are conveyed by spontaneous or evoked EEG activity rather than by muscles contractions In years to come scientists want to reconnect the brain to paralyzed limbs to enable them to function again http en wikipedia org wiki Avatar computing By integrating such applications into the routines of such patients as a method of assisted living their quality of life can be greatly improved through giving them more independence as well as emotion
60. y impairs accuracy There has to exist a compromise between speed and accuracy 52 5 Discussion of Results The aim of this thesis was to use the acquisition hardware available in the BioSig laboratory at Hochschule Heilbronn to reproduce a P300 visual speller on an open source software platform From the many freely available platforms FieldTrip and OpenViBE were singled out because of their modularity approach providing ready to use tools and algorithms FieldTrip is a Matlab based Toolbox for EEG EMG analysis that includes no graphical interface therefore it relies on the Matlab console and still requires a good understanding of this programming language OpenViBE on the other hand has an integrated graphical interface and uses a more simplified graphical language to enable the user to prepare pipelines for signal processing visualization and applications like BCls without having to write any line of code see chapter 2 Work started with an analysis of both platforms at the end of which OpenViBE was chosen This was followed by a literature review to get acquainted with the many aspects of brain computer interfaces including signal processing neuroscience and the EEG human computer interaction and assisted living for the disabled The approach taken in this thesis was not do develop an own visual speller but rather recreate an off the shelf application from OpenViBE with the kind of hardware we had in the laboratory On account of t

Dokument_1.

Contents

Download Pdf Manuals

Related Search

Related Contents