assignment
Contents
1. e Select Insert gt Querying Example Row then click OK e Double click in the Image Name cell and type a name for the image e g query 1 and click OK e Click the Browse button and select the image you want to associate with this query row and click Open Once the new query rows are added view the results for each query which happens automatically if the network has already been trained Observe the results Q12 How accurate was the detector at identifying these additional bills If there were misidentifications explain what you think the reasons are Include the new images you used in your report 192. Press return again and the Edit Grid dialog box will appear This time the Example row will already contain a name and type You can set the cell value the Input Output name and the type can be set to I for input O for output X for exclude or S for serial The mode can be set to Real Integer Bool or Text Any type of Input Output column can be inserted into the Grid using the functions on the Insert menu 2 A 4 Creating more Example rows Move the marker one cell down by pressing the down arrow key Now press the return key and a prompt will appear that says Create new Example row Answer Yes Press return again and the Edit Grid dialog box will appear This time the Input Output column will already contain a name and type You can set the cell value Example name and the type can be set to T for training V for validating Q for querying or X for exclude Any type of Example row can be inserted into the Grid using the functions on the Insert menu 2 4 5 Copying Example rows and Input Output columns Double click on the name to select the whole row or column Cut will remove the selected row or column and place it on the clipboard Copy will place a copy of the selected row or column on the clipboard Paste will insert a copy of the clipboard before the currently selected row or column If the clipboard contains a row then Paste will insert the row into the Grid If the clipboard contains a column then Pas
3. e E Ir 39 9 Runners Distance landicep class Beakeesk d m p O O T Hj ha p ese D Eee eme a i2 E Exe h o ale mnse eme E me o eme me aee erue Ho False me eme tme ferae mee tme ferae frare W i i a ei m m m ey Cn ae m i i io i i true t t false Tt t co false false t t t X t t t t rue rue Pie rue e false rue false false false true false lle je e lle rue rue rue ru EU ru ru t false false true false t false tru ru Iu false t false false true false true false false false true true true true alse 1 rene reno Foal on t t t t t t t co false na false i t t e false Fu ru Fu ru Fu Fu ha e mle co ele a Eo E 1 E 1 a e Hs e co e e ix ta is Ce ue t e t t false t t e rue rue rue rue rue rue rue rue rue rue rue rue rue s i iEn un 14 Part 4 Majority Learning in a Neural Net In this part of the lab project do the following and include results in your lab report l Create a new JustNN document Open the RacesEmpty tvq file after downloading it from the class Schedule
4. be selected either using the arrow keys or the mouse A single click will select the cell and a double click will start the Edit Grid dialog A double click on the Example name cell will select the whole row and a double click on the Input Output name cell will select the whole column The row or the column can be deselected by pressing the Esc key M how Distance Handicap Class Stake gt 5k Odds gt 2 Win uj 2 sie fae D fame tase feru Q Rownames faj a p fase eme eise eme 5 S a mue ha franse heme fere ic io ho me b me franse tase n 6 b kas a fase tose eme comics t5 aie fase h fae raise rarse L5 us p o eme e eme me fase uj isa J me h me eue ale TE uz fs 13 fase a else franse me us a fme A me ue me nap ja l fas eme fale eme 15 us he me b me ue me un B ase h me eue hme ns us o p me b aie eue tase o f bp Eae a Eae froise me pij he ae b aie eue eme 22 e o fale 6e ale false rale 2s bo ie bas k hme feme me ta e b me b me jue ale 251 ts he S dme b Ease tue ale i ja fas 6e Ease fale tarse Us fe e eme s Ease fale me s e rale b rale tase
5. eme In the picture above the first row represents a single training example to be fed to the network It has the first Input Feature Runners which has the value 11 the second Input Feature Distance which has the value 7 etc Note the last value in the row Win is color coded in red It is an Output That is this is what the network is supposed to return when the example is fed into it In other words it is the Ideal Output for this example 2 A 1 Creating a New Grid A new Grid is created by pressing the toolbar button or using the File gt New menu command The new Grid will be empty except for a horizontal line a vertical line and an underline marker that shows the current position in the Grid New Grid rows and columns are created at the current position 2 4 2 Creating the first Example row and Input Output column Press return and a prompt will appear that says Create new Example row Answer Yes Another prompt will appear that says Create new Input Output column Answer Yes again You will now see that the Grid has one cell containing a row name containing T 0 and a column name containing I 0 The indicates that the cell has no value the T 0 indicates that it is a Training Example in row 0 and the I 0 indicates that is is an Input in column 0 Press return again and an Edit Grid dialog box will appear that allows you to enter the cell value Using the same dialog you can change the Input
6. f f j l A l j j A j i i 1 J l i l i l Import h Networkview i Edit N e Change controls Stop learning Add query Save o f id vi Change query value iia Importance view TT Support Information Forget learnin rni New network Start learning 7 Learning progress Here is a description of the toolbar buttons you will use New L3 Create a new document with a blank neural network grid Open cm Open an existing grid and neural network Import ME gt Import a TXT CSV XLS BMP or binary file into the neural network grid save Save the active neural network document This will contain both the training data and the network you build from it o do Cut copy and paste in the Grid The row or column is selected by double clicking the row or column name Grid view View and edit the Grid The Grid is like an Excel spreadsheet Each row is an example or query for the Network see Grid View below o Network view View the neural Network that 1s created from the data in the Grid Importance view View the importance of the inputs e g a list of the network s inputs in descending order of weight Learning progress View the learning progress graphs The graphs show the minimum maximum and average error rates as the network is trained on more and more cycles iterations over the ba
7. 120 second durations 18 Q9 What differences did you see with learning performed for different durations Are there benefits or tradeoffs in longer periods of training Experiment 5 13 This sort of neural network could be used in a commercial vending machine bill change machine or bill counterfeit detection machine Q10 How would you incorporate such a neural network into one of these machines Describe in general non technical terms how the neural network could be used to identify denominations of bills when it would reject bills and how accurate it could be Q11 Suppose we wanted our network to reject non 1 5 and 10 bills from being processed maybe our vending machine can t store enough change for these What change would you make to the network e g entries in the Grid to accomplish this Experiment 6 14 Test your preferred neural network bill detector on three additional bills To do this take photos using a phone camera or download images from the Internet You will likely need to crop and resize the images you gather so they are each 128x54 pixels in size There are numerous online tools that can help with this such as Pic Resize or others found via a Google search for online image crop and resize Add these new pictures into your neural network by first copying them into the same directory lab5bills and then adding them one by one as new query rows To add additional query rows do the following
8. Dialog box and allowing JustNN to determine the optimum number of nodes and connections When you create a new network you ll see a New Network dialog box followed by a Controls dialog box to set up things like the network s learning rate alpha validation process and when to automatically stop training so we don t get caught in infinite learning when convergence never happens Let s take a look at the New Network box Cancel button Jj a N e asa unes t Change every cycles or 3 seconds Cancel Input layer input layer Hidden layers r Output layer Created with Grow layer number 1 M a EMI Created with b nodes 1 node from minimum nodes 3 E 2 connected to Y connected to grid Inputs In maximum nodes EN 5 B Hidden lavers Output layer f Growth rate network is found Input layer columns in the grid Hidden layers nodes Always make sure Grow Layer Number 1 is checked P Output layer columns in the grid OK button Press to accept all the settings and close the dialog Cancel button Press to reject all the settings and close the dialog 10 A network is produced when the cycles or seconds elapses until the optimum The number of nodes in the input layer is determined by the number of input The hidden layers are grown from the minimum to the maximum number of The number of nodes in the output layer is determined by the number of output Now let s take a lo
9. Lab 5 Feed Forward Artificial Neural Networks Evolution and Learning in Computational and Robotic Agents MSE 2400 Dr Tom Way Introduction In this lab you will build several feed forward neural networks and test how well they work that is develop learning agent models on various training input domains You will also gain some experience in building your own training set for neural networks Finally you will gain some first hand understanding of the limits of the training of a neural network and you will observe how the network generalizes The software we will use for this lab 1s for Windows computers and the work to be done is ample enough that we will work in teams of 1 2 or 3 students each If you have a Mac and do not have a way to run Windows programs on your Mac then make sure to team up with somebody who has a Windows laptop You can work in a team of up to three people Note that significant exploration of the software how it works what it can do and how it can be used likely will be required Worth e 100 points Due e Your lab report will be due the first regular lecture class meeting after your team completes the lab e This lab contains a lot of steps so it may take 2 lab sessions to complete In addition you may want to spend time outside of class to work on the lab exercises and writing of the lab report What to hand in e A typewritten lab report containing answers to all of the questions in Part 4
10. Output column name mode and type The dialog can also be used to change the Example row name and type Here is an example of the Edit Grid Dialog Box and an explanation of what its fields mean In the lab projects you will be using Input Output column types bottom of box that are either Bool Boolean or Image This box appears every time you edit a cell in the Grid Q Edit Grid Min 0 Max 4 scaled 0 1 0 5 Example rows N Example row Training Validating Querying Exclude 3 Input Output column Input Output column ES iini C Real Integer Bool Text Image Input C Output C Exclude OK Value Value 2 0000 Min 0 Max 4 scaled 0 1 2 0 5 The value in the selected cell with minimum maximum and scaled value o Example row Example row Hea Training Validating C Queming C Exclude Enter the example row name and set the type of row 9 Input Output column M Input D utput ealumn Spades Real Integer Bool C Tex Image Input C Output C Exclude Enter the column name Set the column type and mode Input or Output Click to complete the edit and close the dialog OK 2 4 3 Creating more Input Output columns Move the marker one cell to the right by pressing the right arrow or tab key Now press the return key and a prompt will appear that says Create new Input Output column Answer Yes
11. There are always as many input nodes as there are features inputs for an example in the Grid o Hidden node Hidden nodes are fully connected to input nodes output nodes or other layers of hidden nodes Output node 3 Kilograms Output nodes are connected to the output columns in the grid o Connection weights The input layer is fully connected to the first hidden layer Each connection has a weight that is updated while the network is learning Hidden layers are fully connected to the next hidden layer or the output layer Red connections represent weights with negative values and green connections represent weights with positive values The thinner or thicker the connection line the smaller or larger the absolute value of the weight is Dashed lines represent weights extremely close to 0 2 B 1 How to create a new neural network A new neural network can be created from the Grid by pressing the New Network toolbar button or selecting Action New Network This will produce the New Network dialog This dialog allows the neural network configuration to be specified The dialog will already contain the necessary information to generate a neural network that will be capable of learning the information in the Grid However the generated network may take a long time to learn and it may give poor results when tested A better neural network can be generated by checking Grow Hidden Layer 1 in the Create Netwrok
12. a much longer training period Why do you think learning was different Include screen grabs of anything that helps to explain this difference such as the network or learning curve e Third grow the network a third time 39 with all three hidden layers 7 4 and 4 and have it stop after 60 0 seconds Observe the results Q6 How did learning differ with this approach using three hidden layers and an even longer training period Why do you think learning was different Include screen grabs of anything that helps to explain this difference such as the network or learning curve Experiment 3 11 Try growing a new network as you did for Experiment 2 one more time This time you may select any number of hidden layers the maximum sizes of the layers and any other learning parameters you would like to experiment with Explore in any way you like making sure to note what approach you used Before starting make a note of what you think the result will be Q7 Describe your hypothesis for the approach you are going to use What do you think will happen Q amp S Explain the approach you used how well it learned compared to other approaches you tried earlier and why you think it differed As usual include any screen grabs that help to illustrate your results Experiment 4 12 Determine the effect 1f any of different amounts of training Select one of the above network configurations and repeat learning Stop after 30 60 and
13. alidating stops Accept what it says Fixed period stops Fised periad stops Stop after 20 0000 seconds Stop on 1 M cycles Select the Stop On option for our labs and specify 200 or 1000 cycles Reasonable values are 200 for the projects in Part 4 of the Lab and 1000 for Part 5 the images of bills OK button Press to accept all the settings and close the dialog Cancel button Press to reject all the settings and close the dialog 12 Part 3 JustNN Exercises You will now run three exercise projects on JustNN to familiarize yourself with the interface learn how to create training example sets on the Grid learn how to edit the Grid learn how to create networks and learn how to query them These can be started by clicking the Getting Started button on the Tip of the Day or using the menu command Help Getting Started The network files you ll use to work with these exercises are already in the JustNN folder the Installer created for you If you find you have to save one of these network files after you make changes to them please save them as a different file otherwise you won t be able to go back to the exercise 1f you want to check things out again YOU DO NOT HAVE TO HAND IN ANYTHING FROM THESE EXERCISES FOR THE LAB REPORT 1 XOR In the first exercise you will open train and query a simple neural network that simulates XOR exclusive or XOR is a logical operator that results in th
14. and Part 5 of this lab instruction document e Make sure the names of all team members are on the report e You can hand in the report via email or on paper Lab Steps Part 1 Software Setup 1 Goto JustNN com website and download the JustNN software package 2 Install it on the laptop your team will be using for the lab by clicking on the installer you downloaded Accept all the default prompts and let it install in the folders the installer asks to install 1n 3 After installation find the desktop icon for JustNN and double click on it to start the program If it starts successfully you can exit it and move on to Part 2 Part 2 Software Orientation JustNN User Manual Excerpts If you are comfortable with the JustNN software you can skip this part otherwise review this part to learn more about JustNN This part will be easier to read an understand in the online version of this handout as you can see the colors of the icons and menu options In the next part of this lab you will run through three sample exercises with JustNN to get some familiarity with the software When you start the JustNN system you ll get a window with the following toolbar just below the standard Windows Pulldown Menus at the top of the window A cT Jo 2 i ul k x XJ IM J f f f f f l f j J j l T A A j j J j j j 1 j f
15. ckpropagation algorithm If you provide validating examples examples that are not used for training just testing the graph will also show average validating error vs learning cycles New network Opens the New network dialog to create a neural network from the Grid Start learning Starts the learning process Pressing this will make the network iterate over the Backpropagation algorithm we discussed in class and update the weights in the network to make it respond correctly for all the training examples in the Grid Stop learning Stop the learning process Forget learning Forget learning The network s weights are all reset to random number values between 0 5 and 40 5 o Add query 1 Q Adds a querying row to the Grid Change query value E V Sr Increases decreases maximises or minimizes the query value 2 A Grid View Now let s look at the two main views provided by JustNN that you ll use in this project First the Grid see next page You can view it by pressing the button in the toolbar The Grid view shows all the Examples arranged in rows and all the Input Outputs arranged in columns Input is the same as a feature The first column contains the Example types and names The first row contains the Input Output types and names Everything on the Grid can be edited by moving to the cell containing the value and then pressing the enter key to start the Edit Grid dialog The cell can
16. did the network do well at learning to recognize What sort of problems with learning did you observe such as bills that were misrecognized as an incorrect value etc Briefly explain why you think this happened 8 Once complete with Experiment 1 close the network without saving changes and exit JustNN Do this at the conclusion of each experiment to ensure that each one is a clean experiment and the results are not affected by the previous experiment Experiment 2 9 Two possible reasons the network might have learning problems in the first experiment are that there were no hidden layers in the network and training was done for a very short time 10 Repeat the first experiment three more times as follows e First open the BillDetector tvq file in JustNN e Grow the network as before 3 only this time include one hidden layer consisting of a maximum of 7 nodes and learning for 1000 cycles Observe the results 17 O4 How did learning differ with this slightly different approach that used a hidden layer Why do you think learning was different Include screen grabs of anything that helps to explain this difference such as the network or learning curve e Second grow the network again 32 with one hidden layer max 7 nodes and have it Stop after after 20 0 seconds Be sure to uncheck the Stop on checkbox Observe the results Q5 How did learning differ with this approach that used a hidden layer with
17. e output being true if one of the inputs but not both is true If both inputs are true the output 1s false 2 Color Circle In this exercise you will be guided through a series of steps to make a neural network that learns which secondary color is produced when any two of the three primary colors are mixed together You will open a partially completed grid file You will edit the grid and then create train and query the neural network 3 Races In the Races exercise you will start with an empty grid You will then import the results of 370 horse races that our horse ran in Then you will create train and validate a neural network When it is completed the neural network will be used to predict the results of other horse races The columns of data in this example are e Runners how many horses ran in this particular race e Distance how long was this race in furlongs e Handicap do better horses carry more weight in order to even out the competition e Class higher number means a more competitive or prestigious race anything from 1 to 6 e Stake gt 5k was the entry fee to run in the race greater than 5 000 meaning it is more prestigious e Odds 2 were the odds for our horse greater than 2 1 odds e Win did our horse win the race 13 Example Data from Races tvq This data can be used for reference when working on Part 4 gt Justin Races tvg SE File Edit View Zoom Defaults Insert Action Query Tidy Window Help D x ag
18. le and the folder you extract someplace convenient for easy access Experiment 1 2 From JustNN open the BillDetector tvq file It contains references to the images in the image files in the labSbills folder The images are all 128 pixels dots across and 54 pixels down Double click on one of the image names e g one 1 check the Show all images in grid box then click OK Troubleshooting If the images do not appear in the grid you need to correct this Double click on the first image name for which no image appears then click the Image File button Note the file name in the box then click the Browse button locate the file of the same name and Open it The reset of the images should now appear in the grid 3 Look at the Grid It will have 18 rows 12 examples 6 validating Each example and validating row has a single input an image and three Boolean attributes first column 1 second column 5 third 10 that are set true or false depending on which type of bill 1s on that row Note there are also 7 query rows at the bottom of the grid with fall values for all three attributes 4 Explore the grid and network gt views to see how things are starting out Ol Describe the initial state of the grid and neural network and include a screen grab of the network 5 Make a network from this data 2 You ll see that even though there is one input for examples the network will ha
19. mn example rows that have different values from the earlier rows In two of the rows leave 2 of the input values attributes empty and for the third leave 4 of the input values empty Use them as Query examples in your grid See what answers the network gives for these examples List these examples in your write up and tell what answers the network gave Do the answers make sense given the values you did fill 1n If not why not Are these examples in some way too different from the training examples Make another 14 examples don t use the three you made in 8 set them as Validating but make sure there are seven where the output should be false and seven where the output should be true Put these in your report Do you get a better learning curve when you have 24 training examples How do your 3 queries work out now Much better or only marginally better than before 15 Part 5 Image Recognition In this part of the lab you will design a neural network to recognize images of 1 5 and 10 and possibly other bills This technology you will develop could be the basis for a bill reader in a vending machine Note There is no UNDO feature in JustNN so be careful For your report including copious screen grabs to document what you do and your responses to the questions indicated with a Q below 1 Save and extract the lab5bills zip file from the class Schedule page which creates a folder called labSbills Save the fi
20. ok at the Controls Dialog Box 1 Learning Controls Learning Learning rate oE E Decay v Optimize Momentum o s Decay W Optimize Validating 2 Cycles before first validating cycle 100 Validating Cycles per validating cycle 100 Select fo examples at random fram the Training examples 18 Slow learning Delay learning cycles by fo millizecs 3 Slow learning Learning Learning 4 Target errar staps x Target error stops Stop when Average error is below o 00333 orf stop when All errors are below 5 Validating stops Stop when 100 z of the validating examples are Within 1a x of desired outputs or Corect after rounding Validating staps G ix amp d period stops Fixed period stops Stop after 20 0000 seconds Stop on 100 cycles OF Cancel T OK button Cancel button Learning rate 0 6 Decay v Optimize Momentum 0 8 Decay W Optimize Always make sure for our labs that you check the Optimize box E Validating Validating Cycles before first validating cycle 100 Cycles per validating cycle 100 Select o examples at random from the Training examples 19 Typically if you have no Validating examples in the Grid set this to select a number that is no more than 25 of the training examples Slow learning Ignore this for our labs Target error stops Accept what it already shows for our labs V
21. page Starting with the one empty training row define 10 instances of 7 column training examples with values similar to those in the Races exercise data To create a new instance click Insert 2 Training Example Row Fill in any values you like for each of the columns or attributes Repeat that 10 times to create the 10 new rows Select 2 out of the 10 examples you just added for validation For each of those two rows double click on the start of the row hit Enter then in the dialog that pops up change the row type from Training to Validating then click OK Save it Click the Grow New Network button which will lead you through several steps of the backpropagation learning process On the Controls dialog set the Fixed period steps to Stop on 1000 cycles Include a screen shot of your trained network in the lab report Did the network make any inputs weigh much more than others In your report describe whether you see any relationship between the stronger weights and the 1s and Os true and false values of the examples you trained with Include a screen shot of the learning curve for the network in the lab report In your report explain why your network s curve looks the way it does It will probably have poor performance Look at your examples and report whether the choice of examples you created and the specific validation examples had an impact on the results and why Create three more of the 7 colu
22. te will insert a column into the Grid The invisible Grid data limits and defaulted values will be regenerated after a Paste column thus any neural network that has already been generated from the Grid will be invalidated 2 4 6 Query Example Rows As stated in 3 A 4 you can make a Grid row represent a Query when you create it That is it will be an example that 1s fed into a trained network You won t have to edit the output cell s of this example They will be automatically set by the action of the trained neural network Be sure you don t get confused between Query example rows and Training Example rows Similarly Validating example rows represent examples that you do not want to train the network on only test the network once it is trained on all the training examples 2 B Network View Now let s take a look at the view of the network that is created from the data in the Grid Open the Sample Diggers tvg You view it by pressing the x button in the toolbar Diggers tvq Net Input am 0 661717 Negative Weight M JustNN mmmg 0 340354 Positive Weight Node 8 details ias oc 1 245302 md Error Em 0 070807 Insignificant Weight Left Click Node for Details D Diggers Input node Output node Connection weights The Network view shows how the nodes in a JustNN neural network are interconnected Input node O Diggers This represents the input feature value being read in from the Grid
23. ve THREE input nodes This is because for pictures JustNN looks at all the pixels color dots in the picture and calculates for us 3 values PC pair code EC edge code and BC block code e PC is based on the sum of differences of the first 1000 pairs of pixels in the image 16 e EC is based on the locations of the sharpest edges on the top bottom left and right sides of the image and e BC 1s based on the location of the largest block of same color pixels in the image 6 Follow the prompts to train the net First in the New Network dialog make sure all three hidden layers are unchecked 1 e Grow layer number 1 2 and 3 are unchecked and click OK Next click Yes to set the controls In the Controls dialog check the box in the lower right area that says Stop on and have it stop on 1000 cycles Click OK Yes and OK to have the network start learning 7 Once learning is complete which will happen quickly observe the query rows In each query row double click on each true and false to see how confident the network is about each answer on a scale from 0 0 to 1 0 False is 0 0 True is 1 0 and anything else in between falls somewhere on the range from false to true Q2 Which input features in this neural network PC EC and BC had stronger weights Include a screen grab of the network and of the learning curve Q3 How was the learning performance What
Download Pdf Manuals
Related Search