Home

digital photo frame implementation on fpga system

1. 39 Nios I EDS 9 1 k i Y el el or further information check http www uclinux org xecution Finished Exiting Sash command shell version 1 1 1 b 1 2 new full speed USB device using isp1362 hed and address 2 New USB device found idVendor 8951 idProduct 1642 New USB device strings Mfr 1 Product 2 SerialNumber 3 Product DT 181 G2 Manufacturer Kingston SerialNumber B81CCHEC2F39F86145CB1BCH configuration 1 chosen from 1 choice SI emulation for USB Mass Storage devices 8 Direct Access Kingston DT 181 G2 1 66 PQ ANSI 2 sda 7818184 512 byte hardware sectors 4 88 GB 3 72 GiB gt sdal Write Protect is off sda Assuming drive cache write through sda Assuming drive cache write through u 88 Dn a La LLO va Nay LL r i San Sat SA Aa see NNNNNUN S93599 S1 GO DOS e e sda Attached SCSI removable disk iger jpg iming png Scad presentation_1 pptx mnt gt Figure 4 5 mount the USB storage content However the picture was failing to display on the VGA monitor screen due to some unknown reason and unknown bug in the open source picture viewer Figure 4 6 show the pictures are failing to open in uClinux OS Figure 4 7 show the nano 40 X environment will closed unexpectedly when the NxView was executed to load the picture from the USB pendrive Nios II EDS 9 1 IECAD presetation System Volume Information
2. Figure 2 10 FPGA Vs ASICs design cycle 2 9 FPGA Design Advantages 1 Faster time to market FPGA design can develop in shorter time as it no need layout masks or other manufacturing steps 2 Lower Non Recurring Expenses NRE FPGA tools are cheap or even free example like Quartus II free web edition software 3 Simpler design cycle Software able handles much of the routing placement and timing 4 More predictable project cycle The FPGA design flow eliminates potential re spins wafer capacities 5 Reusability Designs of system can easily change using Hardware Description Language HDL code if any design faults was detected Then program it to the FPGA and test it again 6 Field Reprogram ability The FPGA can be reprogrammed in a short period of time whereas an ASIC will take expensive cost and more than 4 6 weeks to make the same changes CHAPTER 3 METHODOLOGY 3 1 Introduction This chapter will discuss the methods tools and software to develop the digital photo frame prototype Two different development platform was used to implement the digital photo frame that is Altera DE2 development board and Arrow SoCKit Evaluation Board A VGA monitor was used to display the photos from the development board There are three layers in this prototype digital photo frame system and the development environment is as follows 1 Bottom layer FPGA system hardware e g Altera DE2 development boar
3. hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps 10 hps qspi_inst_IOO hps 10 hps 10 gspi inst 100 hps io hps io gspi inst IOI hps io hps gspi inst IO1 hps io hps io gspi inst 102 hps io hps io gspi inst IO2 hps io hps gspi inst 103 hps io hps io qspi_inst_IO3 hps 10 hps 10 qspi_inst_SSO hps 10 hps 10 gspi inst SSO io 10 hps hps 10 hps hps 10 gspi_inst_CLK gspi_inst_ CLK hps 10 hps sdio_inst_CMD hps 10 hps 10 sdio_inst_CMD 10 10 sdio_inst DO hps hps 10 hps hps 10 sdio_inst D0 hps 10 hps 10 sdio_inst DI hps 10 hps 10 sdio_inst D1 hps 10 hps 10 sdio_inst hps 10 hps 10 sdio_inst hps 10 hps 10 sdio_inst CLK CLK D2 hps 10 hps 10 sdio_inst D2 hps 10 hps 10 sdio_inst D3 hps 10 hps 10 sdio_inst D3 hps 10 hps 10 usbl_inst DO hps 10 hps 10 usbl_inst D0 hps 10 hps 10 usbl_inst DI hps 10 hps 10 usbl_inst D1 hps 10 hps
4. memory mem dgs memory mem dgs memory mem dgs n memory mem dgs n memory mem odt memory mem odt memory mem dm memory mem dm memory oct rzgin memory oct rzgin hps 0 hps io hps io emacl inst TX CLK hps 0 hps io hps io emacl inst TX CLK hps 0 hps io hps io emacl inst TXDO hps 0 hps io hps io emacl inst TXDO hps 0 hps io hps io emacl inst TXDI hps 0 hps io hps io emacl inst TXDI1 hps 0 hps io hps io emacl inst TXD2 hps 0 hps io hps io emacl inst TXD2 hps 0 hps io hps io emacl inst TXD3 hps 0 hps io hps io emacl inst TXD3 hps 0 hps io hps io emacl inst RXDO hps 0 hps io hps io emacl inst RXD0 hps 0 hps io hps io emacl inst MDIO hps 0 hps io hps io emacl inst MDIO hps 0 hps io hps io emacl inst MDC hps 0 hps io hps io emacl inst MDC hps 0 hps io hps io emacl inst RX CTL hps 0 hps io hps io emacl inst RX CTL hps 0 hps io hps io emacl inst TX CTL hps 0 hps io hps io emacl inst TX CTL hps 0 hps io hps io emacl inst RX CLK hps 0 hps io hps io emacl inst RX CLK hps 0 hps io hps io emacl inst RXDI hps 0 hps io hps io emacl inst RXD1 hps 0 hps io hps io emacl inst RXD2 hps 0 hps io hps io emacl inst RXD2 hps 0 hps io hps io emacl inst RXD3 0 hps hps io hps io emacl inst RXD3 hps hps hps hps hps hps hps hps hps hps hps hps hps hps
5. 10 usbl_inst D2 10 10 usbl_inst D2 hps hps 10 hps hps 10 usbl_inst D3 hps 10 hps 10 usbl_inst D3 hps 10 hps 10 usbl_inst D4 10 10 usbl_inst D4 hps hps 10 hps hps 10 usbl_inst D5 hps 10 hps 10 usbl_inst D5 hps 10 hps 10 usbl_inst D6 oO OO OO os os os S S Oo S S S S S S S oo S S S S O oo Oo oo S oo oo 0 S 00 0 5 hps 10 hps 10 usbl_inst D6 61 hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps hps 62 usbl_inst_D7 spim0_inst_CLK spim0_inst_MOSI spim0_inst_MISO spim0_inst_SS0 spiml_inst_ CLK spiml_inst_MOSI spiml_inst_MISO spiml_inst_SS0 gpio_inst_GPIO00 gpio_inst_GPIO00 gpio_inst_GPIO09 0_hps_io_hps_io O_hps_io_hps_io_usbl_inst_D7 0 hps io hps io usbl inst CLK 0 hps io hps io usbl inst CLK 0 hps io hps io usbl inst STP 0 hps io hps io usbl inst STP 0 hps io hps io usbl inst DIR 0 hps io hps io usbl inst DIR 0 hps io hps io usbl inst NXT 0 hps io hps io usbl inst NXT 0 hps
6. Be reset sink Reset Input cock aan avalon_slave_0 Avalon Memory Mapped Slave clock A 0x0003 0000 jox0003_000 pams E alt_vip_vir_vga Frame Reader Boer ese clock reset Clock Input PI stream Configuration amp Programming en er Sr Eas E clock master Clock input cik SR ROCES ja clock master_reset Reset Input clock master akas Gilis callan avalon slave Avalon Memory Mapped Slave clock reset a 0x0000_0100 0x0000_017 Herin Components avalon_streaming_sou Avalon Streaming Source clock reset E Microcontroler Peripherals avalon_master Avalon Memory Mapped Master clock_master E alt vip_itc_0 Clocked Video Output Meer s_ck_rst Clock input pil stream BL is_ck_rst_reset Reset Input sok rat A iGayahpercormect din Avalon Streaming Sink Iis ck rst Ii vermicaton clocked_video Conduit alt_vip_ite_0_clocked_vid Window Bridge a a Palas gt refcik Clock Input cik 0 sai Reset Input etek H outcko Clock Output pl_stream_o out Clock Output PI_stream_o oed Conduit E El clock_bridge_65 Clock Bridge in_ck Clock input pil stream out ck Clock Output clock bridge 65 out cik clock bridoe EA 3 Warnings Path gt A The module properties SIMULATION_MODEL_IN_VERILOG and SIMULATION_MODEL_IN_VHDL can not both be set when using the SIMULATION file property System alt_vip_vfr_voa A pll_stream locked must be exported or
7. 1 2 Problem Statement The System on Chip SoC technology on FPGA is still new and not many people understand it FPGA implementation of SoC enables the integration of Intellectual Property IP from different sources to rapidly build an embedded system FPGAs are reconfigurable meaning that products can be developed and marketed in a short time Therefore understanding the SoC knowledge is important in order for us to implement any embedded product in the future To gain an understanding of the SoC design process the digital picture frame was used as a case study Most of the commercial digital photo frame available on the market today has a single function By exploring the SoC design options the digital picture frame offers an expandable platform which can support new features such as face detection motion detection and video playback 1 3 Research Objective There are 3 objectives of this project 1 To learn and understand SoC technology on FPGA Ze Prototype a digital photo frame on FPGA for hardware verification and early software development 3 To implement a user friendly digital photo frame that can used by everyone 1 4 Scopes of Project In order to achieve the objective of the project there are several scopes had been outlined l The Altera DE2 Development Board and Arrow SoCKit Evaluation Board were used 2 The application was developed using C C and Verilog Hardware Description Language 3
8. 4 16 4 17 4 18 FPGA Configuration Mode Switch Putty Interface Reboot button Qsys partl Qsys part2 Full compilation zImage file uClinux Operating System USB mass storage detection mount the USB storage content Fail to open picture nano x closed unexpectedly Full compilation Block Diagram Preloader and U Boot Kernel compilation Kernel compiles successfully Original picture Zoom in picture Zoom out picture Rotated picture 128x64 dots LCD Logo of digital photo frame Xli 33 33 34 35 35 36 37 38 39 39 40 40 41 41 42 42 43 44 44 45 45 46 46 FPGA SoC IP JPEG HDL ASIC CLB NRE IDE SOPC OS USB OTG HD HPS VGA LCD xiii LIST OF ABBREVIATIONS Field Programmable Gate Array System On a Chip Intellectual Property Joint Photographic Experts Group Hardware Description Language Application Specific Integrated Circuits Configurable Logic Blocks Non Recurring Expenses Integrate Development Environment System on a Programmable Chip Operating Systems Universal Serial Bus On The Go High Definition Hard Processor System Video Graphics Array Liquid Crystal Display XIV LIST OF APPENDICES APPENDIX TITLE PAGE A Top level design entity verilog code 51 CHAPTER 1 INTRODUCTION 1 1 Background Ever since photographs were invented the picture frame was used to display photos prominently However the conventional photo frame has some limitations Firstly the space
9. Linux OS into SoCKit board 33 BG sm R296 CODEC_SEL 4 Figure 3 17 FPGA Configuration Mode Switch USB 2 0 OTG cable is used to connect the keyboard to the SoCKit board Connect a VGA monitor to the VGA connector of the SoCKit board Putty terminal was used to control the Linux on the SoCKit board Putty also able to show the Linux booting process and notify user which part was stuck when booting failed Figure 3 18 show the Putty interface that can let user to control the Linux OS on the SoCKit board EP COM6 PuTTY Figure 3 18 Putty Interface 34 WARM_RST button on the lower left corner of the socket board KEY 6 was used to rebooting the Linux once the button was being pressed Figure 3 19 red circle part show the WARM_RST button which acts as a reboot button Figure 3 19 Reboot button 3 4 3 Hardware system Integration The hardware design should at least consist of the main components needed for the digital photo frame application The components include a VGA controller and necessary memory to hold the program The MicroSD card memory was used to hold the program and embedded operating system The hardware design and kernel image are needed for the application Qsys Integration development tool in Quartus II 13 0sp1 was used to include an ARM processor and other components into a stable system Figure 3 20 and Figure 3 21 show the component included to build a system using Qsys Integration
10. University 2007 Chang Y T Huang C M Wu C M Chen C Y Lin Y S Kuo C T Liu T C and Wey C L A modularized FPGA based embedded system development platform JECON 2010 36th Annual Conference on IEEE Industrial Electronics Society IEEE 2010 1697 1702 Farooq U Marrakchi Z and Mehrez H FPGA Architectures An Overview In Tree based Heterogeneous FPGA Architectures Springer 7 48 2012 Hamblen J O Hall T S and Furman M D Rapid prototyping of digital systems SOPC edition Springer 2007 Muralidharan N and Noel A The System on Chip Technology Proceedings of LACCEI 2004 4 Altera DE2 Board User Manual URL ftp ftp altera com up pub Webdocs DE2_UserManual pdf Arrow SoCKit Evaluation Board User Manual URL http www rocketboards org pub Documentation ArrowSoCKitEvaluationBoard SoCKit User manual pdf Lariviere D and Edwards S A uClinux on the Altera DE2 Columbia University 2008 uClinux dist Developers Guide Armadillo 2004 URL http download atmark techno com common uclinux_dist_ developers_guide_en 1 1 paf Atitallah A B Kadionik P Masmoudi N and Levi H FPGA implementation of a HW SW platform for multimedia embedded systems 13 14 15 16 17 18 19 20 21 22 23 50 Design Automation for Embedded Systems 2008 12 4 293 311 Chu P P Embedded SoPC Design with NIOS II Processor and Verilo
11. Zabidi for constant encouragement and guiding me throughout the process of this project Also I would like to thank my supervisor again for his support and lend me the latest Arrow SoCKit Evaluation board to complete my project I would also like to thank our Vecad Lab Assistant for their co operations helps and lend me the Altera DE2 development board for my project Besides that I would like to extend my sincere gratitude to my seniors and my friends who had given me a lot of guidance and help during the implementation of this project I truly appreciate their technical advices and concern throughout the process of this project Then I would like to thank my family who always giving me full support during my academic years I am appreciative I had gained a lot of knowledge as well as problems solving skills throughout the project ABSTRACT Nowadays with the mass popularity of digital cameras and high pixel camera Smartphones people are able to capture interesting things surrounding them However most of the captured pictures are stored in the storage card computer and only a small part of the pictures are printed out Few reasons people are seldom to print out their pictures is cost printing is expensive and limitation of space to put the photo frame in their living room This report presents how to develop a simple embedded system prototype of Digital Photo frame using FPGA technology The development platforms that were researched are the
12. compilation 7 Block Diagram soc_system led_pio_external_connection ipsw_pio_external_connection utton_pio_external_connection Figure 4 9 Block Diagram 42 Figure 4 10 show the Preloader and U Boot are able to work in the SoCkit board and load the kernel image file EP COM6 PuTTY Figure 4 10 Preloader and U Boot Figure 4 11 show the kernel start compiled under Linux platform when make ulmage command are typed Figure 4 12 show the kernel image file are successfully created and ready to use pang pang VirtualBox linux pang pang VirtualBox linux make uImage CHK CHK include generated uapi linux version h include generated utsrelease h include generated mach types h is up to date kernel bounds s include generated bounds h arch arm kernel asm offsets s include generated asm offsets h scripts checksyscalls sh scripts mod modpost o scripts mod modpost init main o include generated compile h init version o init do_mounts o init do_mounts_rd o init mounts o init noinitramfs o init calibrate o init init_task o init built in o arch arm vfp vfpmodule o arch arm vfp entry o Figure 4 11 Kernel compilation 43 pang pang VirtualBox linux sort done marker at 519ca4 5YSMAP System map OBJCOPY arch arm boot Image arch arm boot Image is ready arch arm boot compressed head o arch arm boot compressed piggy gzip arch arm boot compressed piggy gzip
13. constraint only a limited number of photo frames can place in our living room Secondly the photos can get tarnished over the years due to environmental conditions Hence the digital photo frames was introduced to overcome the limitations of the traditional picture frame A digital photo frame is basically an LCD display that displays photo without the need of a computer or printer which mean that we can display collection of our favorite photos on a single location Also there is no risk of the image being destroyed by environmental factors because the photos are digital Digital photo frames also can provide great opportunities for increasing social interactions in the home that will improve the experience of people s lives by supporting and increasing the emotional connections among them This project aims to implement a digital photo frames using field programmable gate array FPGA System on a Chip Technology The implementation can be divided into hardware and software parts Two different development platforms were used to achieve the objectives of this project that is Altera DE2 development board and Arrow SoCKit Evaluation Board Integration tools was used to integrate the needed components in hardware part such as VGA output and USB devices On the software part the Linux platform was used to build the digital photo frame application as Linux has strong support for different architectures especially for the ARM architecture
14. features Press lt Esc gt lt Esc gt to exit lt gt for Help lt gt for Search Legend built in excluded lt M gt module lt gt Libc is None Default all settings lose changes NEW Customize Kernel Settings NEW Customize Application Library Settings NEW Update Default Vendor Settings NEW lt Exit gt lt Help gt Figure 3 6 Kernel Customization 23 The source code of ISP1362 in uClinux kernel needed to modify in order the USB port of the DE2 board able to detect the incoming USB devices The Figure 3 7 shows that the ISP1362 can t setup when uClinux OS was booted into the DE2 board Figure 3 8 shown is the original code of ISP1362 v come with the uClinux distribution source Figure 3 9 shows the ISP1362 was modified in order it can detect any incoming USB devices Nios EDS 9 1 eth dm9BBBa at 806810f8 80G6818fc IRQ 6 MAC G6 66 60 00 06 68 chip iriver isp1362 hed 2805 84 84 isp1362 hed isp13 hed ISP1362 Host Controller isp1362 hed isp1362 hed new USB bus registered assigned bus number i i 2_hc_reset Clock not ready after 100ms isp1362 hcd 4sp1362 hcd can t setup isp1362 hed isp1362 hed USB bus 1 deregistered 2_probe init error 19 ng USB Mass Storage driver red new interface driver usb storage support registered registered Registered protocol family 17 tered udp transport module ed tcp transport module sed kernel memory 628k freed 6xa
15. frame project Then some comment and suggestions for the future improvement will be provided in this chapter CHAPTER 2 LITERATURE REVIEW This chapter covers what have been done on some other researcher projects and prototype of digital photo frames Some of the components and design are being considered when designing my project of digital photo frames using FPGA 2 1 Altera SOPC system used electronic digital photo frames Figure 2 1 shows the prototype of digital photo frame created by two students from Feng Chia University China 1 They use the Altera DE2 70 Cyclone II EP2C70F896C6N board as their development platform and a 4 3 inch touch screen LCD to display the digital photo This development board equipped with almost 70 000 logic elements and offers a rich set of features that make it suitable to be used for the development of any sophisticated digital systems This digital photo frame can perform two image features which is the negative image extraction and the gray level slicing function The on board button on DE2 70 board was configured as user interface for user to control the digital photo frame such as forward or backward the picture on the screen Besides the touchscreen LCD enables the user to change the picture on the screen by just touching the screen Users can easily access various components on the DE2 board from their host computer using the control panel facility software provided by Altera Altera DE2 con
16. gspi inst CLK inout wire hps 0 hps io hps io sdio inst CMD inout wire hps 0 hps io hps io sdio inst DO inout wire hps 0 hps io hps io sdio inst DI output wire hps 0 hps io hps io sdio inst CLK inout wire hps 0 hps io hps io sdio inst D2 inout wire hps 0 hps io hps io sdio inst D3 inout wire hps 0 hps io hps io usbl inst DO inout wire hps 0 hps io hps io usbl inst D1 inout wire hps 0 hps io hps io usbl inst D2 inout wire hps 0 hps io hps io usbl inst D3 inout wire hps 0 hps io hps io usbl inst D4 inout wire hps 0 hps io hps io usbl inst D5 inout wire hps 0 hps io hps io usbl inst D6 inout wire hps 0 hps io hps io usbl inst D7 input wire hps 0 hps io hps io usbl inst CLK output wire hps 0 hps io hps io usbl inst STP input wire hps 0 hps io hps io usbl inst DIR input wire hps 0 hps io hps io usbl inst NXT output wire hps 0 hps io hps io spim0 inst CLK output wire hps 0O hps io hps io spim0 inst MOSI 0 0 output wire hps hps_io_hps_io_spim0_inst_SS0 output output input output input output inout inout inout inout inout inout inout inout inout inout inout input input input input input output inout ifdef input output input input output endif ifdef output output output 57 wire hps_0_hps_io_hps_io_spiml_inst_CLK wire hps_O_hps_io_hps_io_spiml_inst_MOSI wire hps_O_hps_io_hps_io_spiml_inst_MISO wire hps 0 hps
17. io hps io spim0 inst CLK 0 hps io hps io 0 hps io hps io spim0 inst MOSI 0 hps io hps io 0 hps io hps io spim0 inst MISO 0 hps io hps io 0 hps io hps io spim0 inst SSO 0 hps io hps io 0 hps io hps io spiml inst CLK 0 hps io hps io O hps io hps io spiml inst MOSI 0 hps io hps io 0 hps io hps io spiml inst MISO 0 hps io hps io O hps io hps io spiml inst SSO 0 hps io hps io 0 hps io hps io uart0 inst RX 0 hps io hps io uart0 inst RX 0 hps io hps io uart0 inst TX 0 hps io hps io uart0 inst TX 0 hps io hps io i2c1 inst SDA 0 hps io hps io i2c1 inst SDA 0 hps io hps io i2c1 inst SCL 0 hps io hps io i2c1 inst SCL 0 hps io hps io 0 hps io hps io 0 hps io hps io 0 hps io hps io gpio inst GPIOO9 63 hps_0_hps_io_hps_io_gpio_inst_GPIO35 hps_O_hps_io_hps_io_gpio_inst_GPIO35 hps_0_hps_io_hps_io_gpio_inst_GPI048 hps_O_hps_io_hps_io_gpio_inst_GPI048 hps_0_hps_io_hps_io_gpio_inst_GPIO53 hps_O_hps_io_hps_io_gpio_inst_GPIO53 hps 0 hps io hps io gpio inst GPIO54 hps 0 hps io hps io gpio inst GPIO54 hps 0 hps io hps io gpio inst GPIO55 hps 0 hps io hps io gpio inst GPIO55 hps 0 hps io hps io gpio inst GPIO56 hps 0 hps io hps io gpio inst GPIO56 hps O hps io hps io gpio inst GPIO6I hps 0 hps io hps io gpio inst GPIO61 hps 0 hps io hps io gpio inst GPIO62 hps 0 hps io hps io gpio inst GPIO62 led pio external connection in p
18. o arch arm boot compressed misc o arch arm boot compressed decompress o arch arm boot compressed string o arch arm boot compressed libifuncs o arch arm boot compressed ashldi3 o arch arm boot compressed vmlinux OBJCOPY arch arm boot zImage Kernel arch arm boot zImage is ready UIMAGE arch arm boot uImage Image Name Linux 3 8 0 00111 g85cc90f dirty Created Sat May 3 01 48 43 2014 Image Type ARM Linux Kernel Image uncompressed Data Size 2861992 Bytes 2794 91 kB 2 73 MB Load Address 00008000 Entry Point 00008000 Image arch arm boot uImage is ready pang pang VirtualBox linux _ U Figure 4 12 Kernel compiles successfully Figure 4 13 show the final result of implementation digital photo frame using Arrow SoCkit Evaluation board Once the SoCKit board was powered on the application will start up automatically and searching the picture folder to display the picture on the VGA monitor In additions slide show mode is enabled on this digital photo frame application The keyboard will be used as a controller for the digital photo frame Figure 4 14 shows the enlarged picture when button on the keyboard being pressed Figure 4 15 shows the zoom out picture when button on the keyboard being pressed Figure 4 16 shows the picture is rotated when CTRL R button on the keyboard being pressed Figure 4 13 Original picture Figure 4 14 Zoom in picture 44 45 Figure 4 15 Zoom out picture
19. ownload jpg iger jpg iming png cad presentation_1 pptx mnt gt nxview tiger jpg amp 31 mnt GdLoadImageFromFile can t open image tiger jpg Figure 4 6 Fail to open picture yclone Il EP2C35F672C6 DigitalFrame al 1732 kB 164 kB 5456 kB 1356 kB 164 kB home gt nxview download jpg GdLoadImageFromFile can t open image download jpg Can t load image file download jpg mnano client closed socket 5 home gt ish 31 Child 34 died Figure 4 7 nano x closed unexpectedly 41 4 2 Arrow SoCKit Evaluation Board Implementation Result The digital photo frame was successfully implemented on the SoCKit board Implementation was in software and hardware Firstly Quartus II 13 0sp1 software are needed to build the system of the digital photo frame Osys integration tools will be used to integrate the needed component like ARM processor DMA frame buffer VGA output and other components into a system After generating the system top entity Verilog code is created before running full Ouartus compilation using Ouartus II 13 0sp1 software Figure 4 8 show the full compilation was successfully run under the Ouartus II software Figure 4 9 show block diagram of the created system under Quartus II software Dev FOUR ETE oF ranes medafunctons attera_pl v K ga A pil_va pll_voa_o002 v EG BA Figure 4 8 Full
20. project a stable digital photo frame prototype was implemented which have ability to display photos automatically on the VGA monitor screen with slide show mode once the SoCKit board are powered on 5 2 Future Works and Recommendations Although this project is considered successful there are still many improvements can be done in future time to optimize the performance of the digital photo frame 48 Below are the suggestions for the future improvement Implement hardware JPEG decoder to reduce power To integrate the motion detection function into the digital photo frame This function will be helping us save the power consumption in our house as if no people s motions are detected then the digital photo frame will switch off automatically and switch on if people s motion are detected To integrate a Wi Fi function into the digital photo frame This Wi Fi function will bring convenience to users as they can easily upload their picture into the picture frame using a Wi Fi connected smartphone Accept hot plug USB thumb drive and auto playback of all image files found on thumb drive 10 11 12 REFERENCES Zan S Z and Hao K R Altera SOPC system used electronic digital photo frame Feng Chia University 2009 Olsson M Skanberg S and Eriksson T Digital Photo Frame Lund University 2009 Hong C Deng R and Ye Y Multi Functional Digital Albums Based on the Nios II Processor Beijing Jiaotong
21. system can be built in a short time An embedded operating system of uClinux was implemented to the Nios II processor to realize the complexities function of digital albums Then software application of this digital photo frame system was developed under Linux operating system Figure 2 4 Multi Functional Digital Album 24 A modularized FPGA Based Embedded System Development Platform A digital photo frame project was developed using modularized FPGA MorFPGA platform MorFPGA is comprised of FPGA Core Module board with memory modules and peripherals module board An open source processor core LEON3 4 which is a synthesizable VHDL model of 32 bit processor were used to implement the project The feature of the MorFPGA platform is Modularization which enhances the expandability and configurability Hence users can develop other projects by just stack the needed peripheral board on top of the basic FPGA Core Module board Figure 2 5 a shows the block diagram of the FPGA Core Module board which consists of FPGA and its corresponding serial configuration device JTAG buttons and external memory modules The memory module includes the memory components such as SRAM SDRAM and NAND flash Figure 2 5 b illustrates the peripheral module which includes switches buttons keypad 7 segment display LCD RS232 VGA PS 2 and audio Figure 2 5 c shown is the result after the peripheral module board is stacked on top of the FPGA co
22. tool Fie Edit System View Tools Help Component Library System Contents Address Map Clock Settings Project Settings instance Parameters System inspector HDL Example Generation fa use Connections Name Description Export Clock Base End Project X m B cik_0 Clock Source 10 New Component B oe kn Clock input leik 2 log generator 97 en cik in reset Reset Input reset Other me Clock Output eko System a em AA AAA A ok reset Reset Output Library YU E hps_0 Hard Processor System zultipie E Bridges f t h2f_reset Reset Output hps_0_h2f reset Es Clock and Reset A f2h axi clock Clock Input cik 0 Configuration amp Programming T PP rai slove AXI Slave 121_ax_clo 0x0000_0000 sp gt n2f_exi_clock Clock Input EH Embedded Processors 7 T h2f axi master AX Master Interface Protocols gt h2f Iw axi clock Clock Input GI Memories and Memory Controller T T h2f_lw_axi_master AXI Master Merin Components memory Conduit memory Microcontroller Peripherals hps_jo Condut hps 0 hps io Peripherals El master secure JTAG to Avalon Master Bridge BALL ok Clock Input cik E Qsys Interconnect ck reset Reset Input E Verification master Avalon Memory Mapped Master teng Window Bridge T master reset Reset Output B sysid_gsys System ID Peripheral gt ck Clock Input cik_0 i gt reset Reset Input teng
23. wire vga wire wire wire wire wire wire wire wire audio wire wire wire wire wire temp_cs_n temp_sclk temp_mosi temp_miso vga_clk vga_hs vga_vs 7 0 vga_r 7 0 vga_g 7 0 vga_b vga_blank_n vga_sync_n aud_adcdat aud_adclrck aud_bclk aud_dacdat aud dacirck 59 output wire aud_i2c_sclk inout wire aud_i2c_sdat output wire aud_mute output wire aud_xck endif wire 1 0 fpga debounced buttons wire 3 0 fpga_led_internal wire hps_fpga_reset_n debounce debounce_inst clk clk_50m_fpga reset_n hps_fpga_reset_n data_in user_pb_fpga data out fpga debounced buttons defparam debounce_inst WIDTH 2 defparam debounce_inst POLARITY LOW defparam debounce_inst TIMEOUT 50000 defparam debounce_inst TIMEOUT_WIDTH 16 pll_vga pll_vga_inst refclk cik 50m fpga rst 1 b0 outclk_0 vga_clk_25m wire vga_clk_25m wire 7 0 vga_color soc_system u0 clk_clk clk_botl reset_reset_n hps_fpga_reset_n memory_mem_a memory_mem_a memory_mem_ba memory_mem_ba memory_mem_ck memory_mem_ck memory_mem_ck_n memory_mem_ck_n memory_mem_cke memory_mem_cke memory_mem_cs_n memory_mem_cs_n 60 memory_mem_ras_n memory_mem_ras_n memory_mem_cas_n memory_mem_cas_n memory_mem_we_n memory_mem_we_n memory_mem_reset_n memory_mem_reset_n memory_mem_dg memory mem dg
24. 2 control siave Avalon Memory Mapped Slave eng amp 020001_0000 0x0001_000 El B led_pio PIO Parallel VO T gt ck Clock input cik 0 i Moo E i ma Reset Input CK dk E O ER dd IEA M M DI s1 Avalon Memory Mapped Slave icik Ox0001_0040 0x0001_004 New Edt Add B LI terna connection __ Cont lari nin external connec J gt is Path The module properties SIMULATION_MODEL_IN_VERILOG and SIMULATION_MODEL_IN_VHDL can not both be set when using the SIMULATION file property System att vip vfr vga As pil stream locked must be exported or connected to a matching conduit System pll_stream 2 Interrupt sender alt_vip_vfr_vga interrupt_sender is not connected to an interrupt receiver Systemat_vip_vfr_vga File Edit System View Tools Help Figure 3 20 Qsys partl Component Library Clock Settings Project Settings instance Parameters System Inspector HOL Example Generation Name Description Export Clock Base End ck reset keset input A u master Avalon Memory Mapped Master ck 5 er master_reset Reset Output tae El intr_capturer_0 Interrupt Capture Module a clock Clock input ciko ime 0 Ime 31
25. 24000 BxabeBBB gt ked to run file etc re Command hostname uClinux Command mount t proc proc proc o noexec nosuid nodev ommand mount t sysfs s ys 0 noexec nosuid nodey mand mount t devpts devy ev pt 0 noexec nosuid Command mount t usbfs none proc bus usb Connand mkdir var tnp command mkdir var log onnand mkdir var run Command mkdir var lock onnand mkdir var enpty Command ifconfig lo 127 0 0 1 onnand route add net 1 8 8 netmask 255 0 0 8 lo sommand cat etc motd lelcome to Figure 3 7 ISP 1362 setup fail 24 isp1362 hcd c if isp1362_hcd gt board 8 isp1362_hcd gt board gt reset isp1362_hcd gt board gt reset hcd gt self controller 1 msleep 20 if isp1362_hcd gt board gt clock isp1362_hcd gt board gt clock hcd gt self controller 1 isp1362_hcd gt board gt reset hcd gt self controller 0 else isp1362_sw_reset isp1362_hcd chip has been reset First we need to see a clock t jiffies msecs_to_jiffies timeout while clkrdy 8 time_before_eq jiffies t spin_lock_irgsave amp isp1362_hcd gt lock flags clkrdy isp1362_read_reg16 isp1362_hcd HCUPINT amp HCuPINT_CLKRDY spin_unlock_irqrestore amp isp1362_hcd gt lock flags if clkrdy msleep 4 m spin_lock_irgsave amp isp1362_hcd gt lock flags 1sp1362 write reg16 isp1362 hcd HCuPINT HCuPINT_CLKRDY spin unlock irgrestore amp isp1362 hcd lock flags if
26. 28 embedded system Once the SoCKit was powered on the pictures were read from the MicroSD card file system and decode to RGB format The decoded image streams are then sent via DMA to the Video IP chain within the FPGA before it s sent to the video output block This SoCKit board contains 2gb DDR3 memory hence a high definition HD of JPEG picture was able to display on the VGA monitor This project will use both the HPS and FPGA portion of the SoC concurrently to implement an embedded system The embedded operating system was used to speed up the development of prototypes The picture output for the digital photo frame interface is driven from the VGA connector on the SoCKit board Altera s Video IP suite was used to implement video pipeline in the FPGA portion of the SoC Open source picture viewer was installed in the embedded OS to speed up the prototyping time of digital photo frame The final result of this project will show that digital photo frame work under SoCKit board and can display picture in slide show mode on a VGA monitor 3 4 2 Implementation of Linaro linux operating system 3 4 2 1 Building Linux kernel Embedded kernel was compiled under the Ubuntu Linux operating system Stable Linux kernel version of 3 8 was download from the RocketBoards org website 1 Source was downloaded using command git clone git git rocketboards org linux git 2 Archieve file of the source code was unpack and obtain
27. Altera DE2 development board and the Arrow SoCKit Evaluation board Qsys Integration tool in Quartus II was used to integrate the needed peripheral into the digital photo frame system The picture output for the digital photo frame interface is driven from the VGA connector on the development boards An embedded operating system will be customized and port into the development board to work together with the FPGA The Linux kernel was added a custom frame buffer driver to support the functionality of VGA at FPGA part The open source picture viewer was then installed in the embedded OS to speed up the prototyping time of digital photo frame The final result of this project shows that digital photo frame works steadily under the FPGA development board and can display picture in slide show mode on a VGA monitor vi ABSTRAK Kini dengan populariti massa kamera digital dan Telefon pintar kamera yang tinggi pixel manusia boleh menangkap perkara yang menarik di sekeliling mereka dengan mudah Walau bagaimanapun kebanyakan gambar gambar yang ditangkap disimpan dalam kad penyimpanan komputer dan hanya sebahagian kecil daripada gambar gambar yang akan dicetak keluar Antara sebab yang menyebabkan kebanyakan orang jarang mencetak gambar gambar mereka adalah kos percetakan yang mahal dan had ruang untuk meletakkan bingkai foto di ruang tamu mereka Oleh itu kemunculan bingkai foto digital menyediakan manusia dengan pilihan yang baik Laporan ini akan me
28. CPU will load a predefined image from the SD card and write it to the 16MByte Micron SDRAM The CPU will just run an idle loop after completed loading the image and waits for interrupts from the VGA controller The interrupt routine fetches one row from the SDRAM and writes it to the BRAM buffer in the VGA controller at each time the interrupt received from the VGA controller Atthe VGA controller side interrupts are sent when the end of the visible scan line is reached VGA monitor Figure 2 3 Architecture overview 2 3 Multi Functional Digital Albums Figure 2 4 shown is the multi functional digital album created by students from the Information Science institute Beijing Jiaotong University 3 The digital album was implemented using an Altera DE FPGA board and a 5 7 inches LCD screen This development board consists of an Altera Cyclone II 2C20 FPGA chip with 20000 logic elements The multi functional digital album designed by them able perform a lot of functions like can play background music while viewing photos digital watermark embedding and extraction digital photo editions and processing effects and digital photo network transmission This digital album was designed to read the image file from the SD card Therefore the Avalon bus and the Nios II processor were used to control the SD card data reading and writing SOPC Buider s powerful system integration feature was used to shorten the design cycle as a stable hardware
29. DIGITAL PHOTO FRAME IMPLEMENTATION ON FPGA SYSTEM WITH EMBEDDED OS PANG CHUN CHET UNIVERSITI TEKNOLOGI MALAYSIA PSZ 19 16 Pind 1 07 UNIVERSITI TEKNOLOGI MALAYSIA DECLARATION OF THESIS UNDERGRADUATE PROJECT REPORT AND COPYRIGHT Author s full name PANG CHUN CHET Date of Birth 10 2 1990 Title l DIGITAL PHOTO FRAME IMPLEMENTATION ON FPGA SYSTEM WITH EMBEDDED OS Academic Session 2013 2014 declare that this thesis is classified as CONFIDENTIAL Contains confidential information under the Official Secret Aci 1972 RESTRICTED Contains restricted information as specified by the organization where research was done a OPEN ACCESS agree that my thesis to be published as online open access full text acknowledged that Universiti Teknologi Malaysia reserves the right as follows 1 The thesis is the property of Universiti Teknologi Malaysia 2 The Library of Universiti Teknologi Malaysia has the right to make copies for the purpose of research only 3 The Library has the right to make copies of the thesis for academic exchange Certified by 4 SIGNATURE SIGNATURE OF SUPERVISOR PM Muhammad Mun im Bin Ahamd 900210 13 7819 Zabidi NEW IC NO PASSPORT NAME OF SUPERVISOR Date 22 JUNE 2014 Date 22 JUNE 2014 NOTES si If the thesis is CONFIDENTAL or RESTRICTED please attach with the leiter from the organization with period and reasons for confidentiality or restriction I hereby dec
30. Figure 4 16 Rotated picture Figure 4 17 shows the 128x64 dots LCD on SoCKit board will display Digital Photo Frame when the application is startup 46 Figure 4 17 128x64 dots LCD Figure 4 18 shows the logo created for this digital photo frame prototype When user power on the SoCKit board this logo will appear in the VGA monitor left top area before it start the digital photo frame application on embedded Linux AE Figure 4 18 Logo of digital photo frame CHAPTER 5 CONCLUSION In this chapter all of the results of this project will be concluded here In addition some future works and recommendations are also presented 5 1 Conclusion The main objective of this project was to prototype a digital photo frame using FPGA board This objective were successfully archieved at the end of the project A complete digital photo frame prototype was successful implemented on Arrow SoCKit Evaluation board Implementation was divided into hardware and software parts Digital photo frame hardware design system was built using Qsys integration tool For the software part an embedded Linux operating system was used to speed up the development time as it is free and have a strong support source from the internet In addition the kernel is flexible and can be enhanced by embedded developers The open source picture viewer was successfully installed on embedded OS to display the image file from the MicroSD card At the end of the
31. MODPOST vmlinux o WARNING modpost Found 2 section mismatch es To see full details build your kernel with make CONFIG_DEBUG_SECTION_MISMATCH y GEN version include linux compile h include linux compile h init version o init built in o vmlinux SYSMAP System map OBJCOPY arch nios2 boot vmlinux bin arch nios2 boot vmlinux gz arch nios2 boot compressed piggy o arch nios2 boot compressed vmlinux DBJCOPY arch nios2 boot zImage Kernel arch nios boot zimage ts ready Makefsf rteavtng dtreetury 7U35T7tocal src mios2 linux linux 2 6 cp usr local src nios2 linux uClinux dist linux 2 6 x arch nios2 boot zImage u sr local src nios2 linux uClinux dist images zImage make 2 Leaving directory usr local src nios2 linux uClinux dist vendors Alte ra nios2 make 1 Leaving directory usr local src nios2 linux uClinux dist vendors root pang VirtualBox usr local src nios2 linux uClinux dist Figure 4 2 zImage file 1 Nios I EDS 9 1 driver isp1362 hcd 2885 84 84 isp1362 hed isp1362 hed ISP1362 Host Controller isp1362_hc_reset Clock not ready after 10Gms isp1362 hed isp1362 hed can t setup isp1362 hed isp1362 hed USB bus 1 deregistered ispi362_probe init error 19 Initializing USB Mass Storage driver usbcore registered new interface driver usb storage USB Mass Storage support registered TCP cubic registered Registered protocol family 17 Registered udp transport module Registered tcp transport mod
32. Open source picture viewer that can support a wide range of image formats including JPEG PNG Bitmap and TIFF The main format of the image was used throughout this project is JPEG Joint Photographic Experts Group 1 5 Outline of Thesis This report consists of five chapters and each chapter will be briefly discussed here Chapter 1 will give an overview of this digital photo frame project The project background the problem statement project objective scope of the project and project outline will be included in this chapter The next chapter which is chapter 2 will be focusing on the previous works that have done by other researchers some FPGA theory and SoC embedded system knowledge It will be discussed on the various types of techniques and methods to implement a digital photo frame prototype Besides that chapter 3 describes the details on the methodology of this project This project consists of three layers which is FPGA system hardware embedded Linux and user space application program Next this chapter will explain the steps and methods to install an embedded OS into the development boards The explanation of the hardware design will be discussed here also Next Chapter 4 is discussing about the result obtained for this project Then some discussions on the results of the project will be included in this chapter also Finally Chapter 5 discusses the conclusion based on the result obtained for this digital photo
33. SP1362 OTG FSPEED OTG INTO to the ISP1362 OTG INTO OTG INTI to the ISP1362 OTG INTI OTG LSPEED from the ISP1362 OTG LSPEED OTG RD N from the ISP1362 OTG RD N OTG RST N from the ISP1362 OTG RST N OTG WR N from the ISP1362 OTG WR N in port to the buttons KEY Out port from the RedLed LEDR zs addr from the sdram DRAM ADDR zs ba from the sdram DRAM BA 1 DRAM BA 01 zs cas n from the sdram DRAM CASN zs cke from the sdram DRAM CKE z8s cs n from the sdram DRAM CS N zs dg to and from the sdram DRAM DQ 55 zs_dgm_from_the_sdram DRAM_UDQM DRAM LDQM zs_ras_n_from_the_sdram DRAM_RAS N zs_we_n_from_the_sdram DRAM WEN ext_bus_data ext_bus_data iCLK_25_to_the_vga_controller_0 CLK_25 read_n_to_the_sram read_n_to_the_sram reset_n KEY 0 rxd_to_the_uart0 UART_RXD select_n_to_the_sram select_n_to_the_sram txd_from_the_uartO UART_TXD write_n_to_the_sram SRAM WEN address_to_the_sram SRAM_ADDR be_n_to_the_sram be_n_to_the_sram endmodule A 2 Arrow SoCKit Evaluation Board include module output output output output output output output output output output inout inout inout output output input output output output c5sx_soc wire 14 0 wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wire wi
34. a linux directory Go into the linux directory using command cd linux 3 A new branch was created from the remote arrow_sockit_vga using command below 29 git checkout b neek_soc_38 origin arrow_sockit_vga 4 PATH variable was set in the rightway to ensure the compile tool arm linux gnueabihf gcc was available 5 Following command was run to setup the build environment export ARCH arm export CROSS_COMPILE arm linux gnueabihf export LOADADDR 0x8000 6 Default configuration was import to enable new drivers make socfpga_defconfig 7 Command to start linux kernel compilation make ulmage 8 Device tree was build in order to get the drivers to work make socfpga_cyclone5 dtb Some of the files created after run the compilation are shown in Table 3 2 Table 3 2 Files created File Description vmlinux Linux Kernel ELF file arch arm boot ulmage Linux Kernel image arch arm boot dts socfpga_cyclone5 dtb Device Tree Binary 30 3 4 2 2 Linaro Root File System In order to get Linaro Linux build on Arrow SoCKit Evaluation board Linaro images are downloaded from the https releases linaro org 13 04 ubuntu quantal images nano After unzip the tar gz file a binary folder will obtained The linaro rootfs are located in the binary subdirectory where the tar gz file was extracted The following steps is
35. clkrdy pr_err Clock not ready after lums n timeout ret ENODEV un Ed original Figure 3 8 original code of ISP1362 isp1362 hcd c Y unsigned long t unsigned long timeout 100 unsigned long flags int clkrdy 0 pr_info s n __func__ if isp1362_hcd gt board amp amp isp1362_hcd gt board gt reset isp1362_hcd gt board gt reset hcd gt self controller 1 msleep 20 if isp1362_hcd gt board gt clock isp1362_hcd gt board gt clock hcd gt self controller 1 isp1362_hcd gt board gt reset hcd gt self controller 0 else isp1362_sw_reset isp1362_hcd chip has been reset First we need to see a clock t jiffies msecs_to_jiffies timeout while clkrdy 88 time_before_eg jiffies t spin_lock_irqsave amp isp1362_hcd gt lock flags clkrdy isp1362_read_reg16 isp1362_hcd HCuPINT amp HCuPINT_CLKRDY spin_unlock_irqrestore amp isp1362_hcd gt lock flags if clkrdy msleep 4 m J spin_lock_irqsave amp isp1362_hcd gt lock flags 1sp1362 write reg16 isp1362 hcd HCuPINT HCuPINT_CLKRDY spin unlock irgrestore amp isp1362 hcd lock flags if clkrdy pr_err Clock not ready after lums n timeout ret ENODEV 3 return 1 lt change tol m Figure 3 9 Modified ISP 1362 code 3 3 4 Hardware system Integration The hardware design should at least consist of the main components needed for the application The components i
36. connected to a matching conduit System plL_stream A interrupt sender alt vip vfr vaa interrupt sender is not connected to an interrupt receiver Figure 3 21 Qsys part2 System alt_vip_vfr_vga 35 CHAPTER 4 RESULTS AND DISCUSSIONS This section will discuss about the final result of my final year project Results are obtained in the hardware part and software part for two different development platforms 4 1 Altera DE2 Development Board Implementation Result The uClinux image file was succesfully loaded through Nios II command shell into the Altera DE2 FPGA board Two steps are needed to configure the DE2 board to run the uClinux operating system Firstly FPGA must be configured to implement the Nios II processor system Figure 4 1 shows the result of system was succefully be generated and compiled using Quartus II software 43 Compilation Report Flow Summary 33 Compilation Report Flow Summary SB Legal Notice BEE Flow Summary SIS Flow Settings SES Flow Non Default Global Setting BEE Flow Elapsed Time BEE Flow OS Summary SB Flow Log 1 81 Analysis amp Synthesis paa Fitter 81 Assembler a g Timing Analyzer Flow Status Quartus Il Version Revision Name Toplevel Entity Name Family Device Timing Models Met timing requirements Total logic elements Quartus II O Full Compilation was successful 540 warnings Total combinational functions Dedicated logic re
37. cture of digital photo frame using the Arrow SoCkit Evaluation board Custom FPGA peripheral will be added to the ARM processor using Qsys system integration tool Embedded Linux was used in this project as it is freeware for all users MicroSD card will be used to store the application system and the image file The open source picture viewer was used to decode the image file before it s able to display on the VGA monitor The open source picture viewer was used as it is free and can shorten the development time of the digital photo frame prototype The USB keyboard will be connected to the SoCkit board through the OTG cable and act as the controller of digital photo frame it can used to zoom in zoom out rotate and move forward or backward the image on a VGA monitor 27 Figure 3 12 System Architecture using Arrow SoCKit Board Figure 3 13 shows the overall view of the steps to design and implement the digital photo frame prototype using the SoCKit Cyclone v platform Embedded Linux ae Images stored in OS was used to ys pda _ MicroSD card speeds up the generated using the development of the Qsys integration tool prototype JPEG picture was bedded Li display in VGA monitor Open Source Picture mn ne Ae aes a in slide show mode once viewer was installed tino ihe Processor the SoCKit Board was powered up Figure 3 13 Design steps The image file will stored on the MicroSD card to obtain a stands alone
38. d Arrow SoCKit Evaluation Board 2 Middle layer Embedded linux e g uClinux linaro Linux 3 Top layer User space application program 3 2 Tools Used There are several development tools are used to develop the prototype of digital photo frame These include the Integrate Development Environment IDE used for hardware and software design and the Linux operating system was used for software development Integrated Development Environment tools of QuartusII 9 1 and NiosII 9 1 were used to prototype digital photo frame on the Altera DE2 Development Board Whereas for the SoCKit Evaluation Board development tools of QuartusII 13 0sp1 and SOC EDS 13 0 was used 16 The Ubuntu Linux operating system was installed on the Oracle VM VirtualBox to speed up the prototyping time VirtualBox is a cross platform virtualization application that enable us to extend the capabilities of our existing computer so that it can run multiple operating system at the same time 3 2 1 Quartus II and Embedded Design Suite EDS Altera Quartus II is a development tool produced by Altera It is used for analysis and synthesis of HDL design which is used to compile the hardware design configure the target device with the programmer and perform timing analysis by clock setting The hardware description language that used to complete this project is Verilog HDL coding QuartusII 9 1sp1 was used on the Altera DE2 development board as it has stable performa
39. era SoC integrate an ARM based hard processor system HPS consisting of processor peripherals and memory interfaces with the FPGA fabric using a high bandwidth interconnect backbone This board contains a powerful chip which integrates a 28nm Cyclone V FPGA 5CSXFC6D6F31 with a ARM Cortex A9 processor The FPGA chip consists of 110K programmable logic elements In addition this board includes hardware such as of two low power DDR3 memory a graphic LCD MicroSD card slot VGA output Ethernet networking and much more Figure 3 2 shows the Arrow SoCKit Evaluation board hardware platform 00 INTEGRATE Soc Figure 3 2 Arrow SoCKit Evaluation Board Figure 3 3 shows the bottom view of SoCKit board FPGA Configuration Mode Switch QSPI Flash 128MB Micro SD Connector Figure 3 3 Bottom view of SoCKit Board LCD Connector 18 19 3 3 Implementation of system on Altera DE2 Development Board 3 3 1 System architecture At the beginning of the project the Altera DE2 development board was used to implement the digital photo frame prototype The design of digital photo frame was realized by the technique of SOPC based on Nios II soft core processor An embedded operating system of uClinux was installed on the SDRAM of the FPGA board The problem with using an operating system on DE2 board is the memory concerns as its uses most of the memory in the SDRAM hence the kernel size is needed for proper customized in
40. follow to copy the Linaro Root Filesystem to the Linux partition 1 mount dev sdx2 sdcard D cp binary sdcard B umount sdcard 4 sync 3 4 2 3 Preloader and U Boot Customization The Hard Processor System HPS boot starts when a processor is released from reset for example power on the SoCKit board and executes code in the internal boot ROM at the reset exception address The boot ROM contains software code that executes after a reset button was pressed The typical HPS boot flow was shown in Figure 3 14 Description of the different boot stages are shown in Table 3 3 User Software Operating SB Figure 3 14 Typical Boot Flow 31 Table 3 3 Description of the different boot stages Stage Description BootROM Performs minimal configuration and loads Preloader into 64KB OCRAM Preloader Configure clocking pinmuxing DDRAM and loads U Boot into DDRAM U Boot Configures FPGA loads Linux kernel Operating System Applications development The function of the preloader is user defined Typical functions of preloader include initializing the SDRAM interface configuring clocking and configuring the HPS I O pins Initializing the SDRAM allows the preloader to load the next stage of the boot software which is the open source boot loader or known as U boot Preloader are generated based on the handoff information folder from Quartus Handoff information folder is obtained when a hardwa
41. g Examples John Wiley amp Sons 2012 Daya B Rapid Prototyping Of Embedded Systems Using Field Programmable Gate Arrays Florida University 2009 Devic F Torres L and Badrignans B Securing boot of an embedded linux on FPGA Parallel and Distributed Processing Workshops and Phd Forum IPDPSW 2011 IEEE International Symposium on IEEE 2011 189 195 Han D D Zhang T C and Zhang J Design and Implementation of Intelligent Digital Photo Frame Based on FPGA Applied Mechanics and Materials 2013 380 3296 3299 Kim J and Zimmerman J Cherish smart digital photo frames for sharing social narratives at home CHI 06 Extended Abstracts on Human Factors in Computing Systems ACM 2006 953 958 Lu Z Zhang X and Sun C An Embedded System with uClinux based on FPGA Computational Intelligence and Industrial Application 2008 PACIIA 08 Pacific Asia Workshop on IEEE 2008 vol 2 691 694 Paulose L and Cheedella P T Enhancement of Digital Photo Frame Capabilities With Dedicated Hardware Ph D Thesis 2012 RocketBoards org 2014 URL http www rocketboards org foswiki Main WebHome Williams J A and Bergmann N W Embedded Linux as a platform for dynamically self reconfiguring systems on chip Ersa 04 the 2004 International Conference On Engineering of Reconfigurable Systems and Algorithms CSREA Press 2004 163 169 Yang K p and Beaubouef T A field programmable gate array media
42. gisters Total registers Total pins Total virtual pins Total memory bits Embedded Multiplier bit elements Total PLLs OK Successful Thu Jun 05 10 52 36 2014 9 1 Build 350 03 24 2010 SP 2 SJ Web Edition DPF DPF Cyclone II EP2C35F672C6 Final Yes 3 358 33 216 10 3 063 33 216 9 2 114 33216 6 2182 106 475 22 0 46 848 483 840 10 4 70 6 0 4 0 Figure 4 1 Full compilation 37 Secondly kernel file was customized and compiled under the Linux platform before download into SDRAM of Altera DE2 Development board Nios II 9 1 command shell was used to load the embedded Linux operating system into the DE2 board The Command Shell window is aCygwin environment and is performed like the Linux shell Nios II command shell is a console window on the computer which was used to explore uClinux once the uClinux had booted successfully in the DE2 board Standard Linux commands such as Is cd kill ps can be used in uClinux Figure 4 2 show the uClinux kernel file was successfully compiled under Linux platform Figure 4 3 show an uClinux embedded operating system was successfully loaded into the DE2 board Altera DE2 board contain an 8MB SDRAM chip which will be used to store the root file system program memory and data memory hence if too many processes are running simultaneously the kernel will run out of memory and crash root pang VirtualBox usr local src nios2 linux uClinux dist
43. io hps io spiml inst SSO wire hps 0 hps io hps io uart0 inst RX wire hps 0 hps io hps io uart0 inst TX wire hps 0 hps io hps io i2c1 inst SDA wire hps 0 hps io hps io i2c1 inst SCL wire hps 0 hps io hps io gpio inst GPIOO0 wire hps 0 hps io hps io gpio inst GPIO35 wire hps 0 hps io hps io gpio inst GPIO48 wire hps 0 hps io hps io gpio inst GPIO53 wire hps 0 hps io hps io gpio inst GPIO54 wire hps 0 hps io hps io gpio inst GPIOS5 wire hps 0 hps io hps io gpio inst GPIOS6 wire hps 0 hps io hps io gpio inst GPIO6I wire hps 0 hps io hps io gpio inst GPIO62 cik 100m fpga cik 50m fpga cik topi clk_botl fpga_resetn wire clk_i2c_sclk wire clk_i2c_sdat user_peripheral 3 0 user dipsw fpga 3 0 user_led_fpga 3 0 user_pb_fpga wire irda_rxd wire fan_ctrl ddr3 14 0 ddr3_fpga_a 2 0 ddr3_fpga_ba ddr3_fpga_casn output output output output output inout inout inout output input input input input endif ifdef output output output input endif ifdef output output output output output output output output endif ifdef input input input output inout 3 0 31 0 3 0 3 0 58 ddr3_fpga_cke ddr3_fpga_clk_n ddr3_fpga_clk_p ddr3_fpga_csn ddr3_fpga_dm ddr3_fpga_dg ddr3_fpga_dqs_n ddr3_fpga_dqs_p ddr3_fpga_odt ddr3_fpga_rasn ddr3_fpga_resetn ddr3_fpga_wen ddr3 fpga rzg temp sense wire wire wire
44. is the FPGA design flow The flow of the design using FPGA outlines the whole process of device design and guarantees that none of the steps is overlooked Thus it ensures that we have the best chance of getting back a working prototype that will correctly function in the final system to be designed HDL Coding of the Design Verificstion of Functionality Synthesis Translate Place and Route Program the FPGA Figure 2 8 Design Flow of FPGA 12 2 8 Rapid Prototyping of Embedded Systems using FPGA Figure 2 9 show time to market and market window It shows the trend of the life cycle of embedded product is becoming increasing smaller This situation will lead to new developments taking place more frequently to replace the outdated products In addition the functionality and complexity of the embedded system are rapidly increasing and this definitely will require more time and manpower for the embedded system design cycle The consumer demand for variety functionality of a product directly increased the complexity of the embedded system on a chip 6 However complexity gap was exists between the application requirements and the capabilities of current silicon technologies f Typical time to market constraint 8 months enusnoy siqissog Market Window Time Figure 2 9 Time to market and market window Nowadays the demand for high complexity and variety functionality of an embedded product i
45. l 100 0 Use Connecti Module Name Description Clock Base End Tags IRQ mM El cpu_0 Nios Il Processor instruction master Avalon Memory Mapped Master clk data_master Avalon Memory Mapped Master IRQ 0 IRQ 31 jtag_debug_module Avalon Memory Mapped Slave Ox01b00800 0x01b00fff El B tri_state_bridge_0 Avalon MM Tristate Bridge avalon_slave Avalon Memory Mapped Slave cik tristate master Avalon Memory Mapped Tristate Master Y C E cfi_flash_0 Flash Memory Interface CFI s1 Avalon Memory Mapped Tristate Slave cik 0x01400000 0x017fffff 7 El sdram_0 SDRAM Controller s1 Avalon Memory Mapped Slave cik 50 0x00800000 0x00ffffff 7 E jtag_uart_0 JTAG UART d avalon_jtag_slave Avalon Memory Mapped Slave clk 0x01b010b0 0x01b010b7 gt LV E uart_0 UART RS 232 Serial Port s1 Avalon Memory Mapped Slave cik 0x01b01000 0x0ib0 gt 7 E timer 0 Interval Timer s1 Avalon Memory Mapped Slave cik 0x01b01020 H 7 E 15P1362 ISP1362 avalon_slave_0 Avalon Memory Mapped Slave cik 0x01b010a0 0x0 avalon_slave_1 Avalon Memory Mapped Slave Ox01b010b8 0x0 p T El vga_controller_0 Binary_VGA_Controller avalon_slave_0 Avalon Memory Mapped Slave cik 0x01800000 0x019fffff x la vv x Address Map Fiters Filter Default Figure 3 11 Hardware system integration 3 4 Implementation of system on Arrow SoCKit Evaluation Board 3 4 1 System architecture Figure 3 12 shows the design archite
46. lare that I have read this report and in my opinion this report is sufficient in terms of scope and guality for the award of the degree of Bachelor of Engineering Electrical Microelectronics la Mm PM MUHAMMAD MUN IM BIN AHMAD ZABIDI June 22 2014 Signature Name Date DIGITAL PHOTO FRAME IMPLEMENTATION ON FPGA SYSTEM WITH EMBEDDED OS PANG CHUN CHET A report submitted in partial fulfilment of the requirements for the award of the degree of Bachelor of Engineering Electrical Microelectronics Faculty of Electrical Engineering Universiti Teknologi Malaysia JUNE 2014 li I declare that this report entitled Digital Photo Frame Implementation On FPGA System With Embedded OS is the result of my own research except as cited in the references The report has not been accepted for any degree and is not concurrently submitted in candidature of any other degree Signature UN end li Name PANG CHUN CHET Date June 22 2014 111 Dedicated in appreciation for the support understandings and encouragement to my beloved parents sister and brother iv ACKNOWLEDGEMENT First of all I would like to take the opportunity to thank to University Technology Malaysia for giving me the opportunity to utilize academic and practical experiences in carrying out this final year project Next I would like to take the opportunity to express my deepest gratitude to my supervisor PM Muhammad Munim Bin Ahamd
47. lopment Board 20 3 2 Files created 29 3 3 Description of the different boot stages 31 3 4 HPS BOOTSEL and CLKSEL setting 32 FIGURE NO 21 2 2 3 2 4 23 2 6 2 7 2 8 2 9 2 10 3 1 3 2 3 3 3 4 3 5 3 6 3 7 3 8 3 9 3 10 3 11 3 12 3 13 3 14 3 15 3 16 LIST OF FIGURES TITLE Digital photo frame implemented on Altera DE2 70 board and a 4 3 inch Terasic touch screen LCD Nexys 2 FPGA and a VGA monitor used to implement a digital photo frames Architecture overview Multi Functional Digital Album Design Concept a FPGA Core Module Board b Peripheral Module Board and c Integrated Platform Implementation of digital photo frame on modularized FPGA platform Overview of FPGA architecture Design Flow of FPGA Time to market and market window FPGA Vs ASICs design cycle Altera DE2 Development board Arrow SoCKit Evaluation Board Bottom view of SoCKit Board System Architecture using Altera DE2 Board Platform selection Kernel Customization ISP 1362 setup fail original code of ISP1362 Modified ISP 1362 code Hardware design Hardware system integration System Architecture using Arrow SoCKit Board Design steps Typical Boot Flow Preloader Overview BOOTSELECT and CLOCKSELECT jumpers setting xi PAGE 10 11 12 14 17 18 18 19 22 22 23 24 24 25 26 27 27 30 31 32 3 17 3 18 3 19 3 20 3 21 4 1 4 2 4 3 4 4 4 5 4 6 4 7 4 8 4 9 4 10 4 11 4 12 4 13 4 14 4 15
48. lude Memory Management Unit MMU hardware virtual addressing is not supported The operating system required is therefore is MMU less and uses a flat memory addressing Based on the Nios II processor requirements and the application requirements the uCLinux embedded operating system was chosen to be ported on the Cyclone II FPGA In additions uClinux are considered due to its open source characteristics and availability of Nios II IDE Plug In Table 3 1 Comparison of operating system for the Altera Development Board Company Name OS Nios II IDE Plug In Real Time Type eCosCentric eCos Y Open source Euros Euros RTOS Y Commercial Evidence Erika Enterprise Y Y Commercial Express Logic Threadx Y Y Commercial Mentor Graphics Nucleus Plus Y Commercial Micrium MicroC OS Il 1 Y Y Commercial Microtronix pCLinux Y Open source Segger embos OY Commercial Vector OSCAN 2 Y Commercial 21 The original micro controller Linux uClinux was derived from the Linux 2 0 kernel and intended for microcontrollers like the Nios II processor that do not have a Memory Management Unit 11 The Nios II processor is ideal for running the operating system uClinux was first ported to the Motorola MC68328 DragonBall Integrated Microprocessor The first target system to successfully boot was a PalmPilot device in 1998 Currently uClinux includes kernel releases 2 0 2 4 and 2 6 as well as libraries and t
49. mbentangkan membangunkan sistem prototaip mudah bagi bingkai foto digital menggunakan teknologi FPGA Platform pembangunan yang dikaji adalah Altera DE2 Development Board dan Arrow SoCKit Evaluation Board Integrasi sistem bagi bingkai foto digital projek ini telah dilakukan dengan menggunakan alat Integrasi Osys dalam Ouartus II Pengeluaran gambar untuk gambar bingkai digital dipandu dari penyambung VGA di lembaga pembangunan Satu sistem operasi terbenam akan disesuaikan dan pelabuhan ke dalam lembaga pembangunan untuk bekerja bersama sama dengan FPGA itu Linux kernel ditambah pemandu penimbal bingkai adat untuk menyokong fungsi VGA di bahagian FPGA Sumber terbuka gambar penonton kemudian dipasang di OS terbenam untuk mempercepatkan masa prototaip bingkai foto digital Hasil akhir projek ini menunjukkan bahawa bingkai foto digital mampu beroperasi dalam keadaan yang baik di bawah lembaga pembangunan FPGA dan boleh memaparkan gambar dalam mod pertunjukan slaid pada monitor VGA CHAPTER TABLE OF CONTENTS TITLE DECLARATION DEDICATION ACKNOWLEDGEMENT ABSTRACT ABSTRAK TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES LIST OF ABBREVIATIONS LIST OF APPENDICES INTRODUCTION 1 1 Background 1 2 Problem Statement 1 3 Research Objective 1 4 Scopes of Project 1 5 Outline of Thesis LITERATURE REVIEW 2 1 Altera SOPC system used electronic digital photo frames 2 2 Digital photo frames 2 3 Multi Functional Digital Albums 2 4 A mod
50. nce and easy to used In order to develop an embedded system of digital photo frame on latest Arrow SoCKit Evaluation board latest development tools of QuartusII 13 0sp1 are needed to complete the objective of this project Altera Nios II EDS is a comprehensive development package for Nios II software design It is automatically installed as part of the Quartus II software In this project the Nios II EDS will be used to load the uClinux kernel into DE2 board Altera SoC Embedded Design Suite EDS is a comprehensive tool suite for the embedded software development on Arrow SoCKit Evaluation board It was used to develop the application software part for this project 3 2 2 Altera DE2 Development Board This project will use the FPGA board with Cyclone II EP2C35F672C6 which consists of 35000 logic elements This board consists of 8MB SDRAM USB port VGA output LCD display 8 Hence the image files are obtained through the USB port and decode through the open source picture viewer in the embedded operating system before sending the image data to the frame buffer and display it on VGA monitor screen Figure 3 1 show the Altera DE2 board hardware platform 17 Figure 3 1 Altera DE2 Development board 3 2 3 Arrow SoCKit Evaluation Board Arrow SoCKit Evaluation Board combines the latest Cortex A9 embedded cores with industry leading programmable logic thus give engineers ultimate design flexibility with unparalleled ease 9 Alt
51. nclude a VGA controller ISP1362 or USB controller and necessary memory to hold the program The SDRAM was used to 25 hold the program and embedded operating system The hardware design and kernel image are needed for the application It is preferred if the application begins automatically when the operating system startup The application needs to use the USB flash drive mounting capability and image viewer of the uClinux operating system SOPC buider system development tool in Quartus II 9 1sp1 was used to create 32 bit Nios CPU and automated Avalon switch fabric was used to form the total line to connect the system together with peripheral components USB on DE2 development board use ISP1362 controller chip which is the composition of an OTG controller a host controller and a peripheral controller They are connected with each other through a data bus interface and an external processor The Figure 3 10 shows the hardware design for the digital photo frame application The Nios II softcore processor was set up using the SOPC builder and the components are selected and added as shown in Figure 3 11 Nios II Softcore Processor Avalon Switch Fabric SDRAM ISP1362 USB VGA Controller Controller Controller SDRAM Chip VGA Monitor USB Pendrive Figure 3 10 Hardware design 26 Target Clock Settings Device Family Cyclone Il X Name Source MHz cik 50 External 50 0 cik Externa
52. ool chains Recently work has been done on porting uCLinux to the Nios II processor The uCLinux project community develops patches and supporting tools for using Linux on microcontrollers or embedded processors The documentation is limited with regard to the Nios II processor There is a Nios forum www niosforum org that assists with using uClinux on Nios The kernel of the operating system can be customized for a specific use Prebuilt kernels are available but to decrease memory usage custom kernels are very effective The main features of the uClinux OS that are of great interest are 1 Mount command used to mount access different file systems including USB flash drives Ze NxView Open source picture viewer to decode the image file and display it on a VGA monitor 3 3 3 Implementation of uClinux operating system A Linux operating system is required to install customize and build the distribution of the uClinux Kernel then using the command shell of Nios II to load the uClinux OS into the Nios II processor The hardware design of the Nios processor has to be specified in order to support uClinux During the uClinux build process the Nios hardware system file sopc is used to generate a list of peripheral names addresses and IRQs Many of the device drivers within uClinux use hardcoded names for the peripheral being accessed 22 Thus designing a Nios II system to be compatible with uClinux for Nios II req
53. order to fix into the 8 MB SDRAM SDRAM NXview UCLINUX sg Y Figure 3 4 System Architecture using Altera DE2 Board The image could come directly from the computer via the RS232 communications In order to develop a stand alone embedded system the USB flash drive was decided upon to hold the image to be displayed on the VGA monitor The USB flash drive allows the user to store their favorite photos This embedded system was configured to detect and mount the USB pendrive The use of an embedded operating system can solve the USB driver implementation problem on DE2 board and allow for a quick launch of a completed prototype Embedded uClinux operating system comes with a pre installed open source image viewer called NxView and it 20 will be used to decode the image file from the USB pendrive and display iton a VGA monitor The keyboard will be used as a controller to control the picture frame application This makes the application user friendly During the design stage the JPEG type of image is considered to be displayed on the VGA monitor since it is most common type of image which generated by most devices 3 3 2 Embedded Operating System The operating systems that are compatible with the Altera DE2 board are mostly commercial operating systems OS as seen in Table 3 1 Two types of operating systems that are open source available on the market are eCos and uCLinux 10 Since the Nios II processor does not inc
54. ort fpga led internal led pio external connection out port fpga led internal dipsw pio external connection export user dipsw fpga button pio external connection export user pb fpga hps 0 h2f reset reset n hps fpga reset n alt vip itc 0 clocked video vid clk clk 65 alt vip itc 0 clocked video vid data vid_r vid_g vid_b alt vip itc 0 clocked video underflow clocked video vid datavalid vid datavalid alt vip itc alt vip itc clocked video vid v sync vid v sync alt vip itc clocked video vid h sync vid h sync clocked video vid f clocked video vid h clocked video vid v O alt vip itc alt vip itc ooo ooo alt_vip_itc clock_bridge_65_out_clk_clk clk_65 wire clk_65 wire 7 0 vid r vid g vid b wire vid v sync wire vid h sync wire vid datavalid assign fan ctrl 1 bl assign vga_blank_n assign vga_sync_n assign vga_clk assign vga_b vga_g vga_r assign vga_vs assign vga_hs endmodule 64 1 bl 1 b0 cik 65 vid b vid g vid r vid_v_sync vid_h_sync
55. player for realmedia files Journal of Computing Sciences in Colleges 2008 23 6 133 139 Wang Z Wei Y and Zhong X Embedded design based on the frame buffer digital photo frame Automatic Control and Artificial Intelligence ACAI 2012 International Conference on IET 2012 971 973 APPENDIX A TOP LEVEL DESIGN ENTITY VERILOG CODE A1 Altera DE2 Board module DPF CLOCK 50 SW the button pio LEDR the led red KEY the switch pio UART TXD UART RXD DRAM DO DRAM ADDR DRAM LDOM DRAM UDOM DRAM WE N DRAM_CAS_N DRAM_RAS_N DRAM_CS_N DRAM BA 0 DRAM BA 1 DRAM CLK DRAM CKE SRAM DO SRAM ADDR SRAM UB N SRAM LB N SRAM WEN SRAM CE N SRAM OE N OTG_DATA OTG_ADDR OTG_CS_N OTG_RD_N OTG_WR_N OTG_RST_N OTG FSPEED OTG LSPFED OTG INTO OTG INTI OTG DREOO OTG DREOI OTG DACKO N OTG DACKI N TDI TCK TCS TDO VGA_CLK VGA_HS VGA_VS VGA_BLANK VGA_SYNC VGA R VGA_G VGA B input CLOCK_50 input 3 0 KEY input 17 0 SW output 3 0 LEDR output UART_TXD input UART_RXD inout 15 0 DRAM DQ output 11 0 DRAM ADDR output DRAM LDQM output DRAM_UDOM output DRAM WE N output DRAM CAS N output DRAM RAS N 32 output output output output output inout output output output output output output inout output output output output output output output input input input input output output inp
56. re 2 0 31 0 3 0 3 0 3 0 top config_soc v memory_mem_a memory_mem_ba memory mem ck memory mem ck n memory mem cke memory mem csn memory mem ras n memory mem cas n memory mem we n memory mem reset n memory mem dg memory mem dgs memory mem dgs n memory mem odt memory mem dm memory oct rzgin hps 0 hps io hps io emacl inst TX CLK hps 0 hps io hps io emacl inst TXDO hps 0 hps io hps io emacl inst TXDI 56 input wire hps hps_io_hps_io_spim0_inst_MISO output wire hps 0 hps io hps io emacl inst TXD2 output wire hps 0 hps io hps io emacl inst TXD3 input wire hps 0 hps io hps io emacl inst RXD0 inout wire hps 0 hps io hps io emacl inst MDIO output wire hps 0 hps io hps io emacl inst MDC input wire hps 0 hps io hps io emacl inst RX CTL output wire hps 0 hps io hps io emacl inst TX CTL input wire hps 0 hps io hps io emacl inst RX CLK input wire hps 0 hps io hps io emacl inst RXDI input wire hps 0 hps io hps io emacl inst RXD2 input wire hps 0 hps io hps io emacl inst RXD3 inout wire hps 0 hps io hps io gspi inst IOO inout wire hps 0 hps io hps io gspi inst IOI inout wire hps 0 hps io hps io gspi inst IO2 inout wire hps 0 hps io hps io gspi inst IO3 output wire hps 0 hps io hps io gspi inst SSO output wire hps 0 hps io hps io
57. re design is created and compiled in Quartus The Preloader is based on the Secondary Program Loader SPL which is a component of U Boot The U Boot responsible to loads the operating system and passes software control to the operating system The Figure 3 15 shows an overview of Preloader is generated using the tools provided with SoC EDS Handoff Folder Figure 3 15 Preloader Overview 32 3 4 2 4 Boot the Linux System There are few steps needs to follow in order to succesfully boot the embedded Linux into the Arrow SoCkit board Firstly create Linux booting image file into MicroSD card and then install the MicroSD card in the SoCKit board SoCKit board Bootsel jumpers BOOTSEL 2 0 are set as shown in Figure 3 16 to 101 for booting from MicroSD card Table 3 4 show the HPS BOOTSEL and CLKSEL setting on the SoCKit board Figure 3 16 BOOTSELECT and CLOCKSELECT jumpers setting Table 3 4 HPS BOOTSEL and CLKSEL setting Board Reference Signal Name Setting J15 CLKSELO Short Pin 2 and 3 Logic 0 J16 CLKSEL1 Short Pin 2 and 3 Logic 0 J17 BOOTSELO Short Pin 2 and 3 Logic 0 J18 BOOTSEL2 Short Pin 1 and 2 Logic 1 J19 BOOTSEL1 Short Pin 2 and 3 Logic 0 After that MSEL jumpers MSEL 0 4 are set to 0000 and SW6 are set to 1 to enable the ARM processor to configure the FPGA and enable normal operation of Linux Figure 3 17 show the setting need to follow in order to boot the
58. re module board EEEEEE 7 sesse J sesse J seese J sesos H senen H sesos no f soose f sesos seo g torse a bed Ed Ed Ed Ea e A Basic ru Peripheral Module F5 RER A Figure 2 5 Design Concept a FPGA Core Module Board b Peripheral Module Board and c Integrated Platform This digital photo frame can support several image formats like JPEG Bitmap and PNG In addition music format like MP3 and WMA also supported by digital photo frames Five functions ware implemented on the digital photo frames which is auto play zoom in zoom out next and previous mode Figure 2 6 shows the digital photo frame implemented using modularized FPGA platform Figure 2 6 Implementation of digital photo frame on modularized FPGA platform 2 5 FPGA Field programmable Gate Arrays FPGAs are pre fabricated silicon devices that can be electrically programmed in the field to become any kind of digital circuit or system It can be programmed by the designer after manufacturing and during designing This is also known as on site programmable For low to medium volume productions FPGAs provide cheaper solution and faster time to market as compared to Application Specific Integrated Circuits ASIC which normally require a lot of resources in terms of time and money to obtain a first device 5 The programming of the FPGA is done using Hardware Description Language HDL The programmable logic blocks are called configurable logic blocks and
59. reconfigurable interconnects are called switch boxes Logic blocks can be programmed to perform complex combinational functions or simple logic like AND and XOR In a majority of FPGA s the logic blocks also include memory elements which can be as simple as a flip flop or as complex as complete blocks of memory 10 2 6 FPGA Architecture FPGA architecture are variations of the Figure 2 7 shown however the final architecture depends on the seller Essentially the architecture consists of Configurable I O blocks Programmable Interconnects and Configurable Logic Blocks CLB It also has a clock circuitry to drive the clock signals to each logic block Also other resources like ALU s Decoders and Memory may be available Figure 2 7 shown is an overview of FPGA architecture Configurable logic blocks CLBs are arranged in a two dimensional grid and are interconnected by programmable routing resources whereas I O blocks are arranged at the periphery of the grid and they are also connected to the programmable routing interconnect 5 The programmable reconfigurable term in FPGAs indicates their ability to implement a new function on the chip after its fabrication is complete 3 z rairar ra La i La l kas d L d PAP APA La ml La la mi P1 APA k s ua kas sll Lan ad PAP APA La ml La ml ba ad r APA La ol La ol La al ie tt Figure 2 7 Overview of FPGA architecture 11 2 7 FPGA Design Flow Figure 2 8 shown
60. s increasing day by day But the real world system on chip SoC complexities were lagging behind the capabilities of the silicon hardware 7 Therefore rapid prototyping of embedded systems using FPGA may reduce the complexity gap problems in order to satisfy the current trends in the embedded system market Rapid system prototyping using FPGA technology will allow the researchers or engineers to discover the design errors as early as possible In addition other design alternatives can be explored to give shorter development period Rapid development of 13 prototypes definitely will satisfy the need of time to commercial the embedded system product to the market Microcontrollers microprocessors ASICs and FPGAs can be used to develop the embedded system However the FPGA can be considered as a powerful solution to develop any kind of embedded system such as real time face detection applications due to its reconfigurable characteristic In general the impact of FPGAs occurs not only in the prototyping phase of development but also the final stage of product development The difference between ASICs and FPGAs depends on costs tool availability performance and design flexibility FPGA design is more flexible due to its reconfigurable characteristic and the designer can easily change their design by using the hardware description language HDL Whereas for the ASIC it is not re programmable and thus cannot be changed after a certain poin
61. t in the design process is passed Therefore modifying or change the design of ASICs will need to go through a long process of development Some large and complex embedded system using ASIC technique may take a year or more to complete the design Therefore a good way to shorten development time of an embedded system is to make prototypes using FPGAs at the beginning stages and then switch to an ASIC Prototype of the design can be implemented on FPGA which could be verified for almost accurate results before implement it in an ASIC From the Figure 2 10 shown the design cycle for FPGA is much simpler than ASIC design cycle The simpler design of the FPGA is due to the available software that can handle much of the routing placement and timing Hence manual intervention is less In addition the FPGA design flow also eliminates the complex and time consuming floorplanning verification of 2nd and 3rd order effect timing and analysis Hence developing rapid prototypes using FPGA technology is a good choice as it has an efficient design cycle compare to others 14 FPGA Functional Spec AS C Functional Spec HDL S Simulation DFT Insertion a Equivalen synthesis Checking Floorplanning Hand Off to Fou Place amp Route Wait 1 3 Months Static Timing i Place amp Route Sacra y HDL Z Synthesis in Im Behavioral f gt mb M F Download and Analysis Verify in Circuit gt Equivalency
62. trol panel was utilized to transfer the image file from computer to DE2 board flash memory Image file was then decoded and display on the LCD screen Figure 2 1 Digital photo frame implemented on Altera DE2 70 board and a 4 3 inch Terasic touch screen LCD 2 2 Digital photo frames Figure 2 2 shows the project done by students from Lund University 2 They implement a digital photo frame using a Digilent Nexys 2 FPGA board a VGA monitor on board push button and an SD card reader module plugged into the FPGA board The development environment of Xilinx EDK was used to complete the project The Xilinx Spartan 3E FPGA chip was used and it consists of 500K gates Figure 2 2 Nexys 2 FPGA and a VGA monitor used to implement a digital photo frames The digital photo frames were used a part of image buffer on the VGA controller due to memory size constraints on the Digilent Nexys 2 FPGA SDRAM memory will be used to hold the VGA buffer contents On board buttons was implemented as controller to control the functions of digital photo frame such as resets the entire board start or stop a slide show of images forward and backward the images on the screen Besides C code was implemented in the SD card module to enable the FPGA board to read the image file from the SD card and display it on the VGA monitor Figure 2 3 shows the digital photo frame architecture designed by Lund University s students The software running on the MicroBlaze
63. uires that specific names be used for peripherals in the SOPC Builder Failure to do this will result in the loss of functionality when running the operating system The uClinux kernel image needs to be ported to the embedded processor A Linux kernel image is typically called a zImage The hardware design of the embedded processor and zImage are sent to the FPGA using a JTAG Uart cable and Nios II Command Shell The Figure 3 5 shows the interface to select the desired development board in the Linux OS Figure 3 6 shows the interface to customize the uClinux kernel root pang VirtualBox usr local src nios2 linux uClinux dist Vendor Product Selection Arrow keys navigate the menu lt Enter gt selects submenus gt Highlighted letters are hotkeys Pressing lt Y gt includes lt N gt excludes lt M gt modularizes features Press lt Esc gt lt Esc gt to exit lt gt for Help lt gt for Search Legend built in excluded lt M gt module lt gt Select the Vendor you wish to target Vendor Altera gt Select the Product you wish to target ltera Products nios2 gt lt Select gt lt Help gt Figure 3 5 Platform selection root pang VirtualBox usr local src nios2 linux uClinux dist Kernel Library Defaults Selection Arrow keys navigate the menu lt Enter gt selects submenus gt Highlighted letters are hotkeys Pressing lt Y gt includes lt N gt excludes lt M gt modularizes
64. ularized FPGA Based Embedded System Development Platform 2 3 FPGA 2 6 FPGA Architecture 2 7 FPGA Design Flow vii PAGE ii iii iv vi vii xi xiii xiv V N N Ne m 10 11 2 8 Rapid Prototyping of Embedded Systems using FPGA 2 9 FPGA Design Advantages METHODOLOGY 3 1 Introduction 32 Tools Used 3 2 1 Quartus II and Embedded Design Suite EDS 22 Altera DE2 Development Board 3 2 3 Arrow SoCKit Evaluation Board 3 3 Implementation of system on Altera DE2 Develop ment Board 3 3 1 System architecture 3 3 2 Embedded Operating System 3 3 3 Implementation of uClinux operating system 3 3 4 Hardware system Integration 3 4 Implementation of system on Arrow SoCKit Evaluation Board 3 4 1 System architecture 3 4 2 Implementation of Linaro linux operating system 3 4 2 1 Building Linux kernel 3 4 2 2 Linaro Root File System 3 4 2 3 Preloader and U Boot Cus tomization 3 4 2 4 Boot the Linux System 3 4 3 Hardware system Integration RESULTS AND DISCUSSIONS 4 1 Altera DE2 Development Board Implementation Result 4 2 Arrow SoCKit Evaluation Board Implementation Result CONCLUSION 5 1 Conclusion vili 12 14 15 15 15 16 16 17 19 19 20 21 24 26 26 28 28 30 30 32 34 36 36 41 47 47 5 2 REFERENCES Appendix A Future Works and Recommendations 1X 47 49 31 LIST OF TABLES TABLE NO TITLE PAGE 3 1 Comparison of operating system for the Altera Deve
65. ule Freeing unused kernel memory 620k freed xa24608 BxaheBBB gt Shell invoked to run file etc rc hostname uClinux mount t proc proc proc o noexec nosuid nodev mount t sysfs sysfs sys o noexec nosuid nodev mount t devpts devpts dev pts o noexec nosuid mount t usbfs none proc bus usb mkdir var tmp mkdir var log mkdir var run mkdir var lock mkdir var empty ifconfig lo 127 0 0 1 route add net 127 8 8 8 netmask 255 0 0 0 lo cat etc motd Welcome to For further information check http www uclinux org Execution Finished Exiting Figure 4 3 uClinux Operating System eth dm9BB0Ba at 806810Gf8 886810fc IRQ 6 MAC 80 080 08 80 80 88 chip isp1362 hed isp1362 hed new USB bus registered assigned bus number 1 38 Kernel file was customized to supports a VGA monitor USB devices USB storage mass keyboard in the DE2 board Figure below show the USB stor age mass was succesfully recognized under uClinux operating system When the USB pendrive was inserted into the USB port of the DE2 board the uClinux will automatic recognised it and display the details on the terminal Figure 4 4 show the USB mass storage detection on uClinux OS Figure 4 5 shows the USB mass storage was successfully mounted under uClinux operating system After successfully mounted the USB mass storage we can list out the contents in the mass storage by using Unix command of 1s
66. ut input input output output output output output output output output output 15 0 17 0 15 0 1 0 9 0 9 0 9 0 53 DRAM CS_N DRAM BA 0 DRAM BA 1 DRAM CLK DRAM CKE SRAM DO SRAM ADDR SRAM UB N SRAM LB N SRAM WE N SRAM CE N SRAM_OE_N OTG DATA OTG ADDR OTG CS N OTG RD N OTG WR N OTG RST N OTG FSPEED OTG LSPEED OTG INTO OTG INTI OTG DREOO OTG DREOI OTG DACKO N OTG DACKI N TDI TCK TCS TDO VGA_CLK VGA_HS VGA_VS VGA BLANK VGA SYNC VGA R VGA G VGA B 54 assign OTG_FSPEED 1 bz assign OTG_LSPEED 1 bz wire CPU_CLK wire CLK_18 4 wire CLK_25 SDRAM PLL PLLI inclk0 CLOCK 50 c0 DRAM CLK cl CPU CLK c2 CLK 25 SocDPF SocDPF_inst clk 0 CLOCK 50 clk CPU CLK VGA BLANK from the vga controller 0 VGA BLANK NGA B from the vga controller 0 VGAB VGA CLK from the vga controller 0 VGA CIK NGA G from the vga controller 0 VGAG NGA HS from the vga controller 0 VGA HS NGA R from the vga controller 0 VGAR VGA SYNC from the vga controller 0 VGA SYNC NGA VS from the vga controller 0 VGA NVS OTG ADDR from the ISP1362 OTG ADDR OTG CS N from the ISP1362 OTG CS N from OTG DACKO N from the ISP1362 OTG DACKO N OTG DACKI N from the ISP1362 OTG DACKI N OTG DATA to and from the ISP1362 OTG DATA OTG FSPEED from the I

digital photo frame implementation on fpga system

Contents

Download Pdf Manuals

Related Search

Related Contents