Home

Xilinx XAPP794 1080P60 Camera Image Processing

1. Export Design Logicto Memory Interconnect Simulation Flow ff xs ahon 5 AMEA Connection Legand Programmable Logic PL Generate HDL Files r DD Arrow direction shows control Dats flows both directions om Configurable AXIS 32 bivG4 bit AXIS 64 bit AXIS 32 bit AHB 32 bit APB 32 bit Search IP Catalog Clear Launch Simulator amp Project IP Catalog E Design Summary x Graphical Design View x System Assembly View x Sees Console aoe a gWARNING E0K 1954 No Top level Input clock for the design a WARNING EDK 1954 No Top level Input clock for the design E pWARNING EOK 1954 No Top level Input clock for the design J a I I console i Warnings Errors X794_51_102612 Figure 51 Hardware Project Opened from ISE XPS XAPP794 v1 2 January 2 2013 www xilinx com 44 Rebuilding the Hardware Platform XILINX Exporting the Base Hardware Platform to SDK Because the base hardware platform is used with the Software Development Kit SDk information about the hardware platform must be provided to the SDK to allow development of software platforms and applications 1 Import a predefined ZC702 evaluation board specific configuration template to create a Zynq device configuration This template guides the generation of MIO initialization code Click the left pointing red arrow just below the Bus Interfaces tab Figure 52 Xilinx Platform studio EDK_P
2. If the host computer cannot establish a connection it might be necessary to disable virus X794_17_102612 Figure 17 1080p60 Camera Design Web based GUI scanning the firewall or both This is not typically required To test whether the computer can communicate over the Ethernet connection open a Command Prompt window this is done in Windows 7 by selecting Start Button gt All Programs gt Accessories gt Command Prompt and type ping 192 168 1 10 or ping followed by the IP address assigned to the ZVIK in the previous section If this is not successful review the IP configurations set in Preparing the SD Card and Configuring the Host IP Address XAPP794 v1 2 January 2 2013 www xilinx com 20 Running the Demonstration XILINX Adjusting the Image Sensor Gain and Exposure The image sensor gain and exposure can be adjusted using the controls shown in Figure 18 Image Sensor Settings Processor Control Auto Gain Auto Exposure E Target Intensity 200 Image Sensor Auto Gain Exposure C Target Intensity 184 Exposure THY 90 Analog Gain 0 Digital Gain 10 X794_18_102612 Figure 18 Image Sensor Gain and Exposure Controls The check boxes shown in Figure 18 allow the gain and exposure to be adjusted automatically by either of these two methods e Processor Control One of the Zyng 7000 SoC processors collects information about the image from the Image Statistics IP core and uses this infor
3. Sound video and game controllers Storage volumes 1 System devices Universal Serial Bus controllers H cS co a a X794_14_102612 Figure 14 Determining COM Port in Windows In the example illustrated in Figure 14 the COM port is COM4 This can be different for each computer XAPP794 v1 2 January 2 2013 www xilinx com 13 Running the Demonstration g XILINX To restart the boot process press the POR_B button SW1 located close to the SD card connector or power cycle the ZC702 board This boot sequence should be observed The ZC702 board is powered on The DONE LED is off The first stage bootloader takes approximately 20 seconds Camera design hardware is loaded into programmable logic The DONE LED turns on U Boot takes approximately 30 seconds Linux Kernel boot takes approximately 15 seconds ONO aAP OND Camera design software is executed this takes approximately 5 seconds The total boot sequence should take approximately 70 seconds just over one minute U Boot generates this output on the serial console U Boot 2010 09 01918 g068cc03 Jan 20 2012 14 02 15 Xilinx Pele Emulation Platform DRAM 256 MiB Unknown FLASH on Bank 1 Size 0x00000000 O MB Flash 0 Bytes MMC SDHCI 0 Using default environment In serial Out serial Err serial Hit any key to stop autoboot 0 Copying Linux from SD to RAM Mounting SD card to mnt Configuring IP Addr
4. This read write acceptance and issuance setting imposes a limitation on getting minimum acceptable bandwidth for every master in a multi master system The optimum setting of issuance and acceptance reduces throttle on the bus and compensates for long latencies The AXI register interface is clocked at 75 MHz The Zynq 7000 AP SoC PS GPO port acts as the master on this interconnect and connected slaves have register maps AXI TPG and AXI VTC are examples of slaves connected to this interconnect The operations of the video www xilinx com 34 Hardware Platform AXI AXI VTC DPC XILINX pipeline are controlled by registers inside every IP core Depending upon the data flow required in the video pipeline the processor writes these registers through the AXI4 Lite interconnect The AX14 Lite interconnect accepts write or read transfers from the processor performs address decoding selects a particular slave and establishes a communication channel between the processor and the slave device For detailed information about the complete feature set and a functional description of the AXI interconnect IP refer to LogiCORE IP AXI Interconnect Ref 5 Video System Configuration AXI VDMA AXI VDMA has an AX14 Stream interface on one side and an AXI memory mapped interface on the other side The AXI VDMA has two channels MM2S and S2MM The MM2S channel is not implemented in this design The S2MM channel receives data from the master de
5. www xilinx com zc702 For more information on the Avnet FMC IMAGEON FMC Module with ON Semiconductor image sensor refer to http Awww em avnet com fmc imageon v2000c XAPP794 v1 2 January 2 2013 www xilinx com 3 Running the Demonstration Running the Demonstration XAPP794 v1 2 January 2 2013 XILINX This section describes how to run the 1080p60 camera image processing reference design on the ZVIK Reference Design File The reference design files for this application note can be downloaded from https secure xilinx com webreg clickthrough do cid 199792 Table 1 shows the reference design matrix Table 1 Reference Design Matrix Parameter Description General Developer name Xilinx Target devices Zynq 7000 AP SoC Source code provided Yes Source code format VHDL some sources encrypted IP used EDK and CORE Generator Video IP Cores Simulation Functional simulation performed N A Timing simulation performed N A Test bench used for functional and timing N A simulations Test bench format N A Simulator software version used N A SPICE IBIS simulations N A Implementation Synthesis software tools version used XST 14 2 Implementation software tools version used ISE Design Suite 14 2 System Edition Static timing analysis performed Yes pass timing in PAR TRACE Hardware Verification Hardware verified Yes H
6. 3 When the SDK tool opens a welcome screen is displayed Close the welcome screen after browsing through the displayed information When the welcome screen is closed the Project Explorer tab is displayed as empty To import the Linux applications into the SDK workspace 1 Inthe SDK tool select File gt Import 2 Inthe import wizard expand the General folder and select Existing Projects into Workspace Click Next 3 To specify the project sources root directory in the Select Root directory click Browse and select zvik_camera sw 4 After the root directory is specified the import wizard shows a list of projects available to import as shown in Figure 54 Make these selections e Ensure that the linux_app project is selected e Select the Copy projects into workspace checkbox to make a local copy of the projects in the workspace when the import is done Note The projects in the import wizard might appear in a different order than that shown in Figure 54 ct gl ee Import Projects YY Select a directory to search for existing Eclipse projects gt Select root directory C zvik_camera sw Browse Select archive file Projects 7 zvik_camera_linux_app C Users chrisar Desktop zynq zvik_camr Select All 7 zvik_camera_linux_webserver C Users chrisar Desktop zynq zvik Deselect All Refresh m r V Copy projects into workspace Working sets F Add proj
7. All Programs gt Xilinx Design Tools gt ISE Design Suite 14 2 gt EDK gt Xilinx Platform Studio e Ona Linux host enter xps at a command prompt If necessary close out the previous project by selecting File gt Close Project In Project Commands select Open Project Browse to zvik_camera hw and select system xmp Click Open Figure 51 akay Xilinx Platform Studio EDK _P 7xd proj emb_apps sudhakar embedded zynq xlinux zc702 hw_systems fsb _O1 PCW systems zC702 system xmp system Assembly View Eile Edit View Project Hardware Device Configuration Debug Simulation Window Help amp A e BB ett Gae Book ee SDA Zynq Businterfaces Ports Addresses P b Navigator X E cataog E PACKE Description Baas EDK Install w Analog Bus and Bridge pun pacs Clock Reset and Interrupt Communication High Speed Communication Low Speed OMA and Timer Debug FPGA Reconfiguration General Purpose IO Ra Generate Netlist 4 Interprocessor Communication Memory and Memory Controller 1o Peripheral Controller f neto Coresigniri Generate Bitstream Processor USER Utility Verification Project Local PCores NEON FPU Engine Cortex Aa mMPCore cPpu NEON FPU Engine Cortox A9 MPCore cPu MMU MMU 32KBI i Cache sic 32KBD Cache s2KBI Cache 32KBO Cache Programmable
8. 7xd proj emb_apps sudhal ar smbedded zynq xlinux zc702 hw_systems fsb _O1 PCW systems ZC702 system xmp system Assembly View m x file Edit yiew project Hardware Device Configuration Debug simula Window Help Cee Ga BOHRA ig BOBit ct Navigator X E catagan pnn SOSA Zynq Bus interfaces Ports Addresses IOI Description IP Ve EDK Install Ki ji aa rier Application Processor Unit APU Run DRCS Es snor NEON FPUEngIne g Clock Reset and Interrupt g NEON FPU Engine Communication High Speed xis Cortex A9 Cortoz A9 4 Communication Low Speed a k ee Lay G OMA and Timer 32KB 32KB D 32KB 1 32KB D fal Debug Cache Cache Cache Cache oO FPGA Reconfiguration Snoop Control Unit Generate Netlist General Purpose IO 12KG L2 Cache amp Controller Interprocessor Communication x Memory and Memory Controller y gt ji lie Peripheral Controller ee Generate BitStroam S Processor USER 4 Utility Verification C Export Design Project Local PCores Interconnect Simulation Flow ff n EMIO yA f rammal Cenaa POL Files or 4MBA Connection Legend s Prog ble Logic PL gli gt 1266 Arrow dire ction shows control Dats flows both directions AL bpa Configurable AXIS 32 bIvG4 bit PCH I3 64 BIE AXIS 32 bit AHB 32 bit APB 32 bit le Search IP Catalog Clear Launch Simulator
9. Detector Initialization Video Detector Configuration Image Processing Pipeline iPIPE Initialization Initializing iPipe cores CCM done Gamma done CFA done Stats done Noise done Enhance done Initializing iPipe cores done Configure ZC702 IIC Mux for Port 1 HDMI ZC702 HDMI Output Initialization web avnet console IN tmp zvik_camera_linux_pipe_req OUT tmp zvik_camera_linux_pipe_rsp access PIPE_IN_NAME F_OK done access PIPE _OUT_NAME F_OK done open PIPE_IN_NAME O_RDONLY O_NONBLOCK done web_session_handler started XAPP794 v1 2 January 2 2013 www xilinx com 15 Running the Demonstration amp XILINX Xilinx Zynq 7000 AP SoC Video and Imaging Kit 1080P60 Real Time Camera Demonstration General Commands help Print the Top Level menu Help Screen quit Exit console if applicable verbose Toggle verbosity on off delay Wait for specified delay mem Memory accesses I2C Commands iic0 IIC accesses on FMC IPMI I2C chain iic1 IIC accesses on FMC IMAGEON I2C chain VITA Commands vita VITA commands init status vspi SPI accesses to VITA sensor vreg Memory accesses to VITA receiver again Analog gain 0 10 dgain Digital gain 0 4095 where 128 corresponds to 1 00 exposure Exposure time 1 99 in percentage of frame period 16 66 msec iPIPE Commands dpc Defect Pixel Correction configuration cfa Color Filter Array Interpolation configurati
10. Hardware The block diagram for the 1080p60 camera reference design is shown in Figure 33 Platform L SA HPx Processing System DDR Memory Controller AMBA Switches Hardened Peripherals APU USB GigE Dual Core CAN SPI Cortex A9 OCM UART 12C GPIO M_AXI4_GP AXI4 Stream AMBA Switches AXI Lite Interconnect VITA 2000 VITA On board Camera In Receiver HDMI Out X794_33_111912 Figure 33 1080p Camera Hardware Block Diagram The details of the image processing pipeline are shown in Figure 34 AXI AXI AXI RGB AXI AXI YUV AXI AXI AXI VTC DPC CFA YUV Noise Enhance RGB CCM Gamma VTC AXI STATS X794_34_102612 Figure 34 1080p Camera Image Processing Pipeline XAPP794 v1 2 January 2 2013 www xilinx com 31 Hardware Platform XILINX This design is implemented in a Zynq 7000 AP SoC device KC7Z020CLG484 1 using the ISE Design Suite Embedded Edition 14 2 The PL hardware utilization for the implemented design is shown in Table 9 Table 9 Hardware Utilization FPGA Components Total Available Used Used I Os 200 50 25 LUTs 53 200 29 708 55 Registers 106 400 36 255 34 DSP48s 220 54 24 Block RAM RAMB36E1 FIFO36E1s 140 48 34 RAMB18E1 FIFO18E1s 280 26 9 A more detailed resource utilization summary can be found in this directory zZvik_camera doc system_summary html Note Device resource utilization results are dependent on the implementation tool ver
11. Terminals Right click and select Launch Terminal The zvik_camera_linux_app application can be debugged remotely with the SDK tool by following these steps 1 oF oO N 10 XAPP794 v1 2 January 2 2013 Switch to the C C perspective by selecting Window gt Open Perspective gt Other then select C C Default Select the zvik_camera_1linux_app application Right click and select Debug As Select Debug Configurations In the Debug Configurations dialog box select Remote ARM Linux Application then click the new launch configuration button as shown in Figure 56 i OLES X794_56_102612 Figure 56 New Launch Configuration Button In the Connection list select the ZVIK IP address i e 192 168 1 10 In the Remote Absolute File Path for C C Application field specify the path tmp zvik_camera_linux_app elf Click Apply Click Debug The application is downloaded to the specified remote path and the Debug perspective opens Click Yes when asked to open the debug perspective www xilinx com 49 Rebuilding the Software Applications XILINX 11 In the Console tab click the verbose console mode button as shown in Figure 57 to disable the verbose console mode mw x amp Ee Ab 30 stack list arguments 0 0 0 30 done stack args frame level 0 args gdb 31 stack list locals 0 31 done locals gdb p X794_57_102612 Figure 57 Verbose C
12. The best image quality is usually achieved with a value of 1 m Edge Enhancement Strength p X794_23_102612 Figure 23 Image Edge Enhancement IP Core Control Table 6 Edge Enhancement Settings Setting Description 0 Edge Enhancement OFF Edge Enhancement ON 1 4 Number identifies strength of edge enhancement XAPP794 v1 2 January 2 2013 www xilinx com 24 Running the Demonstration g XILINX Configuring the Color Correction Matrix The Xilinx Color Correction Matrix IP core provides color correction of the image to correct for different illumination sources The Color Correction IP core settings can be adjusted using the GUI controls shown in Figure 24 m Color Correction Matrix m Processor Control Auto White Balance O White Balance Daylight H Brightness p ioo CRN b Saturation 100 X794_24_102612 Figure 24 Color Correction Matrix IP Core Controls The white balance list box applies pre determined color corrections for four illumination conditions as shown in Table 7 Table 7 White Balance Settings Setting Description Bypass Color correction OFF all coefficients are 1 0 Daylight Color correction ON for daylight lighting conditions Cool White Fluorescent Color correction ON for cool white fluorescent 4500 K lighting U30 Hot Fluorescent Color correction ON for hot fluorescent 3000 K lighting Incandescent Color correction ON for in
13. These are defective pixels in the image sensor As the Pixel Age Slider is moved back to 0 the defective pixels are effectively removed from the image by the Defective Pixel IP core processing XAPP794 v1 2 January 2 2013 www xilinx com 23 Running the Demonstration g XILINX Configuring the Color Filter Array Interpolation The Xilinx Color Filter Array Interpolation IP core Bayer Phase setting Figure 21 can be changed but only one setting produces a valid result with the VITA 2000 image sensor m Color Filter Array Interpolation Bayer Phase Green Blue 2 gt X794_21_ 102612 Figure 21 Color Filter Array IP Core Control Configuring the Spatial Noise Reduction The Xilinx Image Noise Reduction IP core performs spatial noise reduction using a low pass filter that is edge adaptive The Strength slider shown in Figure 22 allows the settings shown in Table 5 m Spatial Noise Reduction Strength p X794_22_102612 Figure 22 Image Noise Reduction IP Core Control Table 5 Spatial Noise Reduction Strength Settings Setting Description 0 Spatial Noise Reduction OFF 1 4 Spatial Noise Reduction ON Number identifies strength of spatial noise reduction filter Configuring the Edge Enhancement The Xilinx Image Edge Enhancement IP core performs edge detection and highlights the edges in the color image The Strength slider shown in Figure 23 allows the settings shown in Table 6
14. amp XILINX XAPP794 v1 2 January 2 2013 Summary Introduction Application Note Zynq 7000 All Programmable SoC Video and Imaging Kit 1080p60 Camera Image Processing Reference Design Authors Mario Bergeron Avnet Inc Steve Elzinga Gabor Szedo Greg Jewett and Tom Hill Xilinx Inc The Xilinx Zynq 7000 All Programmable AP SoC Video and Imaging Kit ZVIK builds on the Zyngq 7000 AP SoC ZC702 evaluation kit ZC702 Ref 43 by including additional hardware software and IP components for the development of custom video applications The included video reference designs WUXGA color image sensor and video I O FPGA mezzanine card FMC with HDMI input and output enable users to immediately start development of video system software firmware and hardware designs This application note describes how to set up and run the 1080p60 camera image processing reference design camera design using the ZVIK Instructions are also included on how to build the hardware and software components as well as how to create the SD card boot image The intended audience for this document includes video applications embedded system developers hardware developers and system architects To learn more about the Zynq 7000 AP SoC the ZVIK or for further development using the embedded design kit consult the references mentioned in References The Appendix provides a list of acronyms used in this application note This application no
15. conditions of the applicable License 7 the programmed device after operating for some period of time This allows you to evaluz of the Licensee of this core to adhere to terms and conditions of the applicable License 7 X794_53_112912 Figure 53 Exporting the Hardware Design to SDK Rebuilding the Extracting the Application Archives Software In the zvik_camera sw directory extract the archives shown in Table 12 Applications Table 12 Application Archives Archive Name Description zvik_camera_linux_app This is the main application for the 1080p60 camera design zvik_camera_linux_webserver This is the CGI application that services requests from the web based GUI Importing and Building the Linux Applications The Xilinx SDK included in the ISE Design Suite version 14 2 or later can be used to build Linux applications To create a new SDK workspace to import and build Linux applications 1 Launch the SDK tool In Windows select Start gt All Programs gt Xilinx Design Tools gt Xilinx ISE Design Suite gt EDK gt Xilinx Software Development Kit On a Linux host enter xsdk at a command prompt XAPP794 v1 2 January 2 2013 www xilinx com 46 Rebuilding the Software Applications g XILINX 2 When the Workspace Launcher appears create a new directory as zvik_camera sw SDK_Linux_Workspace Specify this directory as the new workspace and click OK in the Workspace Launcher
16. http Avww xilinx com products intellectual property EF DI CFA htm Image Statistics Engine The AXI STATS core is used to gather various image statistics from the image processing pipeline Figure 40 such as e Intensity histograms e Color histograms X794_40_102612 Figure 40 AXI Stats Core Pipeline Position For additional information about the Image Statistics Engine LogiCORE solution and the detailed product guide refer to the Image Statistics Engine product page on Xilinx com http www xilinx com products intellectual property EF DI IMG STATS htm Color Space Conversion The color space conversion cores are used in the image processing pipeline to convert to from the RGB color space and the YCrCb color space as shown in Figure 41 AXI RGB AXI AXI YUV AXI AXI AX CFA YUV Noise Enhance RGB CCM Gamma VTC X794_41_102612 Figure 41 Color Space Conversion Cores Pipeline Position AXI AXI VTC DPC XAPP794 v1 2 January 2 2013 www xilinx com 37 Hardware Platform AXI VTC AXI VTC AXI DPC AXI DPC XILINX For additional information about the Color Space Conversion LogiCORE solutions and the detailed product guides refer to the Color Space Conversion product pages on Xilinx com http www xilinx com products intellectual property RGB_to_YCrCb htm http www xilinx com products intellectual property YCrCb_to_RGB htm Image Noise Reduction The AXI NOISE core is used to remove
17. internal external manual iPIPE Commands dpc Defect Pixel Correction configuration cfa Color Filter Array Interpolation configuration stats s Image Statistics awb Auto White Balance on off age Auto Gain Control on off XAPP794 v1 2 January 2 2013 www xilinx com 18 Running the Demonstration g XILINX aec Auto Exposure Control on off geq Gamma Equalization on off irg i Image Statistics Interrupt manual noise Noise Reduction configuration enhance Image Enhance configuration ccm Color Correction Matrix configuration gamma Gamma Correction configuration Video Source Selection video Video Initialization hdmi hdmii hdmio vita Record Playback Commands rec Save frame buffer image to BMP file play Fill frame buffer image from BMP file zvik_camera gt Type the command help command to obtain detailed help for a specific command For example to save an image from the video frame buffer to the SD card mnt type these commands zvik_camera gt rec help rec help Syntax rec filename gt Write frame buffer image to BMP file zvik_camera gt rec mnt mypicture bmp rec mnt mypicture bmp height 1080 width 1920 n 6220800 zvik_camera gt XAPP794 v1 2 January 2 2013 www xilinx com 19 Running the Demonstration XILINX Using the Web based GUI To access the web based GUI open a web browser such as Internet Explorer and enter the ZVIK IP address http 192 168 1 10 or the ad
18. the video processing pipeline and allows design configuration via the USB serial connection or via the network with a web based GUI The block diagram in Figure 46 illustrates the general architecture of the Linux application To From Web Server STDIN STDOUT Named Pipes video_ipipe fmc_imageon vita_receiver zvik_camera_linux_app elf X794_46_102612 Figure 46 1080p60 Camera Linux Application General Architecture XAPP794 v1 2 January 2 2013 www xilinx com 39 Software Platform XILINX The application has three tasks 1 Main Task main c 2 Image Statistics Task vipp_stats_handler 3 Web Session Task Each of these tasks are described in subsequent sections The block diagram in Figure 47 shows the source files that make up the 1080p60 camera application Local Source Files web_session_handler task Avnet Provided Software Libraries main c task fmc_imageon_demo c vipp_stats_handler task enhance cm Ci X794_47_102612 Figure 47 1080p60 Camera Linux Application Source Files The fmc_imageon_demo c file contains most of the top level initialization code for the camera design The avnet_console_ c files which implement a text based command interface are optional and can be removed from the application This command interface is used by the main task to provide user control via the STDIN and STDOUT pipes The command interface is also used by the web ses
19. 00 Camera Assembly Step 6 The VITA 2000 color camera assembly is complete XAPP794 v1 2 January 2 2013 www xilinx com 9 Running the Demonstration XILINX 7 The IMAGEON FMC Module connects to the FMC2 connector of the ZC702 board FMC carrier The flexibility of the LCEDI cable allows the camera to be positioned in virtually any direction Figure 10 Figure 10 VITA 2000 Camera Assembly Step 7 XAPP794 v1 2 January 2 2013 www xilinx com 10 Running the Demonstration g XILINX 8 The ZVIK package also contains two standoffs and four screws that secure the IMAGEON FMC module to the ZC702 board The package also contains four longer screws standoffs and rubber feet to support the free end of the board Assemble the hardware as shown in Figure 11 X794_11_102612 Figure 11 VITA 2000 Camera Assembly Step 8 Setting Up the Hardware Figure 12 illustrates how to connect the ZVIK for the 1080p60 camera design D VITA 2000 Camera FMC Adapt r Firmware on SD Card La Ethernet HHA 1080P60 HDMI Monitor Web based GUI X794_12_103112 Figure 12 1080p60 Camera Design Hardware Setup XAPP794 v1 2 January 2 2013 www xilinx com 11 Running the Demonstration XILINX Connect the ZVIK hardware as follows 1 2 3 Position the Avnet FMC IMAGEON board on FMC slot 2 of the ZC702 board Connec
20. 2013 www xilinx com 1 Introduction XILINX Processing System Firmware on SD Card DDR Memory Controller w S_AXI4_HPx Lf M_AXI4_GP AMBA Switches gt AXI4 Stream Hardened APU Peripherals PC running USB GigE Dual Core Web based GUI CAN SPI Cortex A9 OCM UART 12C GPIO AMBA Switches Image Camera Input Processing Pipeline VITA 2000 Camera Programmable Logic HDMI Monitor X794_01_102512 Figure 1 1080p60 Camera Design Block Diagram A web based graphical user interface GUI allows configuring each of the Xilinx video IP cores in the image processing pipeline displaying information about the incoming image such as histograms of the data and enables processor based operations on the data such as automatic white balance and automatic exposure The hardware evaluation cores contained in the design time out after approximately four hours resulting in a blank screen At this point the board must be power cycled to reload the design XAPP794 v1 2 January 2 2013 www xilinx com Introduction XILINX Host PC System Requirements The host PC requirements to operate the camera design and its applications are e 32 bit 64 bit host PC with Ethernet port running Windows XP or Windows 7 Professional 32 bit 64 bit or Ubuntu 10 or later 32 bit 64 bit Linux distribution e UART connected terminal for example Tera T
21. 702 Evaluation Kit documentation page http Avww xilinx com support documentation zc702 htm Xcell Journal Issue 81 Fourth Quarter 2012 http www xilinx com publications archives xcell Xcell81 pdf Oct12NL UG585 Zynq 7000 All Programmable SoC Technical Reference Manual DS406 LogiCORE IP Processor System Reset Module Product Specification DS768 LogiCORE IP AXI Interconnect Xilinx Open Source ARM Git Repository http git xilinx com UG798 Xilinx Design Tools Installation and Licensing Guide Xilinx ARM GNU Tools http wiki xilinx com zyng tools Using Git http wiki xilinx com using git UG821 Zyng 7000 All Programmable SoC Software Developers Guide Xilinx Zyngq 7000 All Programmable SoC website http www xilinx com products silicon devices soc zyng 7000 Zynq 7000 All Programmable SoC Product Table http www xilinx com publications prod_mktg zynq7000 Zyng 7000 combined product tab le pdf DS190 Zynq 7000 All Programmable SoC Overview git The fast version control system home page http git scm com Zynq Linux Downloading the Kernel Tree http xilinx wikidot com zyng linux toc7 Zynq Linux Configuring and Building the Linux Kernel http xilinx wikidot com zynqg linux toc8 Xilinx Open Source Linux http wiki xilinx com open source linux Xilinx Device Tree Generator http xilinx wikidot com device tree generator Device Tree general information http devicetree org Ma
22. AXI DPC Core Pipeline Position This core is responsible for correcting defective pixels as illustrated in Figure 37 X794_37_102612 Figure 37 Defective Pixel Correction For additional information about the Defective Pixel Correction LogiCORE solution and the detailed product guide refer to the Defective Pixel Correction product page on Xilinx com http Awww xilinx com products intellectual property EF DI DEF PIX CORR htm Color Filter Array Interpolation The AXI Color Filter Array CFA core is the second processing element in the image processing pipeline as shown in Figure 38 AXI RGB AXI AXI YUV AXI AXI AXI CFA YUV Noise Enhance RGB CCM Gamma VTC X794_38_102612 3 AXI AXI Figure 38 AXI CFA Core Pipeline Position XAPP794 v1 2 January 2 2013 www xilinx com 36 Hardware Platform XILINX The most widespread and cost effective implementation of color image sensors involves placing color filters on top of each pixel in a Bayer pattern arrangement as shown in Figure 39 X794_39_102612 Figure 39 Bayer Pattern Color Filter With this arrangement of color filters each pixel captures only one of the three primary colors The CFA core restores the missing two colors based on neighboring pixels For additional information about the Color Filter Array Interpolation LogiCORE solution and the detailed product guide refer to the Color Filter Array Interpolation product page on Xilinx com
23. Project IP Catalo ip Design Summary x Graphical Design View x System Assembly View x L oj g y y j Console CEC E3 pWARNENG EOK 1954 No Top level Input clock for the design a RWARNING EDK 1954 No Top level Input clock for the design ie QWARNENG EDK 1954 No Top Level Input clock Tor the design E ts Ki m J i consore warings JO Erors Figure 52 Importing Board Specific MIO Initialization Code X794_52_102612 2 Click OK after choosing the ZC702 evaluation board configuration file then click YES to import the xml based configuration file XAPP794 v1 2 January 2 2013 www xilinx com 45 Rebuilding the Software Applications g XILINX 3 Under Implementation Flow click Export Design check Include bitstream and BMM file and click Export Only as shown in Figure 53 The hardware description files along with Zynq device initialization C source code for the chosen evaluation board is stored at this location zvik_camera hw SDK SDK_Export hw PELI Navigator Deag Fom B Generate Ne st ps Generate BES tean Export Design Sexdation Flow Gomer ate VOR Flies Launch Smisao LA rr Cag QA P Catalog DEX ima tuimerteces forts Addresses bheccan a e x x Descnption IP Vers h wal s Em P_ Import Export Surnmary yee Processing System PS Aethmetic Dus and Bridge ssid Reset Application Processor Uni
24. RD Updated Figure 53 page 46 and Figure 54 page 47 Updated the path for the hardware description files and Zynq device initialization C source code in Exporting the Base Hardware Platform to SDK page 45 Added path for system bit file in Preparing the SD Card Boot Image page 52 01 02 13 1 2 Added Reference Design File Updated Figure 27 Figure 28 Figure 30 Figure 31 and Figure 32 The information disclosed to you hereunder the Materials is provided solely for the selection and use of Xilinx products To the maximum extent permitted by applicable law 1 Materials are made available AS IS and with all faults Xilinx hereby DISCLAIMS ALL WARRANTIES AND CONDITIONS EXPRESS IMPLIED OR STATUTORY INCLUDING BUT NOT LIMITED TO WARRANTIES OF MERCHANTABILITY NON INFRINGEMENT OR FITNESS FOR ANY PARTICULAR PURPOSE and 2 Xilinx shall not be liable whether in contract or tort including negligence or under any other theory of liability for any loss or damage of any kind or nature related to arising under or in connection with the Materials including your use of the Materials including for any direct indirect special incidental or consequential loss or damage including loss of data profits goodwill or any type of loss or damage suffered as a result of any action brought by a third party even if such damage or loss was reasonably foreseeable or Xilinx had been advised of the possibility of the same Xi
25. application via a named pipe The inverse of this process occurs for the response of each request or command Rebuilding the Licensing the Video and Image Processing Pack IP Cores Hardware The image processing pipeline iPipe reference design uses several of the Xilinx Video and Platform Image Processing Pack IP cores that must be licensed prior to use Follow these steps to request an evaluation license 1 Got to this website http Awww xilinx com products intellectual property EF DI VID IMG IP PACK htm 2 Click the Evaluate link located on the upper left of the web page as shown in Figure 50 and follow the on line instructions Video and Image Processing Pack Product information Resources FAQ XILINX n High quality video and image processing IP cores to enable faster time to The Xilinx Video and Image Processing Pack provides a low cost bundled licensing option for all of the LogiCORE IP blocks listed in the key features section Video X794_50_102612 Figure 50 Xilinx Video and Image Processing Pack Evaluation License Example XAPP794 v1 2 January 2 2013 www xilinx com 43 Rebuilding the Hardware Platform amp XILINX 3 The generated license file is sent via email Follow the enclosed instructions to add the evaluation license features for the Video and Image Processing Pack Open and Build the Hardware Design Bitstream 1 Start the ISE tools e Ona Windows host select Start gt
26. ardware platform used for verification Zynq 7000 Video and Imaging Kit Installing Design Files Download the ZVIK_Camera_Design_14_2 files to the C drive of the host PC C zvik_camera Note f another location is chosen there should be no spaces in the folder names www xilinx com Running the Demonstration g XILINX Preparing the SD Card Pre built binaries for the camera design are provided in this directory zvik_camera binaries sd_content Create a backup copy of the files on the SD card provided with the kit to enable them to be restored if desired These files are also available on the ZVIK product page Copy the contents of the sd_content directory to the root directory of the SD card By default the design configures the ZVIK for IP address 192 168 1 10 This requires configuring the host computer to a compatible IP address such as 192 168 1 20 If this is not possible the IP address of the ZVIK in the configuration script should be changed Zvik_camera binaries sd_content config_my_ip sh On a Windows host use a text editor that does not alter the end of line characters such as Notepad 6 config_my_ip sh Notepad File Edit Format View Help bin shifconfig eth0 192 168 1 10 netmask 255 255 255 0 X794_02_111512 Figure 2 Modifying the ZVIK IP Address With a subnet mask of 255 255 255 0 the first three number groups of the IP address must be identical in both t
27. ation XILINX To view this serial output open a terminal window using the UART connection program Terra Term or Hyperterminal with these settings 115200 baud 8 data bits No parity 1 stop bit No flow control To determine which host computer COM port is mapped to the ZC702 Silicon Labs driver follow these steps for Windows Note If not already installed refer to the Zynq 7000 All Programmable SoC ZC702 Evaluation Kit and Video and Imaging Kit Getting Started Guide Ref 1 1 a PF oO Right click My Computer and select Properties Select the Hardware tab Click Device Manager Expand the Ports COM amp LPT section Make note of the COM port for the Silicon Labs CP210x USB to UART Bridge item This is the COM port that must be selected in the serial terminal program Figure 14 E Fie Action View Help mal mA o E3 Computer ies CypressUsbConsoleWindowsDriver Hss Disk drives 2 Display adapters J DYDICD ROM drives gy Human Interface Devices Hy IDE ATA ATAPI controllers gy IEEE 1284 4 compatible printers Sy IEEE 1284 4 devices H S IEEE 1394 Bus host controllers H Imaging devices Jungo Keyboards Mice and other pointing devices Monitors i Network adapters BY Ports COM amp LPT Intel R Active Management Technology SOL COM F Silicon Labs CP210x USB to UART Bridge COM4 SB Processors E Programming cables lt gt SM Driver
28. candescent lighting Alternatively the white balance setting can be automatically controlled by one of the Zynq 7000 SoC processors Based on the information received from the Image Statistics IP core the processor can select a weighted sum of one or more of the white balance configurations based on the calculated probability of each of the possible light sources The Brightness and Contrast sliders can be used to change the intensity of the image making it darker or brighter The Saturation slider can be used to change the intensity of the colors A Saturation of 0 produces a grayscale image The color correction coefficients for the four illumination sources were generated by placing an X Rite ColorChecker 24 Patch Classic target Ref 40 in an X Rite Macbeth Judge II light booth Ref 41 Bitmap images were collected using the GUI for the four different illumination settings Daylight Cool White Fluorescent U30 and Incandescent sources with the color correction matrix set to Bypass no color corrections applied Average values for the 24 patches at each illumination setting were calculated by a MATLAB software script A second script is used to model the Xilinx color correction matrix operations and determine a set of coefficients that result in output from the color correction matrix that best matches the known target values for the 24 patches The MATLAB software scripts used to develop the coefficients for the Color Correc
29. dress assigned in Preparing the SD Card The web page shown in Figure 17 appears Pro d fs lt S http 192 168 0 100 P BCx xin Zynq 7000 AP SoC Vi d Xilinx Zynq 7000 AP SoC Video and Imaging Kit 1080P60 Real Time Camera Demo General Settings Resetall settings Image Sensor Settings Processor Control Auto Gain f Auto Exposure Target Intensity 206 Image Sensor Auto Gain Exposure Target Intensity ES 206 Exposure 90 Analog Gain 0 Digital Gain 10 Image Capture Click to Take Snapshot Click to View Full Size Image Automatic Demonstration Mode Click to Start Clickto Stop Interval msec nm 5000 Description off Defective Pixel Correction Pixel Age Ly 0 Spatial Var EE 6000 Temporal Var HE 2 Color Filter Array Interpolation Bayer Phase GreenBlue 2 v Spatial Noise Reduction Strength 0 Edge Enhancement Strength 0 Color Correction Matrix Processor Control Auto White Balance y Brightness TA 100 Contrast TATE 0 Saturation Ta 100 Gamma Correction Processor Control Gamma Equalization Gamma Table Linear Image Statistics Update Click to Start Click to Stop Interval msec 2000 Scale car s Under Exp ia 05 Over Exp INTE 30 Histograms intensity red green blue Video Multiplexers Video Source Image Processing Pipeline Video Output Direct Pass Through v
30. e 62 bin sh echo Starting rcS echo Mounting filesystem mount t proc none proc mount t sysfs none sys mount t tmpfs none tmp echo Setting up mdev echo sbin mdev gt proc sys kernel hotplug mdev s mkdir p dev pts mkdir p dev i2c mount t devpts devpts dev pts echo Mounting SD card to mnt Mount SD Card Automatically mount dev mmcb1k0p1 mnt echo Configuring IP Address Call config_my_ip sh Script on SD Card mnt config my ip sh to Configure IP Address echo Starting telnet daemon telnetd 1 bin sh echo Starting http daemon httpd h var www Configure Web Server To Use Files in WWW httpd h mnt www Directory on SD Card echo Starting ftp daemon tcpsvd 0 21 ftpd ftpd w amp echo Starting dropbear ssh daemon dropbear echo rcS Complete Call launch_my_app sh Script on SD Card X794_62_111512 Figure 62 Modifications to etc init d rcs File XAPP794 v1 2 January 2 2013 www xilinx com 54 Appendix XILINX To customize the etc init d rcs file perform these steps on a Linux PC 1 2 Appendix Copy the RAM disk image to the Linux host Unzip the contents of the image gzip dcv ramdisk8M image gz gt ramdisk8M image Mount the RAM disk image mkdir ramdisk8M sudo mount o loop ramdisk8M image ramdisk8M Modify the contents as necessary sudo vi ramdisk8M etc init d rcsS Unmount the ramdisk image sudo umou
31. e Color Correction Matrix and Xilinx FPGA and SoC products Xilinx cannot provide assistance to customers developing color correction coefficients for the ON Semiconductor or any other image sensor Development of custom color correction coefficients is the responsibility of the customer The MATLAB software scripts are provided as examples to assist customers in developing their own color corrections for different image sensors Consult the Additional Resources tab of the ZVIK product page for additional information XAPP794 v1 2 January 2 2013 www xilinx com 26 Running the Demonstration g XILINX Configuring the Gamma Correction The gamma correction IP core is implemented as a look up table that is applied to all three color channels The gamma correction IP core can be adjusted using the GUI controls shown in Figure 26 Gamma Correction Processor Control Gamma Equalization Strength NN 50 Gamma Table Linear v X794_26_102612 Figure 26 Gamma Correction IP Core Controls The Gamma Table list box allows the manual gamma correction settings shown in Table 8 Table 8 Gamma Table Setting Setting Description Linear Gamma Correction OFF linear look up table Compression Table 1 2 2 Gamma Correction ON gamma 1 2 2 Compression Table 1 1 6 Gamma Correction ON gamma 1 1 6 Expansion Table 1 6 Gamma Correction ON gamma 1 1 6 Expansion Table 2 2 Gamma Correction ON gamma 1 2 2 Alterna
32. e Demonstration XILINX To assemble the camera 1 Remove both protective caps from the lens 2 Figure 4 X794_04_102612 Figure 4 VITA 2000 Camera Assembly Step 1 2 Attach the IR cut filter 1 to the lens 2 Figure 5 The filter screws onto the front of the lens X794_05_ 102612 Figure 5 VITA 2000 Camera Assembly Step 2 XAPP794 v1 2 January 2 2013 www xilinx com 7 Running the Demonstration XILINX 3 Screw the IR cut filter and lens assembly onto the VITA 2000 image sensor module 3 Figure 6 The image sensor module has a lens holder with an opening for standard C mount lenses X794_06_102612 Figure 6 VITA 2000 Camera Assembly Step 3 4 Attach the tripod 4 to the bottom of the VITA 2000 image sensor module Figure 7 The image sensor module lens holder has a hole that accepts standard tripods having a 1 4 inch screw with 20 threads per inch X794_07_102612 Figure 7 VITA 2000 Camera Assembly Step 4 XAPP794 v1 2 January 2 2013 www xilinx com 8 Running the Demonstration XILINX 5 Attach the LCEDI cable 5 to the back of the VITA 2000 image sensor module Figure 8 Both ends of the LCEDI are identical Either end can be connected to the image sensor module X794_08_102612 Figure 8 VITA 2000 Camera Assembly Step 5 6 Attach the other end of the LCEDI cable to the FMC IMAGEON FMC module 6 Figure 9 X794_09_ 102612 Figure 9 VITA 20
33. ect to working sets lt Back Next Cancel X794_54_112912 Figure 54 Importing Linux Applications 5 Click Finish The SDK tool imports the selected projects The software application is compiled during the import process XAPP794 v1 2 January 2 2013 www xilinx com 47 Rebuilding the Software Applications g XILINX Debugging the Linux Application Before starting a remote debug session with the ZVIK make sure the Linux application is NOT running on the ZVIK Open the serial console 115200 baud 2 Type the quit command in the zvik_camera linux application zvik_camera gt quit quit zvik_camera gt zynq gt 3 At the Linux prompt type the ifconfig command to confirm the ZVIK IP address zynq gt ifconfig etho Link encap Ethernet HWaddr 7E CA CF 54 6D 4E inet addr 192 168 1 10 Bcast 192 168 0 255 Mask 255 255 255 0 UP BROADCAST RUNNING MULTICAST MTU 1500 Metric 1 RX packets 93 errors 0 dropped 0 overruns 0 frame 0 TX packets 0 errors 0 dropped 0 overruns 0 carrier 0 collisions 0 txqueuelen 1000 RX bytes 17184 16 7 KiB TX bytes 0 0 0 B Interrupt 54 Base address 0xb000 zynq gt 4 Optionally to permanently prevent the application from auto starting on boot edit the launch_myapp sh script on the SD card and comment out the statement shown using the character zynq gt vi mnt launch_my_app sh mnt zvik_camera_linux_app elf A remote connection can be established be
34. ed according to the formula number of pixels in the 242 255 range gt 0 e Agreen bar on the top indicates that the red color channel is not over saturated The green histogram Figure 31 shows the distribution of green pixel values X794_31_121412 Figure 31 Green Histogram e Ared bar on the top right indicates that the green color channel is over saturated according to the formula number of pixels in the 242 255 range gt 0 e Agreen bar on the top indicates that the green color channel is not over saturated The blue histogram Figure 32 shows the distribution of blue pixel values yeaa Figure 32 Blue Histogram X794_32_121412 e A red bar on the top right indicates that the blue color channel is over saturated according to the formula number of pixels in the 242 255 range gt 0 XAPP794 v1 2 January 2 2013 www xilinx com 30 Hardware Platform t XILINX e A green bar on the top indicates that the blue color channel is not over saturated The contents of the histograms are static by default unless configured otherwise There are two options for updating the histograms see Figure 29 1 To update the contents of the histograms a single time click the Update button 2 To periodically update the histograms click the Click to Start button To stop periodically updating the histograms click the Click to Stop button The frequency of the update can be adjusted by moving the Interval msec slider
35. ed in the previous section o Nog XAPP794 v1 2 January 2 2013 www xilinx com 5 Running the Demonstration XILINX 9 Click the Subnet mask field and ensure that it is populated with 255 255 255 0 The dialog box should appear be similar to Figure 3 6 Figure 3 Host Computer IP Address Configuration Dialog Box Internet Protocol Version 4 TCP IPv4 Properties mE General You can get IP settings assigned automatically if your network supports this capability Otherwise you need to ask your network administrator for the appropriate IP settings Obtain an IP address automatically Use the following IP address IP address 192 168 1 20 Subnet mask 255 255 0 Default gateway Obtain DNS server address automatically Use the following DNS server addresses Preferred DNS server Alternate DNS server E validate settings upon exit Prone X794_03_102512 10 Select OK to close the Internet Protocol Version 4 TCP IPv4 dialog box Select OK to close the Local Area Connection Properties dialog box Assembling the Camera If the camera assembly has been completed go to Setting Up the Hardware The ZVIK package contains these components which must be assembled IR cut filter Lens 2 3 8 mm VITA 2000 C image sensor module with C mount lens holder Tripod LCEDI cable FMC IMAGEON FMC module XAPP794 v1 2 January 2 2013 D o e O N gt www xilinx com Running th
36. erm 4 69 or HyperTerminal e Zip Unzip software for example 7 Zip e Web browser such as Internet Explorer to operate the web based GUI e USB UART driver from Silicon Labs Ref 39 might already be installed For information about installing the USB UART driver see Zynq 7000 All Programmable SoC ZC702 Evaluation Kit and Video and Imaging Kit Getting Started Guide Ref 1 To build the hardware and software components the ISE Design Suite Embedded Edition 14 2 with compatible OS is also required Target Hardware Requirements The target hardware requirements for running the camera design and its applications are e Zynq 7000 All Programmable SoC Video and Imaging Kit including e Xilinx XC7Z020 CLG484 1 SoC based ZC702 evaluation board e Avnet FMC IMAGEON FMC module e VITA 2000 camera module including optics tripod and cable e HDMI cable e SD MMC flash card e 12V power supply e HDMI monitor supporting 1080p60 resolution A DVI monitor can also be used but an HDMI female to DVI D male connector adapter must be obtained not provided with the kit The connector adapter is available at most electronic retailers or through a variety of online sources e SD MMC flash card writer For more information on the Zyng 7000 AP SoC Video and Imaging Kit refer to www xilinx com zvik For more information on the Zyng 7000 AP SoC ZC702 evaluation board including hardware user manual schematics and BOM refer to
37. ese domains run at 150 MHz 75 MHz and 148 5 MHz respectively The first clock generator module receives a 100 MHz input clock from PS FCLKO and generates 75 MHz and 150 MHz The AXI 4 Lite interconnect operates on the 75 MHz clock domain The AXI MM interconnect to the HP port of the PS operates on the 150 MHz clock domain The stream to memory map S2MM and memory map to stream MM2S channels of the VDMA operate on the 150 MHz clock domain www xilinx com 33 Hardware Platform Table 11 System Clocks XILINX The video clock comes from the external clock synthesizer on the Avnet FMC IMAGEON card The video input and output interfaces as well as the video processing pipeline operate on the video clock domain The clock signals are shown in Table 11 Frequency Clock Signal Source MHz Use FPGA_CLK PS FPGA_CLKO 100 Input clock to clock generator clk_100mhz Clock generator MMCM 100 Not used clk_200mhz Clock generator MMCM 200 200 MHz reference clock for deserializer in VITA receiver clk_75mhz Clock generator MMCM 75 Slave clock for AXI4 Lite interconnect clk_150mhz Clock generator MMCM 150 Clock for AXI MM interconnect clk_25mhz Clock generator MMCM 25 Slowest sync clock used for reset logic fmc_imageon_video_clk1 External video clock coming 148 5 Input clock to clock generator from clock synthesizer on FMC module vid_out_clk Clock generat
38. ess GI lp gt tx_bd ffdfb000 lp gt tx_bd_dma 2f220000 lp gt tx_skb ee17d580 GI lp gt rx_bd ffdfc000 lp gt rx_bd_dma 2e47f 000 lp gt rx_skb ee17d680 GEM MAC Oxefcb686e O0x0000fbid 6e 68 cb ef 1d fb GEM phydev ef28ca00 phydev gt phy_id 0x1410e40 phydev gt addr 0x7 eth0O phy_addr 0x7 phy_id 0x01410e40 eth0 attach Marvell 88E1116R phy driver Starting telnet daemon Starting http daemon Starting ftp daemon Starting dropbear ssh daemon rcS Complete OG t t XAPP794 v1 2 January 2 2013 www xilinx com 14 Running the Demonstration g XILINX Finally the camera demonstration software generates this output on the serial console Xilinx Zyng 7000 AP SoC Video and Imaging Kit 1080P60 Real Time Camera Demonstration FMC IPMI Initialization Configure ZC702 IIC Mux for Port 6 FMC2 FMC Module Validation Board Information Manufacturer Avnet Product Name FMC IMAGEON Serial Number 6122 Part Number AES FMCIMAGEON G SUCCESS Detected FMC IMAGEON module FMC IMAGEON Initialization FMC IMAGEON Video Clock Initialization Initialize Video Output for 1080P60 Video Resolution 1080P Configure ZC702 IIC Mux for Port 1 HDMI ZC702 HDMI Output Initialization FMC IMAGEON VITA Receiver Initialization FMC IMAGEON VITA Initialization FMC IMAGEON VITA Configuration for 1080P60 timing VITA Status Image Width 1920 Image Height 1080 Frame Rate 61 frames sec Video
39. he BOOT BIN file consists of the Zynq FSBL binary image zynq_fsbl e1f ARM Linux u boot binary image u boot e1f and the FPGA hardware design bitstream file system bit To update the SD card with custom Linux applications copy the custom ELF files to the SD card in the location indicated in Table 13 Use the mouse to copy this file from the C C Perspective of the SDK user interface then replace the file of the same name in the C zvik_camera binaries sd_content folder To update the SD card with a custom hardware design bitstream copy the system bit file from zvik_camera hw SDK SDK_Export hw to the zvik_camera binaries boot_image directory regenerate the described in Regenerating the BOOT BIN File then copy the custom N file as N file to the SD BOOT B BOOT B card XAPP794 v1 2 January 2 2013 www xilinx com 52 Preparing the SD Card Boot Image XILINX Regenerating the BOOT BIN File The BOOT BIN file can be regenerated using the SDK tool by following these steps 1 Inthe SDK tool window select Xilinx Tools gt Create Boot Image see Figure 61 Create Zynq Boot Image Create 2ynq Boot Image Creates Zynq Boot Image from given FSBL bare metal application and bit streamfoptional file p Basic Advanced Bif file Create a new bif file List of partitions in the boot image Remove File Offset Alignment Allocation Output file i Ca
40. he host computer IP address and the ZVIK IP address For example if the ZVIK is configured with the IP address 192 168 1 10 and subnet mask 255 255 255 0 the host must be configured with an IP address containing identical numbers in the first three positions and a differing number in the fourth group such as 192 168 1 20 Configuring the Host IP Address Configure the IP address of the host computer to one that is compatible with the ZVIK IP address discussed in Preparing the SD Card Record the host PC s original network settings so they can be restored after running the demonstration Configure the computer s IP address to 192 168 1 20 or similar The subnet mask can be 255 255 255 0 The screen captures and icon names shown below might be slightly different depending on the computer s operating system version For Windows 7 Click the Start button and select Control Panel Select Network and Sharing Center Select Change Adapter Settings from the options on the left panel Right click the adapter corresponding to the wired Ethernet port identified as a local area connection and select Properties PON a gt Accept the User Account Control dialog box by selecting Yes Click the Internet Protocol Version 4 TCP IPv4 entry and select Properties On the General tab select the radio button for Use the following IP address In the IP address field enter 192 168 1 20 ora value that is compatible with the ZVIK IP address configur
41. he lighting conditions and applies a weighted sum of one or more of the pre determined white balance corrections based on the calculated probability of each of the possible light sources using the color correction matrix IP core Auto gain Based on the intensity of the image reconfigures the image sensor gain setting Auto exposure Based on the intensity of the image reconfigures the image sensor exposure setting Global contrast Automatically determines a gamma correction look up table www xilinx com 42 Rebuilding the Hardware Platform g XILINX Web Session Task The web based GUI is implemented with several applications Figure 49 illustrates how these applications interact with each other zvik_camera_linux_webserver elf Web Pages STDIN STDOUT Image Statistics Task Web Session Task zvik_camera_linux_app elf X794_49_ 102612 Figure 49 Web Based GUI Block Diagram The web server a standard service provided by the open source BusyBox package in embedded Linux is the engine that generates the reference design web based GUI The web page index htm1 that makes up the web based GUI uses these technologies e JavaScript e CGI When a CGI request is generated by the Web server the zvik_camera_linux_webserver elf application is invoked When this occurs the application translates the CGI request which is in HTTP format to text based commands and sends the command to the zvik_camera_linux_app elf
42. in_Page AMBA AXI4 Stream Protocol Specification http infocenter arm com help index jsp topic com arm doc ihi0051a index html PCI SIG Documentation http Awww pcisig com specifications Xilinx PlanAhead Design and Analysis Tool website http Avww xilinx com tools planahead htm UG873 Zynq 7000 All Programmable SoC Concepts Tools and Techniques UG673 Quick Front to Back Overview Tutorial PlanAhead Design Tool www xilinx com 56 References XILINX Video Documents 25 PG001 LogiCORE IP Color Correction Matrix Product Guide 26 PG002 LogiCORE IP Color Filter Array Interpolation Product Guide 27 PG003 LogiCORE IP Image Edge Enhancement Product Guide 28 PG004 LogiCORE IP Gamma Correction Product Guide 29 PG005 LogiCORE IP Defective Pixel Correction Product Guide 30 PG008 LogiCORE IP Image Statistics Product Guide 31 PGO11 LogiCORE IP Image Noise Reduction Product Guide 32 PGO12 LogiCORE IP Chroma Resampler Product Guide 33 PGO13 LogiCORE IP RGB To YCrCb Color Space Converter Product Guide 34 PGO14 LogiCORE IP YCrCb To RGB Color Space Converter Product Guide 35 PGO16 LogiCORE IP Video Timing Controller Product Guide 36 PGO20 LogiCORE IP AXI Video Direct Memory Access Product Guide 37 UG761 AXI Reference Guide 38 UG934 AX 4 Stream Video IP and System Design Guide Additional Useful Documents Documents associated with other software tools and IP used by the design are available at
43. ipeline Details Video Timing Controller The AXI Video Timing Controller VTC is a general purpose video timing generator and detector The input side of this core automatically detects horizontal and vertical synchronization pulses polarity blanking timing and active video pixels The output side of the core generates the horizontal and vertical blanking and synchronization pulses There are two AXI VTC cores in the image processing pipeline as shown in Figure 35 RGB AXI ER EA EA EEN Tg YUV Noise ER EA EA EEN Lie X794_35_102612 Figure 35 AXI VTC Core Pipeline Position The first AXI VTC core detects the video resolution of the video input coming from the VITA receiver The second AXI VTC core is used to regenerate the VSYNC and HSYNC signals that were not preserved by the image processing pipeline XAPP794 v1 2 January 2 2013 www xilinx com 35 Hardware Platform gt XILINX For additional information about the Video Timing Controller LogiCORE solution and the detailed product guide refer to the Video Timing Controller product page on Xilinx com http www xilinx com products intellectual property EF DI VID TIMING htm Defective Pixel Correction The AXI Defective Pixel Correction DPC core is the first processing element in the image processing pipeline as shown in Figure 36 AXI RGB AXI AXI YUV AXI AXI AXI CFA YUV Noise Enhance RGB CCM Gamma VTC X794_36_102612 AXI AXI VTC DPC Figure 36
44. kpoint e Source file avnet_console c e Function avnet_console_record_command 15 Select Window gt Open Perspective gt C C to view the C C perspective 16 Expand the zvik_camera_1linux_app application then double click the avnet_console c source file 17 Search for the avnet_console_record_command function then double click the left margin to enable a breakpoint Select the command Search gt C C Search and enter the function name The search results appear in a new tab in a window below the source window see Figure 60 e avnet_console c 3 e return if defined LINUX CODE void avnet console record command avnet_console_t pConsole int cargc char cargv int bDispSyntax o if carge gt 1 amp amp stromp cargv 1 help j bDispSyntax 1 else if carge gt 1 X794_60_102612 Figure 60 Breakpoint Search Results 18 Switch back to the Debug perspective 19 On the console type rec mnt image1 bmp then press enter The debugger stops on the specified breakpoint XAPP794 v1 2 January 2 2013 www xilinx com 51 Preparing the SD Card Boot Image 20 21 22 23 24 25 26 27 28 XILINX Click the Resume button or press F8 The contents of the video frame buffer are saved to a BMP file Switch to the Remote System Explorer perspective Select the mnt directory then right click and select Refresh The newly created image bmp is now
45. lens is incorrect for a specific application determine the required field of view FOV using these measurements e Distance to object L e Size of object H x V XAPP794 v1 2 January 2 2013 www xilinx com 17 Running the Demonstration g XILINX The FOV measurements are shown in Figure 16 a ee ee ee X794_16_102612 Figure 16 Determining the Required Field of View FOV A lens calculator such as http www 1stvision com lens lens_calculator htm can determine the FOV and consequently the ideal lens choice for a specific application Using the Text based Console The Linux camera demonstration application accepts commands from a text based console Type the help command to list the supported commands The Linux application also has a web based GUI that significantly improves the ease of use and is described in Using the Web based GUI zvik_camera gt help Xilinx Zyngq 7000 AP SoC Video and Imaging Kit 1080P60 Real Time Camera Demonstration General Commands help Print the Top Level menu Help Screen quit Exit console if applicable verbose Toggle verbosity on off delay Wait for specified delay mem Memory accesses I2C Commands iic0 IIC accesses on FMC IPMI I2C chain irgi IIC accesses on FMC IMAGEON I2C chain VITA Commands vita VITA commands init status vspi SPI accesses to VITA sensor vreg Memory accesses to VITA receiver trig Trigger configuration off stress
46. linx assumes no obligation to correct any errors contained in the Materials or to notify you of updates to the Materials or to product specifications You may not reproduce modify distribute or publicly display the Materials without prior written consent Certain products are subject to the terms and conditions of the Limited Warranties which can be viewed at http Avww xilinx com warranty htm IP cores may be subject to warranty and support terms contained in a license issued to you by Xilinx Xilinx products are not designed or intended to be fail safe or for use in any application requiring fail safe performance you assume sole risk and liability for use of Xilinx products in Critical Applications http www xilinx com warranty htm critapps XILINX PRODUCTS ARE NOT DESIGNED OR INTENDED TO BE FAIL SAFE OR FOR USE IN ANY APPLICATION REQUIRING FAIL SAFE PERFORMANCE SUCH AS APPLICATIONS RELATED TO I THE DEPLOYMENT OF AIRBAGS II CONTROL OF A VEHICLE UNLESS THERE IS A FAIL SAFE OR REDUNDANCY FEATURE WHICH DOES NOT INCLUDE USE OF SOFTWARE IN THE XILINX DEVICE TO IMPLEMENT THE REDUNDANCY AND A WARNING SIGNAL UPON FAILURE TO THE OPERATOR OR IlI USES THAT COULD LEAD TO DEATH OR PERSONAL INJURY CUSTOMER ASSUMES THE SOLE RISK AND LIABILITY OF ANY USE OF XILINX PRODUCTS IN SUCH APPLICATIONS XAPP794 v1 2 January 2 2013 www xilinx com 58
47. mation about the Color Correction Matrix LogiCORE solution and the detailed product guide refer to the Image Edge Enhancement product page on Xilinx com http www xilinx com products intellectual property EF DI CCM htm Gamma Correction The AXI GAMMA core implements look up tables that are used to implement gamma correction in the image processing pipeline as shown in Figure 45 AXI RGB AXI AXI YUV AXI AXI AXI CFA YUV Noise Enhance RGB CCM Gamma VTC X794_45_102612 Figure 45 AXI Gamma Core Pipeline Position For additional information about the Gamma Correction LogiCORE solution and the detailed product guide refer to the Gamma Correction product page on Xilinx com http www xilinx com products intellectual property EF DI GAMMA htm An application to demonstrate the capabilities of the Zynq 7000 AP SoC PS and PL combination for mainstream video embedded systems is implemented on top of Linux kernel 3 x Developers can easily tailor Linux to their particular applications by porting their applications and customizing open source software packages The pre verified Linux platform enables rapid software platform modification to adapt to hardware changes Zynq 7000 AP SoC based 1080p60 Camera Application A unified and multi threaded Linux software application is implemented to demonstrate the capabilities of the Zyng 7000 AP SoC PS and PL combination for mainstream video embedded systems The main application initializes
48. mation to adjust the image sensor exposure and gain settings The Target Intensity slider allows the user to select the desired target intensity e Image Sensor Auto Gain Exposure Uses the image sensor internal auto gain exposure control AEC to control the gain and exposure The Target Intensity slider allows the user to select the desired target intensity Note When the processor control boxes are checked they disable both the manual gain and exposure settings and the image sensor AEC Sliders allow the gain and exposure to be adjusted manually when the auto gain and exposure check boxes are not selected XAPP794 v1 2 January 2 2013 www xilinx com 21 Running the Demonstration XILINX The analog gain provides a course manual adjustment of the image sensor gain as shown in Table 2 Table 2 Analog Gain Slider Settings Setting 0 Description Analog gain 0 00 pey Analog gain 1 14 Analog gain 1 33 Analog gain 1 60 Analog gain 2 00 Analog gain 2 29 Analog gain 2 67 Analog gain 3 20 Analog gain 4 00 O oO NI OIA AJ OIN Analog gain 5 33 oO Analog gain 8 00 The digital gain slider provides a fine manual adjustment of the image sensor gain as shown in Table 3 Table 3 Digital Gain Slider Settings Setting 0 00 31 99 Description Digital gain 0 00 31 99 The exposure slider provides a ma
49. ncel X794_61_111512 Figure 61 Create Zynq Boot Image Dialog 2 Click the Browse button under the Basic tab adjacent to the FSBL elf entry 3 Select the zvik_camera binaries boot_image zynq_fsbl_0 elf file and click Open 4 Click the Add button 5 Select the zvik_camera binaries boot_image system bit file and click Open 6 Click the Add button again 7 Select the zvik_camera binaries boot_image u boot elf file and click Open 8 Click the Browse button adjacent to the Output Folder entry 9 Select the zvik_camera binaries boot_image file directory and click OK 10 Click the Create Image button 11 If the Output file already exists dialog appears click OK 12 Rename the generated file u boot bin to BOOT BIN 13 Copy the updated BOOT BIN file to the SD card XAPP794 v1 2 January 2 2013 www xilinx com 53 Preparing the SD Card Boot Image XILINX Linux Kernel Image and Linux Root File System It is beyond the scope of this application note to provide instructions for regenerating the Linux kernel image and the Linux root file system It is assumed that the Linux root file system is created as described here http wiki xilinx com zyng rootfs or use the pre built RAM disk image available here http xilinx wdfiles com local files zyng release 14 2 14 2 release tar gz For this reference design the etc init d rcsS file inside the ramdisk8M image gz archive was modified as shown in Figur
50. nstration g XILINX The intensity histogram Figure 28 provides information about the overall image exposure X794_28 121412 Figure 28 Intensity Histogram In addition to the histogram distribution of the data the bar at the top of the histogram provides visual feedback on the exposure e Ared bar on the top left indicates that the image is under exposed according to the formula number of pixels in the 0 64 range under exposure threshold gt number of pixels in the 64 192 range e Ared bar on the top right indicates that the image is over exposed according to the this formula number of pixels in the 192 255 range over exposure threshold gt number of pixels in the 64 192 range e A green bar on the top indicates that the image has good exposure neither under exposed or over exposed The under and over exposure thresholds can be adjusted by moving the Under Exp and Over Exp sliders at the top of the third column as shown in Figure 29 m Image Statistics Update Click to Start Click to Stop Interval msec 2000 Scale Ti ho X794_29_102612 Figure 29 Image Statistics Exposure Controls XAPP794 v1 2 January 2 2013 www xilinx com 29 Running the Demonstration XILINX The red histogram Figure 30 shows the distribution of red pixel values X794_30_121412 Figure 30 Red Histogram e A red bar on the top right indicates that the red color channel is over saturat
51. nt ramdisk8M Zip the contents of the image gzip v9 ramdisk8M image Copy the RAM disk image back to the SD card Table 14 lists the acronyms used in this application note Table 14 Acronyms Acronym Definition DPC Defective Pixel Correction EDK Embedded Development Kit FMC FPGA Mezzanine Card FSBL First Stage Boot Loader PS Processing System PL Programmable Logic FPGA logic cell array SDK Software Development Kit TPG Video Test Pattern Generator VTC Video Timing Controller VDMA Video Direct Memory Access ZVIK Zynq 7000 All Programmable SoC Video and Imaging Kit References 1 To search the Answer Database of silicon software and IP questions and answers or to create a technical support WebCase see http www xilinx com support For a comprehensive listing of Video and Imaging application notes white papers reference designs and related IP cores see the Video and Imaging Resources page at http Avww xilinx com esp video refdes_listing htm ref_des These documents provide supplemental material useful with this application note XAPP794 v1 2 January 2 2013 www xilinx com 55 References XILINX Zynq 7000 All Programmable SoC Documents and Links 1 oa Po N 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 XAPP794 v1 2 January 2 2013 Zynq 7000 All Programmable SoC ZC
52. nual adjustment of the image sensor exposure time as shown in Table 4 Table 4 Exposure Slider Settings Setting 1 99 Description Exposure in percentage of frame time 16 6 ms 1 0 16 ms 99 16 5 ms XAPP794 v1 2 January 2 2013 www xilinx com 22 Running the Demonstration g XILINX Capturing a 1080P Image The Image Capture section of the web based GUI allows capturing an image from the video frame buffer as shown in Figure 19 Image Capture Click to Take Snapshot X794_19_ 102612 Figure 19 Image Capture Controls After clicking the Click to Take Snapshot button a scaled down version of the image is displayed in the GUI A full size version of the image 24 bit RGB 1920 x 1080 bitmap BMP can be saved to the host hard drive by right clicking the image and selecting Save Picture As This allows analysis of the image with a separate image viewing program Configuring the Defective Pixel Correction The Xilinx Defective Pixel Correction IP core can be configured with three sliders as shown in Figure 20 Defect Pixel Correction Pixel Age o Spatial Var e000 Temporal Var e X794_20_102612 Figure 20 Defective Pixel Correction IP Core Controls To observe the operation of the Defective Pixel Correction core move the Pixel Age slider to 100 A few small white dots appear on the monitor To better see the dots point the image sensor at a uniform dark background
53. on stats s Image Statistics awb Auto White Balance on off age Auto Gain Control on off aec Auto Exposure Control on off geq Gamma Equalization on off noise Noise Reduction configuration enhance Image Enhance configuration ccm Color Correction Matrix configuration gamma Gamma Correction configuration Video Source Selection video Video Source Initialization and Selection vita ipipe Video Frame Buffer Commands vdma Control frame buffer start stop fill rec Save frame buffer image to BMP file play Fill frame buffer image from BMP file zvik_camera gt XAPP794 v1 2 January 2 2013 www xilinx com 16 Running the Demonstration g XILINX Adjusting the Lens The content captured by the image sensor and processed by the image processing pipeline should be visible on the monitor If the image is blurry adjust the lens focus which ranges from 1 cm to 150 cm If the image is too dark or too light adjust the aperture Figure 15 If the image is black the aperture should be adjusted to allow more light to enter the image sensor Focus Aperture x794_15_102612 Figure 15 Adjusting the Focus and Aperture To adjust the focus aperture or both 1 Unlock the set screws by loosening them on both the aperture and focus ring 2 Twist until the desired result is achieved 3 Lock the setting by tightening the set screw This kit includes an 8 mm lens that is appropriate for a broad range of video applications If this
54. onsole Mode 12 Click the display selected console button and select the zvik_camera_linux_app elf entry as shown in Figure 58 E eS Fi Tasks Terminal 1 E Pro Problems Executables 8 Memory m Sect Elle linux _app Debug Remote ARM Linux atta olan ee 5 15 12 4 53 PM mx fob a S ee oS eM E 1 C Build 2vik_camera_linux_app 2 Bootgen em 3 2vik_camera_linux_app Debug Remote ARM Linux Application arm xilinx linux gnueabi gdb 5 15 12 4 53 PM 4 2vik app elf 5 15 12 4 53 PM vik_camera_linux _app Debug Remote ARM Linux Application tmpfz camera_linux 31 done locals gdb ES lt ill gt i whe X794_58_102612 Figure 58 Display Selected Console XAPP794 v1 2 January 2 2013 www xilinx com 50 Rebuilding the Software Applications g XILINX 13 Click the Resume button or press F8 The console output shown in Figure 59 is displayed E console 23 N Ai Tasks amp Terminal 1 ai Problems F Executables Memory wie z2vik_camera_linux_app Debug Remote ARM Linux Application tmpfzvik_camera_linux_app elF 5 15 12 4 53PM a Ex MEE A Bris F TICU TICO LIIL CLALLCACILOLN Inar y MANL a MANL O 7 way 7 A Record Playback Commands rec Save frame buffer image to BMP file play Fill frame buffer image from BMP file zvik_camera gt v gt X794_59_102612 Figure 59 Example Console Output 14 Add this brea
55. or MMCM 148 5 Clock for video input and video output interfaces and video processing pipeline vita_clk_ref Clock generator MMCM 37 125 Reference clock for VITA image sensor Notes 1 Mixed mode clock manager XAPP794 v1 2 January 2 2013 Reset The proc_sys_reset module implements a reset scheme Input to the proc_sys_reset core is generated by PS FCLK_RESETO The polarity of input reset to this block is indicated by the parameter C_EXT_RESET_HIGH In this design C_EXT_RESET_HIGH is set to 0 because the reset generated by the PS is active Low This block generates various types of resets such as reset for interconnect and peripheral reset All the blocks in the PL are driven by interconnect reset which is active Low For detailed information about the complete feature set and a functional description of the proc_sys_reset IP core refer to LogiCORE IP Processor System Reset Module Product Specification Ref 4 AXI Interconnect The PL design has one interconnect for the AXI memory mapped master and one interconnect for the AXI register interface The AXI memory mapped interconnect is connected to a master the AXI VDMA It is also connected to a slave the HPO port of the Zynq 7000 AP SoC PS This interconnect operates at 150 MHz and the data width is 64 bits The read write acceptance and issuance are set to 8 The acceptance and issuance helps improve system performance The PS HP port can accept a maximum burst length of 16
56. periodically reads the image statistics from the image sensor and performs these application examples auto gain auto exposure auto white balance global contrast gamma equalization XAPP794 v1 2 January 2 2013 www xilinx com 41 Software Platform amp XILINX Figure 48 illustrates how the image statistics handler interacts with the image processing pipeline Gain Control Processing System Image Statistics Handler Exposure Auto Image Auto Global Control Exposure Statistics White Contrast Handler Balance Defective Pixel Correction Camera Input Statistics Color Filter Color Gamma Noise Array Correction Correction Reduction Interpolation Matrix Programmable Logic X794_48_102612 Figure 48 Image Statistics Handler The image processing pipeline consists of these IP cores Defective pixel correction Color filter array interpolation Image statistics Color correction Gamma correction Noise reduction Edge enhance The image statistics IP core provides an important link with the Linux applications It gathers statistics about the camera input including intensity and chroma histograms which are used to automatically adjust the image processing pipeline to improve image quality The image statistics task provides some interesting application examples that are all based on the image statistics IP core XAPP794 v1 2 January 2 2013 Auto white balance Attempts to determine t
57. sion task via named pipes to provide control from a web page serviced by a web server XAPP794 v1 2 January 2 2013 www xilinx com 40 Software Platform amp XILINX Main Task The main task is the main function which initializes the reference design This function also Initializes the VITA image sensor Initializes the image processing pipeline Initializes the video frame buffer Launches the other tasks image statistics task web session task When the reference design has been initialized it provides a text based command interface via the STDIN and STDOUT pipes The main c source code is Main entry point int main Specify Base Addresses of all PCOREs init_base_addresses amp fmc_imageon_demo 0 Initialize FMC IMAGEON Demo fmc_imageon_demo_init amp fmc_imageon_demo Initialize Web Console print_avnet_console_web_app_header start_avnet_console_web_application Initialize Serial Console print_avnet_console_serial_app_header start_avnet_console_serial_application while 1 Process user input from Serial Console if transfer_avnet_console_serial_data user requested to quit break Shutdown the FMC IMAGEON Demo fmc_imageon_demo_quit amp fmc_imageon_demo return 0 Image Statistics Task The image statistics task vipp_stats_handler is launched when the video image processing pipeline is initialized This task
58. sions Exact results can vary These numbers should be used as a guideline There are many advantages to implementing this system in a Zyngq 7000 AP SoC Processing intensive pixel operations are efficiently implemented in the PL Complex decision making algorithms like automatic white balance and automatic exposure are efficiently implemented in the PS Processor operations can easily be modified or added to the PS The hardware system can easily be expanded or customized by adding removing instances of Xilinx IP cores from the Xilinx IP catalog or third party sources The hardware system can be further customized by adding custom logic that either interfaces to the PS or is totally independent A choice of Zynq 7000 AP SoC sizes allows room to shrink or expand the hardware PL to meet system requirements System Features Processing System XAPP794 v1 2 January 2 2013 Two ARM Cortex A9 processors each with a 32 KB instruction cache and a 32 KB data cache and a NEON coprocessor ARM processors at 800 MHz 512 KB of level 2 cache 256 KB of on chip RAM 128 KB of on chip ROM AMBA AXI interconnect Multi protocol 32 bit DDR DRAM controller DDR3 DRAM at 533 MHz Standard peripheral interfaces including flash USB Ethernet UART 12C and more High bandwidth interconnect to and from the PL Power domain independent of the PL www xilinx com 32 Hardware Platform Programmable Logic e One 64 bit AXI interconnec
59. spatial noise in the image processing pipeline as shown in Figure 42 AXI AXI AXI YUV AXI AXI AXI CFA Noise Enhance RGB CCM Gamma VTC X794_42_100912 Figure 42 AXI Noise Core Pipeline Position For additional information about the Image Noise Reduction LogiCORE solution and the detailed product guide refer to the Image Noise Reduction product page on Xilinx com http www xilinx com products intellectual property EF DI IMG NOISE htm Image Edge Enhancement The AXI ENHANCE core is used to accentuate edges in the image and is positioned in the image processing pipeline as shown in Figure 43 AXI l suites X794_43_102612 Figure 43 AXI Enhance Core Pipeline Position For additional information about the Image Edge Enhancement LogiCORE solution and the detailed product guide refer to the Image Edge Enhancement product page on Xilinx com http www xilinx com products intellectual property EF DI IMG ENHANCE htm Color Correction Matrix The AXI Color Correction Matrix CCM core is used to implement various color corrections in the image processing pipeline Figure 44 e White balance e Brightness e Contrast e Saturation AXI RGB AXI AXI YUV AXI AXI AXI CFA YUV Noise Enhance RGB CCM Gamma VTC X794_44_102612 Figure 44 AXI CCM Core Pipeline Position XAPP794 v1 2 January 2 2013 www xilinx com 38 Software Platform AXI VTC Software Platform XILINX For additional infor
60. t APU Clock Reset and interrupt n 2 ji Connie ge Sod el D Communication Low Speed Cortex Ao Cortex Aa i OMA and Timer maine MMU MPCore MMU MPCore Debug Loess cru cru G FPGA Reconfiguration Controt 32KB 32KB D 32KB 32KBD 3 General Purpose 10 Regs Cache Cache Cache Cache Imerprocessor Communication i Memory ang Memory Controller Snoop Control Unit a PO 4 612 KA L2 Cache amp Controller Penpheral Controller 4 por ocm 266KB OCM J Uy interconnect BootROm Verification 4 Video and Image Processing Project Local PCores Proyect Penpherel Repository D USER Video and Image Processing Programm abie Logic to Memory Mereenneet 9S wa regenerate brtstream if necasary and may take some tme t frish XADC f Logic PL Select Directory location for hardware desop on fies SKEXK_Eeort owl ooon Operate a matte asx This allows you to eve the applicable Cicense f to terms and conditions ine This he app to eve conditio able License ammed device after ope of this core to adhere ing for some pericd of time This allows y terms and conditions of the applicable Lii nse 7 rogramed device of this c er operating for seme period of time This allows you to evalt to adhere to terms and conditions of the applicable License F programmed de ting for some period of time This allows you to evs z to terms and
61. t at 150 MHz e Two 32 bit AXI interconnect at 75 MHz e VITA receiver serial LVDS interface e Image processing pipeline e Video frame buffer AXI VDMA e HDMI output interface Device Address Map XILINX The pcores that are instantiated on the M_AXI_GPO port are shown in Table 10 Table 10 AXI GPO Port Address Map Instance Peripheral Base Address High Address axi_vdma_0O axi_vdma_v5_00_a 0x40090000 0x4009FFFF IIC_MAIN axi_lic_v1_O1_a 0x40800000 Ox4080FFFF fmc_imageon_iic_0O axi_lic_v1_01_a 0x40900000 Ox4090FFFF fmc_imageon_vita_receiver_0O fmc_imageon_vita_receiver_v1_09_a 0x40820000 0Ox4082FFFF axi_vtc_2 axi_vic_v3_00_a 0x40440000 Ox4044FFFF xsvi_mux_0 xsvi_mux_v1_00_b 0x40600000 Ox4060FFFF xsvi_mux_1 xsvi_mux_v1_00_b 0x40610000 Ox4061FFFF vic_0 axi_vic_v3_00_a 0x7DE20000 0x7DE2FFFF dpc_0O axi_dpc_v4_00_a 0x75000000 0x7500FFFF cfa_O axi_cfa_v4_00_a 0x64C00000 Ox64COFFFF stats_0O axi_stats_v3_00_a 0x6CE00000 Ox6CEOFFFF noise_O axi_noise_v3_00_a 0x76000000 0x7600FFFF enhance_0O axi_enhance_v3_00_a 0x7B200000 0x7B20FFFF ccm_0 axi_ccm_v3_00_a 0x64C20000 Ox64C2FFFF gamma_0O axi_gamma_v4_00_a 0x6FC00000 Ox6FCOFFFF vic_1 axi_vic_v3_00_a 0x7DE00000 0x7DEOFFFF XAPP794 v1 2 January 2 2013 System Configuration Clocking The PL design has several clock domains AXI MM memory mapped interconnect AXI register interface and video clock s Th
62. t the VITA 2000 camera to the FMC module with the provided LCEDI cable Connect the HDMI monitor to the ZC702 HDMI out connector P1 with the provided HDMI cable If a DVI monitor is used an HDMI female to DVI D male connector adapter must be provided The connector adapter is available at most electronic retailers or through online sources Connect the USB Serial port on the ZC702 board J17 labeled USB UART to the host computer using the provided USB Mini B to USB A cable Connect the Gbit Ethernet connector on the ZC702 to the host computer using the provided Ethernet cable Ensure that the power switch on the ZC702 board is off by moving the switch away from the power connector Connect the 12V power supply to the ZC702 board Insert the SD card into the ZC702 board SD card connector Ensure that the switches are set as shown in Figure 13 allowing the ZC702 board to boot from the SD MMC card YaH JAV 7 2 us a M X794_13_102612 Figure 13 Switch Settings for the SD MMC Card Boot Mode Switch 10 Ensure that the monitor is set for HDMI or DVI if using an HDMI female to DVI D male adapter at 1920 x 1080 resolution 11 Power on the ZC702 board Observing the Linux Console on the Serial Port During boot the Zynq 7000 SoC displays these steps on its serial port XAPP794 v1 2 January 2 2013 First stage boot loader FSBL output U Boot output Linux console output www xilinx com 12 Running the Demonstr
63. te describes the 1080p60 camera image processing reference design that showcases various features of the ZVIK provides a working camera image processing example design and introduces several Xilinx video IP cores Video input is generated by the VITA 2000 image sensor from ON Semiconductor which is configured for 1080p60 resolution The raw Bayer sub sampled image is converted to an RGB image by an image processing pipeline implemented using LogiCORE IP video cores that remove defective pixels de mosaic and color correct the image A video frame buffer is implemented in the processing system PS DDR3 memory making images accessible to the ARM processor cores via the AXI Video Direct Memory Access VDMA The video frame buffer is not required for the operation of the image processing pipeline but is included in the design to enable the capture of input video images for analysis Figure 1 shows a block diagram of the design Copyright 2013 Xilinx Inc Xilinx the Xilinx logo Artix ISE Kintex Spartan Virtex Vivado Zynq and other designated brands included herein are trademarks of Xilinx in the United States and other countries ARM AMBA and CoreSight are trademarks of ARM in the EU and other countries HDMI and High Definition Multimedia Interface are trademarks of HDMI Licensing LLC MATLAB is a registered trademark of The MathWorks Inc All other trademarks are the property of their respective owners XAPP794 v1 2 January 2
64. these vendor or public websites 39 Silicon Labs CP210x USB to UART Bridge VCP Drivers http www silabs com products mcu Pages USBtoUART BridgeVCPDrivers aspx 40 X Rite ColorChecker Classic http xritephoto com ph_product_overview aspx ID 1192 41 X Rite Judge II http Avww xrite com product_overview aspx ID 1131 Additional Useful Sites for Boards and Kits More information on Zynq 7000 family boards FMC extension cards and other kits based on the Zynq 7000 All Programmable architecture is available here 42 Xilinx Zynq 7000 All Programmable SoC Boards and Kits http Awww xilinx com products boards_kits zynq 7000 htm 43 Zynq 7000 All Programmable SoC ZC702 Evaluation Kit http www xilinx com ZC702 44 Xilinx Zynq Embedded Processors Base TRD wiki page http wiki xilinx com zc702 base trd 45 Xilinx Zynq 7000 All Programmable SoC Video and Imaging Kit http Avww xilinx com products boards and kits DK Z7 VIDEO G htm 46 ON Semiconductor Image Sensor with HDMI Input Output FMC bundle http Awww em avnet com fmc imageon v2000c XAPP794 v1 2 January 2 2013 www xilinx com 57 Revision History Revision History Notice of Disclaimer Automotive Applications Disclaimer XILINX The following table shows the revision history for this document Date Version Description of Revisions 11 22 12 1 0 Initial Xilinx release 11 29 12 1 1 Removed references to targeted reference design T
65. tion Matrix IP core are included in the 1080p60 camera image processing design files that can be downloaded from the ZVIK product page on Xilinx com Select the Docs and Designs tab and look for the ZVIK_Camera_Design_14_2 zip download Additional XAPP794 v1 2 January 2 2013 www xilinx com 25 Running the Demonstration g XILINX information on development of the color correction coefficients can be found in an article titled Image Sensor Color Calibration Using the Zynq 7000 SoC in the Xcell Journal Ref 2 Figure 25 illustrates the effects of white balance on the 24 patch ColorChecker chart for the four different illumination sources Daylight Cold White Fluorescent Incandescent Uncorrected Uncorrected Uncorrected Uncorrected Corrected Corrected Corrected Corrected X794_25 102812 Figure 25 White Balance Color Checker Results The top four images show the output of the image sensor and image processing pipeline with no color correction applied for four different illumination sources There are marked differences in color for all four cases The bottom images show the output with the various color corrections applied for each different illumination source and illustrate how the white balance setting corrects the colors These images were taken in the X Rite Macbeth Judge II light booth using the ZVIK camera design Xilinx provides support for the 1080p60 camera image processing reference design IP cores such as th
66. tively the gamma correction setting can be automatically controlled by one of the Zynq 7000 SoC processors Based on the information received from the image statistics IP core the processor can calculate a look up table that makes the most efficient use of the 0 255 values based on the image intensity histogram The strength slider allows the user to adjust the gamma correction strength applied by the processor XAPP794 v1 2 January 2 2013 www xilinx com 27 Running the Demonstration g XILINX Understanding the Image Statistics The Xilinx Image Statistics IP core provides hardware based image analysis to support auto focus auto exposure and auto white balance applications The third column in the web based GUI provides feedback from the image statistics IP core in the form of histograms as shown in Figure 27 Image Statistics Update Clickto Start Click to Stop Interval msec 2000 Scale 9 Under Exp 05 Over Exp 3 0 Histograms intensity red green blue iN X794_27_121412 Figure 27 Histograms from the Xilinx Image Statistics IP Core Four histograms are displayed The content of each histogram is described by the color bar at the bottom of each histogram The intensity histogram has a gray color bar the red channel histogram has a red color bar the green histogram has a green bar and the blue histogram has a blue bar XAPP794 v1 2 January 2 2013 www xilinx com 28 Running the Demo
67. tween the SDK tool and the ZVIK using Secure Shell SSH by following these steps In the SDK tool select Window gt Open Perspective gt Other 2 Select Remote System Explorer then click OK 3 Inthe Remote Systems tab click the new connection icon as shown in Figure 55 Remote System Explorer Xilinx SDK File Edit Navigate Search Run Project Xilinx Tools Window Help me a eE T a Da A aA Remote Systems 2 Us Team mim B zx F E Local Define a connection to remote system X794_55_102612 Figure 55 Define a Connection to a Remote System 4 Select SSH Only then click Next XAPP794 v1 2 January 2 2013 www xilinx com 48 Rebuilding the Software Applications g XILINX 10 11 12 In the Host name field type the IP address of the ZVIK i e 192 168 1 10 then click Finish In the Remote Systems tab open the 192 168 1 10 section then open the Sftp Files section Double click the Root section In the Enter Password dialog box enter root for both user ID and password Select Save User ID and Save password then click OK If a dialog box appears with a warning about the authenticity of the 192 168 1 10 host click Yes If a dialog box appears with a warning that known_hosts does not exist click Yes Navigate to the mnt directory The contents of the SD card should be visible To open a terminal to enter Linux commands on the ZVIK 1 2 In the Remote Systems tab Select Ssh
68. vice connected through the streaming interface The C_S_AXIS_S2MM_TDATA_WIDTH parameter decides the width of the streaming interface Data received on the streaming interface is then written into the system memory through the memory mapped interface The C_M_AXI_S2MM_DATA_WIDTH parameter decides the data width of the memory mapped interface and C_S2MM_MAX_BURST_LENGTH governs the burst length of the write transaction In this design the stream interface data width is set to 32 bits and the memory mapped interface width is 64 bits The AXI VDMA is used in simple register direct mode which removes the area cost of the scatter gather feature Initialization status and management registers in the AXI VDMA core are accessed through an AX 4 Lite slave interface To get the best possible throughput for AXI VDMA instances the maximum burst length is set to 16 In addition the master interfaces have a read and write issuance of 8 and a read and write FIFO depth of 512 to maximize throughput The line buffers inside the AXI VDMA for the read and write sides are set to 4K deep and the store and forward feature of the AXI VDMA are enabled on both channels to improve system performance and reduce the risk of system throttling For additional information about the AXI VDMA LogiCORE solution and the detailed product guide refer to the AXI VDMA product page on Xilinx com http www xilinx com products intellectual property axi_video_dma htm Video Processing P
69. visible Select the image1 bmp file then right click and select Open With gt Default Editor The image file is now visible in the editor window Switch back to the Debug perspective Open a web browser and enter the address http 192 168 1 10 When loaded the web page invokes the rec mnt www webshot bmp command The debugger is stopped on the specified breakpoint again Disable the breakpoint Click Resume or press F8 The ZVIK camera linux application has successfully been remotely debugged Preparing the SD Card Boot To configure and boot the ZC702 board with a customized hardware design and Linux platform the ISE tools and provided script files should be used to prepare the image files described in Table 13 Image Table 13 ZC702 Evaluation Board Image Files BOOT BIN Image Name Description The zyng_fsbl elf u boot elf and system bit files combined and renamed BOOT BIN to satisfy licensing requirements launch_my_app sh Tells the system what application to launch config_my_ip sh Specifies the default IP address devicetree dtb Device tree blob for booting a RAM disk ramdisk8M image gz BusyBox based Linux root file system zimage Linux kernel Image WWW Web based GUI files www cgi bin zvik_camera_linux_webserver elf CGI application for web based GUI zvik_camera_linux_app elf Main Linux application for camera design T

Xilinx XAPP794 1080P60 Camera Image Processing

Contents

Download Pdf Manuals

Related Search

Related Contents