SSX3.0 - emotion3D
Contents
1. e g Side by side N B External Preview slows down the plugin significantly 6 3 5 Depth Analysis Depth Analysis The depth characteristics of the current frame are Mar Disparity Limit illustrated in histogram form under the Depth Min Disparity Limit Analysis tab The green vertical line depicts the Show Depth Budget screen plane 1 e point of zero parallax The red and A blue vertical lines mark the user defined background and foreground disparity values set in the Max Min Disparity Limit section above N B changing these values only modifies the appearance of the histogram and does not affect the depth characteristics of the footage Disparity Histogram near LU 6 3 6 Depth Budget Enable Show Depth Budget to view a colour coded impression of the scene relative to the Min Max Disparity Limit positions set above e RED too much positive depth background e BLUE too much negative depth foreground e GREEN indicates regions within the specified comfort zone Figure 14 Depth Budget Preview blue indicates too close red too far 18 2013 O emotion3D All rights reserved emetiondD 6 4 Disparity Generator The Disparity Map Generator renders depth maps irom stereo content The input to this plugin is always a left and right input stream Note that it is recommended to use the Vertical Alignment plugin before the Depth Generator plugin since this can significantly impro2. Figure 10 Manual Disparity Range Popup Overlay 15 2013 O emotion3D All rights reserved 6 3 2 Left Right Image Check There 1s always the danger that the image sources may be flipped left right right left In general flipped input sources can be detected in either of the following ways 1 11 Checking the Depth Map Preview white represents close objects dark represents more distant objects and this guarantees 1 that left image really represents the left image and right image and 11 that the background BG disparity 1s larger than the foreground FG disparity This should always be the case If it appears the other way round close regions are dark the images are flipped and the Left Right Source dropdown selections should be swapped In the Preview Popup mode after aligning the Background the Foreground Disparity slider does not allow the images to move in the correct direction and align the foreground 16 emetiongD v Depth Map Preview scaled Figure 11 Depth Map Preview 2013 O emotion3D All rights reserved emetion D 6 3 3 Depth Visualizer Settings Export 2 3D Top Side Environment TV F Screen et m 1 85 1 Flat f Viewer To Screen 200 m Interocular Spacing 65 00 Measure Metric JA USA 450m Press mouse wheel to i measure distances _ Depth Coloring J Demo Mode _ Markers emetiongD Figure 12 Depth Visualizer The D
3. All rights reserved emetionSD 7 Troubleshooting The Foreground Background Sliders are locked and won t let me align the Images The left right cameras are swapped Reselect the Left and Right sources the other way around Visual Artifacts Here are some techniques to reduce these Keep baseline adjustments small and ideally reductions rather than increases This translates to keeping Disparity Scale at 100 or less N B You can also use Zoom to effect a baseline increase Run the source footage through the Vertical Alignment plugin before other plugins Avoid strong colour differences between the two cameras or colour match first Try selecting deselecting Disable Camera Fusion Setting up the Disparity Range manually Look carefully at the depth map and aim for an image that corresponds to your perception of depth within the scene with closer objects appearing brighter than more distant objects It s safer to set the disparity range too large than too small Experiment with the Dilation sliders under Expert Settings The edges of the images are missing or not clean This is because some information is missing from the scene Use the Zoom slider to move these edges outside the viewable range of the frame Slow Performance Check that a CUDA compatible graphics card is available and the appropriate drivers are installed External Preview External Preview functionality currently onl
4. This plugin provides tools for analyzing your stereoscopic footage as well as tools for making simple depth adjustments which do not need any disparity maps Advanced Workflow Disparity Generator The Disparity Generator plugin renders disparity maps from stereo content Disparity maps are needed to perform depth correction on your stereoscopic content and will be needed in the Depth Grading Advanced plugins Depth Grading Advanced 3D Use the SSX s depth adjustment features to stay within the comfort zone of your 3D displaying environment and to achieve optimal depth impression with the ability to stretch clamp and scale effortlessly 2013 emotion3D All rights reserved View input Disparity AR 3 l k n pd gt de LEAST i n f ES f z emetiondD Conversion Workflow Depth Grading Advanced 2D The Depth Grading Advanced 2D plugin allows you to convert your 2D content into 3D content All you need is a camera image and the corresponding disparity map Once converted you will be able to perform all of the SSX s depth adjustments in this plugin alone The Disparity Smoothing filter helps you to control the edge sharpness versus smoothness of disparity maps Applying this plugin to your disparity maps and optimizing the settings can improve subsequent the results of depth grading In disparity map correction of disparity maps it is often necessary to rotoscope some o
5. 5 6 1 Colour Matching Equal to the stand alone Color Matching plugin this feature enables global color matching between left and right image The colour matching takes place before a new perspective is generated and performs a global histogram matching between both views Option Match left The colour histogram of the right image are matched to fit the color histograms of the left image Match right The colour histogram of the leit image are matched to fit the color histograms of the right image Note It is recommended to perform colour matching as a separate pre processing stage using the Colour Matching plugin 6 5 6 2 Fusion When working with stereo footage information from left and right view can be exploited for generating a new perspective This feature allows you to prefer a certain source for the creation of the resulting views This 1s especially useful when your video material contains disturbing artifacts like lens flare or blurred objects in just one camera image or when your camera images contain local colour differences which cannot be matched by a global colour matcher Off Disable fusion in case the new perspective shall only be created based on one source view See 6 5 6 3 Preferred Source for source view selection Automatic Fusion is only used for creating internal perspective 0 100 Disparity Scale On Fusion is always enabled In contrast to Automatic also external perspectives are created based o
6. Ee ia oo E rire oo terre 17 ee EN DD AR EE EE EE EE EE see 18 e A TE FA OE O E cee antpbledaeSaasee nalpmene tose entantessaloassesseceees 18 TU lona EE EE sagesnsnadeexade 18 cA Dipy sa 19 0 L BRM rne DEPA e R E E E E E E EE R 19 6 4 2 Technical Information the SSX Disparity Format cccccccncoonnnncnnnnnnnnnononnncnnnnnnnnonnnnannnnnnnnnnos 20 6 4 3 Transform Disparity with After Effects Levels Filter iese sesse ss see ee ee ee ee ee 20 6 Depth Gradins Advanecao Dis ass EE N ee A ER EE de 21 631 AAA E ed Ag Ge EE EE 21 6 5 2 Disparity Manipulator oooonnnnnnnncncnnnnnonnnnnnnnnnnonononnnnnnnnnnnonnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnns 22 On Domn i E E oe ee 22 a OEN e a ek EE ae 23 Oa Rendenne Mu Vie esse PR PEE E E EE DO Ee GE 23 OIGO BPE INS AO OE EE AE EE N 23 SE ii AE N RE EE e TE N EE EE N 2 00 beters Advances DI id didas 2l 2013 emotion3D All rights reserved 7 8 emetiondD O Disa TOE es E EN Ee Ge ei ee DE ea eee 28 ONA EE EE oi die RE OE EE EE EE EE 28 6 Dias MES eise se RR todoo EE EO RE De EE se ee 28 Ost Dier CUPO ri E ed di DA EE ee eg GE N 28 6 8 Depth Map Correction WOTrKIIOW ee ee ee Re ee ee RE ee ee ee ee RE EE 29 Tobie SOO ING oie eie n rss 30 Appendix A 3 Party Software ee ee ee ee ee ee ee ee ee ee ee ee ee 32 4 2013 O emotion3D All rights reserved emetiondD 1 Technical Reguirements 1 Hardware a PC with a minimum of 8 GB memory re
7. as rotational vertical and zoom offsets After the analysis step has been finished the summarized results of this analysis can be exported and used for quality control of your stereo footage The data can be exported into a csv file via the Export button in the Export Data section and viewed in any text editor or program with csv import support such as Microsoft Excel 14 2013 O emotion3D All rights reserved emetiondD This plugin allows you to perform simple Depth Grading tasks and provides powerful tools for analyzing your stereo content For disparity analysis you will first need to setup the disparity range for every shot of the original content with the correct min max parallax which correspond to the nearest and furthest objects in the scene In SSX for AE this depth information is stored as keyframes for foreground and background disparity parameters These values can be set automatically by clicking the Process button under the Setup Work Area heading Review the depth maps in each shot and manually correct any incorrect range settings as described below or they may already be imported by the SSX Wizard Manual Scene Setup a b c d e In the timeline go to a frame with either closest foreground or furthest background objects corresponding to extremities in the parallax within the shot you want to work on Make sure the Preview Popup box is ticked Drag the Background
8. depth and disparity interchangeably e Depth LUT manipulation giving artists the ability to change the depth of individual objects within a scene e 2D to 3D conversion from available left images corresponding depth maps e Quality Control monitoring and analysis of geometry and depth to meet broadcasters specifications 2013 O emotion3D All rights reserved emetiondD Contents IS Ets AMM O ED Ga ei A 5 AE AO MR EE N EER EE OE N o A 5 Dell Prod ctAc va hon license CIO Vil as EE rira EE De We OR toda ED OE ts 6 Ed Fo 0 aes il UE RE EE EE OR EE EE EN 7 2 Worklow Esseitrale asse sees ESE SERE DoE EE Oe Ee EE GE ee Ee AG DE ee EE ER Ge re EE ER RE oe ee EG 9 Ll Importe Content imo Alter Eloise ee Ee EE ee EE ei EE 9 sa de ii EE ER EE NE OE ON Ee N 9 A eV Ian Hier ie Es ese e EE EE ie Ge De EE De Ee T EG 9 F ie arias 11 323 oe SUCUT AAPP A 11 do No COMPOstidM Sl mesial dildo 12 6 MS aa Ch IS IC OI title aislarse 13 OL AAA e A T 13 61 1 Setup Colour Matchine Ello entorn rodri 13 6 1 2 Colour Matching as part of Depth Grading Advanced sesse ees ee ee ee ee ee 13 6 2 Venlical AN CMM ss EE SE EE SE NEE N ER AE GE ARE AE ei E DE Ee ee RR actes 14 AE CS CE AR EE PP EE RE N 14 622 Dit E POU sae A PR eo N AE OUE eko RA DA oe Eb DE EE ae 14 gt Dep Cra Basic OD osito lo E EG AO de cosa neces bance co AD ee 15 OL Dipan IR ACS ol eese E 15 032 Jet Rent mac NEO SN EE EE EE ER OE EE 16 O A De ee
9. following additional elements BB 001 Depth Right 4 Resulting View Folder gt B Source L R This Folder contains all the resulting compositions For example when you finish the Vertical Alignment Setup Left Aligned and Right Aligned are created Figure 5 Folder Structure When setting up a SSX plugin the wizard creates a setup composition in which you usually make your adjustments The wizard therefore adds all the necessary sources and a solid layer which contains the Plugin see Figure But why not simply add the plugin to the top source layer The solid layer is important when working with side by side input content and rendering out side by side video If that s not the case you will not necessarily need the solid layer For consistency reasons it will be created independently of your content Source Name Mode T TrkMat Parent Setup Disparity Generator Normal None 001 Left Source Normal None None fof 001 Right Source Normal __ None None Figure 6 Composition Structure 11 2013 O emotion3D All rights reserved ametionab 5 3 Nested Composition Setup The SSX workflow usually requires the use of multiple plugins For example you start with Colour Matching then directly continue with the Vertical Alignment gt and then set up the Disparity Generator plugin with the resulting compositions Often you would like to perform all these steps without rendering out y
10. from a single view plus depth maps This is particularly useful in 2D to 3D conversions for automating the generation of new camera perspectives where depth maps have been created for each frame by a rotoscope paint team This contains the same controls as the Depth Grading Advanced 3D plugin Figure 25 Converting 2D to 3D content 27 2013 O emotion3D All rights reserved emetion D This tool allows you to e Dilate Extend Foreground or e Erode Extend Background your disparity maps The advantages of dilation have already been highlighted in Chapter 6 5 6 6 Dilation With this tool it is possible to dilate erode the whole disparity map or to dilate erode previously selected sub regions of an image which is useful for disparity map correction see chapter Dilation 6 5 6 6 for further explanation The Disparity Smoothing plugin helps to achieve smoother and structure aware disparity maps based on the corresponding colour images The disparity smoothing filter is assigned to the depth map composition and needs the definition of the guidance images e By moving the Radius slider you can control the amount of spatial information that is taken for smoothing e The Threshold slider controls the ratio between noise and blurriness When correcting rendered disparity maps it is often 3 Disparity Clipping 3 0 necessary to rotoscope some objects and assigning new Min Value BG pixel gray values where th
11. next Source Settings Source Type Choose the Source Type of your Footage Depending on the plugin you ve chosen this can be e Separate individual files e Side by Side e Top Bottom Copy settings from You can copy your settings from any existing plugin composition If you don t want to copy any settings select the unlabeled option The checkboxes indicate which settings can be copied If an option is disabled then the setting cannot be copied by the script Note Currently Disparity Manipulator keyframes cannot be copied by the script and therefore have to be copied manually Options Show all sources the wizard hides some specific compositions that are created during the setup process If you need to see all compositions just enable this option a SSx Setup Wizard m Flugin Eh Vertical Alignment 2 1 m Source Settings Source Type Separate Left Source Right Source Copy settings from 000 Setup Depth Generator Scene Cuts m Options AAA ________ Show all sources Continue Min Max Disparities Figure 3 SSX Wizard plugin selection 2013 O emotion3D All rights reserved emetion D Side by side content with the SSX Advanced Plugins Currently importing side by side video content into the advanced plugins is not possible Alternatively you can use our SplitScript to convert your SBS content into two separate compositions Therefore 1 select your SBS
12. results or just continue without rendering When you click Finish the setup will be finalized Figure 4 SSX Wizard instructions and the wizard will return to the initial screen Window Location We recommend putting the wizard next to the project window so that you can always see what the wizard has done It is also recommended to allow enough horizontal space so that no information is hidden 10 2013 O emotion3D All rights reserved emetionSD When you are using the wizard to setup your plugin compositions the project window soon will look something like this Every time you click Continue in the initial screen the wizard creates Name the following elements v EM 000 Vertical Alignment GE 1 The Plugin Folder ma 000 Setup Vertical Alignment Each plugin folder name consists of a number and the plugins dd Resulting Views name f 000 Left Aligned 2 The Setup Composition 000 Right Aligned The setup composition is the one you are working on If you are w E Source L R using After Effects 6 0 the composition will be opened F f 000 Left Source automatically if you re using an older version you have to open it 13 000 Right Source manually with a double click 3 Source Folder v MM 001 Depth Generator This folder contains all sources used by the plugin EE 001 Setup Depth Generator T MM Resulting Views When you click Finish in the instructions window the wizard creates the EA 001 Depth Left
13. Disparity slider This brings up a window with the left amp right images superimposed see Figure As you slide the mouse to the side the images move relative to each other Find the position where the background objects appear to be aligned it s safer to overshoot to the right rather than undershoot The resulting value indicates the maximum positive parallax in the shot Repeat the step above for the Foreground Disparity This time align the foreground objects The resulting value indicates the maximum negative parallax in the shot Look at the Disparity Preview parameter window which shows a small depth map automatically generated by the software Objects closer to the cameras should appear brighter than those far away Readjust the sliders 1f necessary to obtain a grayscale image that corresponds to the distances in the scene Repeat this process for other frames to check that the foreground background extremes within the shot have been covered Apply the largest values in each case to the keyframe at the start of the shot and remove any other keyframes within the shot that were introduced during this process S fh Effect Controls Setup Depth Grading w gt x Mu Composition 000 Setup Depth Grading a 000 Setup Depth Grading Setup Depth Grading 000 Setup Depth Grading 4 000 Right Source 004 Right Aligned 00000 ca Output Mods Cida hu Cida NN MEER w Setup Work 4 Proces v Disparity Ge w Depth Map f Click to r
14. bjects and modify the disparity values where certain areas of the object have not been assigned correctly The Disparity Clipping tool helps you to assign a certain depth range to an object so that as much depth information as possible is preserved An effective way to correct edges of foreground objects that have been cut off e g hairy objects or semi transparent regions between foreground and background is to use the Disparity Structure plugin to dilate or erode your disparity maps This tool is especially helpful when performing corrections on rendered disparity maps e g when your rotoscoped region is not accurate enough 2013 O emotion3D All rights reserved emetion D Before using the SSX plugins video content has to be imported into After Effects AB The input will be stereoscopic content unless using the Depth Grading Advanced 2D plugin You can import your footage into AE by going to File gt Import and choosing the preferred import option Alternatively you can also drag and drop your content from windows explorer directly into AE s project window A wizard helps to guide through the steps required in each plugin To reveal the wizard select the SSX Wizard version script from the After Effects Window dropdown menu and follow the instructions Plugin Choose the plugin you would like to use Based on your previous selection the SSX Wizard will suggest which plugin to use as
15. commended 12 GB b NVIDIA graphics card with 1 GB memory and CUDA Compute Capability 1 2 support or above see 1f your graphics card is supported at http developer nvidia com cuda gpus 2 Software a Windows 7 Vista 64 bit b After Effects CS5 C 5 5 or CS6 0 c NVIDIA drivers version 266 58 or above Jan 18 2011 and later and supported by AE see http www adobe com products aftereffects tech specs html 2 Installation 1 Download SSX installer software via customer portal http www emotion3d tv customers login Customer login required 2 Start the installer and follow the instructions 3 Start After Effects The SSX plugins will be available in the emotion3D gt SSX sub folder within your After Effects Plug ins folder 4 Set AE to 32 bit mode to exploit the full depth accuracy of the SSX plugins 3 Licensing cuvee O Every time the plugin is used in AE it is checked for a valid license If the license is not available or expired the user is prompted to activate reactivate the product in order to retrieve a valid license r No authorized license for this product Would you like to actrvate the product now No Note Please check your PC s date and clock settings before Ll l activation in order to avoid any licensing issues Figure 1 Security Error on first use 2013 emotion3D All rights reserved emetiongD The product activation process license retrieval uses emo
16. content in the Project window 2 goto File gt Scripts gt SplitScript jsxbin Now two new compositions Split Left and Split Right have been created When you have set up the plugin the instructions Sex Wizard interface is shown depending on which plugin you DEPTH GENERATION have chosen Info 1 Reset Anew project Folder 001 Depth Generator has You can reset the wizard at any time This will EE OE AE bring you back to the initial screen The Depth Generator Plugin has been set up successfully pA Steps Please proceed with the following steps To get the best results using the SSX plugins you should follow the recommended workflow steps us HA listed in the Wizard 2 Open the Effect Controls Window Setup the scene by going to Setup Work Area and 3 Help simply press the Process button When you move your mouse over areas showing a o Please check the setup scene cuts min max question mark a help tip is shown with some extra disparity information regarding the step Finally vou can render the depth maps 4 Important Message P Add to Render Queue 20 Important message fields are displayed in red and i Continue without rendering should not be ignored P By diding Finish two compositions left depth and right 5 Finish Settings depth are going to be created When you re done with all the steps you can chose if you would like to continue by rendering the Finish
17. e Output Mode dropdown menu 11 Set the output format within the render settings Output Module of the depth maps to a 32 bit floating point precision format such as OpenEXR N B store uncompressed e g with Zip16 Note that the left and right depth maps are different and correspond to the depths in the left and right source material respectively 19 2013 O emotion3D All rights reserved emetiondD 6 4 2 Technical Information the SSX Disparity Format Disparities generated by SSX are relative disparities The disparity range values 10 10 are mapped to 0 1 when exporting OpenEXR files 1 e e 10 is mapped to 0 e 0 is mapped to 0 5 and e 10 is mapped to 1 Disparities outside this range are also mapped correctly but cannot be seen when watching the disparity map In addition SSX inverts the relative disparities of the left disparity image before a mapping to 0 1 is performed for inverting back use the After Effects Invert effect 6 4 3 Transform Disparity with After Effects Levels Filter Depth maps calculated by SSX usually do not use the whole image range but just a part of it because with 32 bit unlimited depth levels are possible within this range Since some tools require 8 bit maps and therefore are limited in depth levels 255 1t you may want to convert those disparity maps to the whole image range before converting them to 8 bit so that as less depth informatio
18. e automatically generated seed ie values do not produce optimal results in the final colour image renderings The Disparity Clipping tool helps vou assigning a certain depth to an object without losing its depth structure Therefore choose the minimal allowed value background and the maximal value foreground by choosing the colours in a colour picker window or by selecting the colour directly by clicking into the image using the eyedropper tool You can also use the Tolerance slider to relax your colour selection P 5 Tolerance Figure 26 Disparity Clipping Interpolation By default the min and max values are interpolated linearly between multiple keyframes If you don t want them to interpolate set the keyframes to hold Select a colour by clicking into the image itself Figure 27 The eyedropper tool in AE 28 2013 O emotion3D All rights reserved ametionab 6 8 Depth Map Correction Workflow This section describes the disparity map correction possibilities of the SSX plugins and suggests a way to perform disparity map correction on stereo content If the results you ve received during Depth Grading Advanced are not accurate enough you will have to perform manual corrections on the disparity maps At this point After Effects in combination with the SSX Disparity Tools can be a powerful tool to achieve better results Now we are presenting a sample workflow which shows you how dispar
19. emetionSD SSX3 0 USER MANUAL Do not print this page Last Update April 23 2013 2013 O emotion3D All rights reserved emetionSD SSX3 0 User manual Product Description The SSX is a suite of After Effects plugins for manipulating stereoscopic 3D S3D footage in post It enables fixing of stereoscopic 3D S3D shots e g re aligning geometry colour matching between two views or reducing excessive parallax burnt in during filming Independent convergence and parallax modification also allow footage to be adjusted to the depth budget of the intended viewing conditions The sophisticated visualization tools included in the SSX facilitate these processes The most common uses of the SSX are listed below e Colour matching automatically providing a choice of which eye to match to Expert tools support region based matching e g lens flares reflection differences due to camera polarizations e Geometry alignment to automatically remove pan tilt roll vertical misalignments keystone and Zoom mismatch between stereo cameras e Depth grading for smooth depth transitions at shot boundaries and tailoring the depth script to the narrative e Remastering S3D for different viewing conditions e g converting from TV to mobile version e Converting S3D content to multiview formats for viewing on auto stereoscopic displays e Depth Map generation and exporting for depth based compositing and VFX work N B we use the terms
20. epth Visualizer provides various controls to pre visualize the depth impression on a target display A target display can be defined by its screen size distance from viewer to screen and the interocular distance between the viewer s left and right eyes These settings can be defined either directly via the individual controls or indirectly by choosing a displaying environment preset via the Environment selection For visualizing markers for the screen plane and the disparity limits within the 3D Visualizer the checkbox Markers must be enabled Alternatively a depth colourization of the scene can be achieved by enabling Depth Colouring Apart from the 3D representation the amount of your scene that 1s inside outside your target display s comfort zone is displayed at the bottom of the control window Navigation within the Visualizer Use left and right mouse button to rotate and move your 3D scene and the middle button to get distance information for individual points Scrolling via the middle mouse wheel enables zooms in out 2013 O emotion3D All rights reserved emetiondD Export 3D screen to file Click on the export button in the top menu to export a bmp jpg image of your current 3D scene 6 3 4 External Display For displaying your content in 3D on an external 3D display select External Preview and choose the appropriate output options The 3D mode on the monitor must be set to match the selected External Output Mode
21. es We recommend working on one Process Disparity Generator shot at a time as the software is designed to take all the EE frames of the shot into consideration this is particularly 1 Min Disparity FG important to get the best correct results in the final b Max Disparity BG 2 Scene Cut rendered output Cut Inserted Y Depth Map Preview scaled E Figure 9 Automatic Scene Setup In the SSX for After Effects Shot Boundaries are keyframes of the Layer on which the SSX plugin is applied The SSX automatically detects the boundaries and sets the keyframes if the following steps are followed a Select the Work Area in the Timeline panel Shots will only be detected using range of the frames within the Work Area NOTE All Shots overlapping with Work Area will be affected b Select Automatic Scene Setup mode c Click Process under the Setup Work Area dropdown to generate shot scene cut Boundaries d Once the keyframes have been placed review their positions manually in the viewing window and remove any redundant keyframes You may create move or delete existing keyframes in case the real shot cuts differ from the detected ones On pressing the Process button in the Setup Work Area section the Vertical Alignment plugin analyses the footage within the work area regarding scene cuts minimal and maximal disparities and geometric alignment differences between left and right camera views such
22. f the histogram Interpolation between keyframes ini Since scenes will often contain movement it is important to mia iss interpolate disparity manipulations within a scene Therefore the manipulations will be interpolated linearly between every created keyframe Nevertheless interpolation can also be deactivated by setting keyframes to hold 6 5 3 Depth Transitions This feature facilitates smooth handovers in depth between one shot and the next Simply move the time cursor to the first scene of the incoming shot and the handover frames 22 Figure 17 Selecting Point of Interest in the Depth Transition Window 2013 O emotion3D All rights reserved emetiongD will appear in the window Click on the Point of Interest POI in the outgoing shot top and then the POI in the incoming bottom shot This will apply a Horizontal Image Translation HIT to the first frame of the incoming shot so that its POI is at the same position in depth as the outgoing POI By default this HIT 1s gradually relaxed over the first 5 frames of the incoming shot but this can be modified in the timeline by moving the Convergence Shift keyframes appropriately EE T E Ha d gt o BO ODE coos coms oqo ooo 00012 00013 00014 00015 Layer Name Depth Grading 2 0 E Scene Cut 6 b Convergence Shift fof 000 Left Source f 000 Right Source 2n pe Figure 18 Convergence Shift Applied to Incoming Shot Numerous Output Modes a
23. ging the disparity scale for positive or negative disparities only Disparity Manipulator Disparity Manipulator Disparity Manipulator isparity i Disparity Manipulator 1 00 he 1 00 mar A disparity disparity disparity gt Depth Transition gt Depth Transition gt Depth Transition Figure 16 Disparity Manipulation Depth Transition 1 At the top of the histogram a yellow line can be seen This yellow line indicates that the disparity has the original scale 2 Click on the line to create a new keypoint The numbers beneath the keypoint are showing the scaling factor between the keypoint and its left right neighbors 3 Hold the mouse button and move to another position The scaling factor as well as the line colour are changed according to the direction you are moving indicates that the scaling factor is less than 1 0 1 e the region is being compressed e blue indicates that scaling greater than 1 0 i e the region is being stretched 4 After releasing the mouse button the keypoint will be activated in the new position In general the Disparity Manipulator allows the following interactions e Insert a new keypoint by clicking into the grey area at the top of the histogram e Change the position of an existing keypoint by simply dragging it e See the current scaling factor by clicking on a keypoint and by not releasing the mouse button e Delete a keypoint by dragging it over the top border o
24. ided by the copyright holders and contributors as is and any express or implied warranties including but not limited to the implied warranties of merchantability and fitness for a particular purpose are disclaimed In no event shall the Intel Corporation or contributors be liable for any direct indirect incidental special exemplary or consequential damages including but not limited to procurement of substitute goods or services loss of use data or profits or business interruption however caused and on any theory of liability whether in contract strict liability or tort including negligence or otherwise arising in any way out of the use of this software even if advised of the possibility of such damage 32 2013 emotion3D All rights reserved
25. ine executable object code generated by a source language processor THE SOFTWARE IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND EXPRESS OR IMPLIED INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE TITLE AND NON INFRINGEMENT IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE FOR ANY DAMAGES OR OTHER LIABILITY WHETHER IN CONTRACT TORT OR OTHERWISE ARISING FROM OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE OpenCV Computer vision License Agreement For Open Source Computer Vision Library Copyright C 2000 2008 Intel Corporation all rights reserved Copyright C 2008 2010 Willow Garage Inc all rights reserved Third party copyrights are property of their respective owners Redistribution and use in source and binary forms with or without modification are permitted provided that the following conditions are met Redistribution s of source code must retain the above copyright notice this list of conditions and the following disclaimer Redistribution s in binary form must reproduce the above copyright notice this list of conditions and the following disclaimer in the documentation and or other materials provided with the distribution The name of the copyright holders may not be used to endorse or promote products derived from this software without specific prior written permission This software is prov
26. ing options Figure 7 Colour Matching Interface e Disable No colour matching is performed e Match Left The right image is colour matched so that it fits the left colour scheme e Match Right The left image is colour matched so that it fits the right colour scheme If the following scene should not be matched go to the beginning of this scene and set Colour Matching to Disabled w Effects w Colour Matcher 2 1 Left Source 2 000 Left Source Right Source 3 000 Right Sourc Output Mode Side by Side te Colour Matching Match Left About Figure 8 Setting keyframes in After Effects Timeline If preferred Colour Matching can alternatively be performed as part of the Depth Grading Advanced 3D plugin You can find the Colour Matching option under Expert Settings 13 2013 O emotion3D All rights reserved emetiongD The Vertical Alignment plugin can be used for correcting vertical misalignments as well as pan tilt roll and keystone effects between the left and right camera images After the correction each point in the left image 1s located on the same horizontal line as the corresponding point in the right image Vertical alignment of the images 1s normally done as a second step and will improve the results of the depth map generation and any further depth based processing O mode RE s o Y Setup Work Area The whole clip in the composition needs to be split into individual shots scen
27. ity map correction with SSX Disparity Tools could look like In the following it is assumed that you ve already rendered your disparity maps l First of all open your depth maps To see how your changes affect the result we recommend setting up the Depth Grading Advanced 3D Plugin as the next step Please configure your plugin as follows Go to Expert Settings and set Preferred Source to Prefer Left This will save you considerable effort since you will mostly have to correct the left disparity map only When now working on your disparity maps you will be able to immediately see the impact of your changes Now try to achieve better results by using the Disparity Smoothing plugin Experimenting with the slider values may increase the visual quality of the results Assign the Disparity Structure plugin on the disparity map and experiment with the dilation slider If you see no improvement remove it Artifacts on objects must be corrected the easiest way is to use AE s rotoscoping functionality a Take the original colour image and mark the desired object with the Roto Brush Tool b When finished click Freeze to fix your selection c Now duplicate your disparity layer d Copy the frozen Roto Brush to it 6 Now use the clipping tool and assign min max values Sometimes you will have to perform steps 3 7 multiple times on different objects until the result looks like desired 29 2013 O emotion3D
28. n as possible is lost For this purpose we recommend using the After Effects Levels filter Short Explanation For every scene you will have to adjust the black and white values so that the histogram fits into this border Leaving some buffer is recommended since most scenes are likely to change slightly from frame to frame otherwise some depth precision 1s lost 20 2013 O emotion3D All rights reserved emetion D 6 5 Depth Grading Advanced 3D Use the controls in the Scene Adjustment group to modify the volume and position of your content in 3D space This plugin requires as input the source footage plus corresponding left and right depth maps as rendered by the Disparity Generator plugin The configuration is depicted in Figure 15 N B The left and right disparities have to be consistent for correct results Stere 6 51 Scene Adju stment Setu p Figure 15 Rendering from Stereo3D Z Goal Modify the parallax of the content to fit the desired depth budget The plugin creates 2 virtual cameras from the input left right images whose parallax can be adjusted via the controls and visualized in several ways N B this is not a simple H I T Steps a Specify under Min Disparity and Max Disparity the disparity limits for your target display e g Sky TV s budget is between 1 and 42 b Look at the parallax visualization tools and move the sliders Convergence Shift Dis
29. n both input views 6 5 6 3 Preferred Source One of the following options can be chosen Prefer Closest default This mode will always pick the spatial closest pixel for the creation of the resulting view This typically 1s the recommended setup 1f fusion 1s enabled Prefer Left The resulting view is mainly composed of pixels of the left image only areas which cannot be filled with the information of the left image will contain pixels of the right image Prefer Right The resulting view is mainly composed of pixels of the right image For setting this feature to Prefer Left or Prefer Right it is very important that the disparity maps between the images match Otherwise visual artifacts may occur in the resulting views due to fusion 24 2013 O emotion3D All rights reserved emetionSD 6 5 6 4 Fill In Holes In case no information is available for generating a region in a new perspective e g disoccluded regions that are not available in both source camera views an inpainting algorithm aims to fill in missing holes Disable Disables infilling of missing regions Use this feature in case you want to fill in the missing regions yourself using for example Adobe Photoshop s context aware region filling Occlusions Only holes that were generated by disoccluding regions in the source camera images are filled This is the recommended setup for most use cases such as parallax adjustment or generation of stereo pers
30. our video material We call this a nested composition setup Unfortunately nested compositions have drawbacks e When setting up nested compositions every plugin has to check out the frames from the preceding plugin which can slow down the project especially when using multiple compositions in a row e When using the resulting views of the Disparity Generator as input for the Advanced plugins additional memory needs to be allocated which leads to a very high memory consumption To overcome these drawbacks we generally recommend rendering the intermediate results before continuing with another plugin especially after Disparity Generation 12 2013 O emotion3D All rights reserved emetion D The Colour Matching plugin can be used for correcting colour differences between left and right camera images Due to physical occurrences like differences in camera lenses reflections etc there are almost slight differences between the two camera images After colour matching 1s performed the colours of left and right image and their histograms should look almost the same Left Source 2 0 00 Left Source When applied to a sequence the whole clip is colour Right Source EA di AE matched by default If you would like to colour match a a PE ae ty ae i l 5 Colour Matching Disable certain scene go to the desired location in the timeline and e then click on the stop watch at Colour Matching and select one of the follow
31. parity Scale and Zoom to fit your depth budget specified in the previous step c View the results 1 in 3D on an external 3D display select External Preview and choose the appropriate option The 3D mode on the monitor must be set to match the selected External Output Mode e g Side by side N B External Preview slows down the plugin significantly 11 in 2D e g side by side delta anaglyph etc either on the working monitor depending on the Output Mode or on an external 2D monitor or TV depending on External Preview settings Other controls Disable Camera Fusion use a different method for generating the new camera perspectives use this if the image rectification is imperfect or if there are colour difference between the original left and right cameras Zoom the edges of the generated new camera views sometimes show artifacts or distortions which can easily be removed by zooming in the final composition This zooming is done in the center of the frame If you would like to see a specific part of the zoomed image you can set the Translation X and Translation Y sliders 21 2013 O emotion3D All rights reserved emetionSD 6 5 2 Disparity Manipulator This tool helps you to easily stretch and compress certain disparity ranges In contrast to complicated lookup tables this tool allows powerful depth manipulations based on the disparity histogram directly A common use case is chan
32. pectives from 2D footage Occlusions Alpha Fill all disoccluded regions and also all pixel regions with alpha values smaller equal 0 0 This option can be used in combination with 6 5 6 5 Clipping of small regions 6 5 6 5 Clipping Function The Clipping Function uses the depth map information to selectively hide regions from the foreground and or background move the Foreground Clipping value closer to zero in order to increasingly hide the foreground conversely move the Background Clipping value closer to zero in order to increasingly hide parts of the background d y d em ae a e al Ho e Figure 21 Clipping Function Isolating Foreground 25 2013 O emotion3D All rights reserved emetionSD 6 5 6 6 Dilation Figure 22 Dilation Applied to Clipped Image An effective way to correct edges of foreground objects that have been cut off e g hairy objects or semi transparent regions between foreground and background is dilation of the depth maps The SSX provides dilation sliders in the expert settings for extending the objects in the depth map horizontally X or vertically Y For example if a part of a foreground object in the depth map is wrongly labeled as background than this part would be cut off in the virtual perspective views For example in Figure 23 the right part of the roof is wrongly labeled as background and therefore cut off By increasing the horizontal dilation from 0 to 6 we extend the foregro
33. re supported such as side by side top bottom crosseye multiview see below etc The selected choice is visible in the AE composition panel and also used for rendering Output Mode Multiview mode is used to render additional camera Y Multiview views for autostereoscopic displays that require more A than two views In order to obtain all the views it is necessary to render multiple times with different output y Camera ID camera view numbers v Camera Position a In Output Mode parameter select Multiview EE b Open View Adjustment group a new section T Setup Work Area Multiview appears Figure 19 Multiview Settings c Set Number Of Cameras 8 by default slider d Drag Camera ID slider to preview each camera position e optionally Adjust Camera Position for finer virtual camera control Y Expert Settings Colour Matching Disable Preferred Source Prefer Closest e Fusion Automatic 7 A number of additional controls are provided for fine 5 Fill In Holes Occlusions tuning the results of the depth map and new vw Dilation X perspective generation algorithms 0 F Dilation Y Enable Depth Clipping b 23 i Figure 20 Expert Settings 2013 emotion3D All rights reserved emetiondD Clicking Disable Fusion is recommended if there is considerable colour or lens distortion mismatch between cameras The other controls are described in the following sections 6
34. tion3D License Manager Figure 2 to retrieve a license It starts automatically while using the plugin without valid license or 1t can be run from Windows Context Menu at Programs emotion3D SSX version emotion3D License Manager r Product Information oO Name 2 Version 13 License Issued date j der 2012 Expiration 01 an 2013 hostid ABCDEFG1234567 Activate Deactivate l 1 N Tel 443 1 293 32 63 A il he Figure 2 emotion3D License Manager The license manager displays product and license information activation buttons and contact references 99 66 Please click on the or links to report any difficulty experienced while activating or using the software Proxy Server If you are using a proxy server you will have to enable the Proxy Settings 5 option and enter your connection settings 2013 O emotion3D All rights reserved Left input Right emetionSD 4 SSX 3 0 Overview Simple Workflow Colour Matching Colour Matching can be used for correcting colour differences between left and right camera images Vertical Alignment Camera misalignments such as camera roll pan or tilt differences between two stereoscopic cameras are among the most common stereoscopic issues and should be corrected prior to generating disparity maps or adjusting the depth o G O 09 O e Depth Grading Basic 3D
35. tup exe for installing the SSX Using the msi file will install the plugin but not install C runtime After Effects runs out of memory when trying to render When rendering with high resolution footage like 4K content ultra HD the memory consumption of the SSX plugins can get very high Make sure you have enough memory installed For minimal memory requirements see Section 1 Technical Requirements For other questions contact support emotion3D tv 31 2013 O emotion3D All rights reserved emetiondD 8 Appendix A 3 Party Software The SSX uses the following gru party software libraries and their licenses Name Description License Boost Templates Boost Software License Version 1 0 August 17th 2003 Permission is hereby granted free of charge to any person or organization obtaining a copy of the software and accompanying documentation covered by this license the Software to use reproduce display distribute execute and transmit the Software and to prepare derivative works of the Software and to permit third parties to whom the Software 1s furnished to do so all subject to the following The copyright notices in the Software and this entire statement including the above license grant this restriction and the following disclaimer must be included in all copies of the Software in whole or in part and all derivative works of the Software unless such copies or derivative works are solely in the form of mach
36. und object in the depth map In the novel view this removes the cut off region N a Dilation X 0 b Dilation X 3 c Dilation X 6 Figure 23 Dilation for correcting foreground object edges 26 2013 emotion3D All rights reserved ametionad 6 5 7 Rendering If you use the Wizard for setting up your projects this will automatically add appropriate compositions to the render queue if you allow it Alternatively you may set up renders manually as follows a Verify that the Output mode is set to your desired format side by side top bottom etc b Switch off Show Depth Budget c Add the composition to the render queue d In the Render Queue tab review the output filenames and location e Click the setting in the Output Module and change the Format in the pop up window to OpenEXR Click on Format Options and check the 32 float box Select a non lossy compression format such as Zipl6 f Then click render to generate a video sequence Figure 24 N B the first image in a sequence always takes longer to render than the remaining ones 5 Fill in Holes Render Queue Current Render Elapsed Est Remain Render Render Comp Name Status Started Render Time GN E output_views Queued Log Errors Only F Output To F side by side 4442222 tif Figure 24 Render Queue 6 6 Depth Grading Advanced 2D This mode allows rendering of 2 or more views
37. ve the quality of the resulting depth maps 6 4 1 Rendering Disparity Depth maps can be exported via rendering for further processing in many other software programs e g Photoshop 3DS Max The most common use is in conjunction with compositing and VFX tools such as Autodesk Smoke and Nuke Alternatively the depth maps can be modified in After Effects or other software and re imported into the SSX plugin for rendering see following sections Common modifications include 1 manual improvement of the depth maps with edge correction sharpening or softening 11 re colouring the depth map for artistic 3D effect such as moving individual objects or regions forward or backwards in relation to the rest of the scene N B modified depth maps should only be imported into the SSX in the Depth Grading Advanced 2D mode because the correspondence between original left and right images is no longer represented by the modified depth map When rendering multiple times from the same source images e g with multiple parameter settings speed improvements can be achieved if the depth maps are rendered once as this is a relatively slow process then re imported and used multiple times in separate compositions that can be added to the render queue As the depth maps have not been modified in this case the Depth Grading Advanced 3D mode is recommended To render a sequence of Depth Maps 1 Select Left Depth or Right Depth from th
38. y works properly when used as a single window on a separate screen display e g on a second 3d capable monitor It is intended for real time preview of the generated views If you put other windows on top of it on a single monitor it may not redraw if so navigate back to AE main window Alt Tab and refresh it Error message OpenCL not available This may appear when the plugin is applied to a layer CUDA support has not been found and consequently the rendering takes very long Make sure that appropriate CUDA drivers are installed http www nvidia com Download index aspx Error message invalid Filter Symptoms An invalid effect error pop up shows up in After Effects when applying a plugin to a composition layer Cause This problem occurs in case of a missing dll library file Possible solutions 30 2013 O emotion3D All rights reserved emetiondD 1 If the error only shows up for the Vertical Alignment plugin but not for the other plugins C windows system32 opencl dll is probably missing This library typically comes with your graphics card drivers and for solving this issue install the latest version of your graphics card driver on your system Please make sure the newest graphics card drivers are installed and your graphics card fits the SSX requirements 2 Please assure you 1 have Admin rights when installing the SSX so all dll files can be copied to your system directories and 11 use se
Download Pdf Manuals
Related Search