User Guide
Contents
1. 1 X Jspec cell array of spectral component structures see Table 3 Table 1 Specification of the mixture structure mix_str Field Description Value time_dep Stationarity of mixing indep for time invariant mixing gt dep for time varying mixing mix_type Mixing type inst for instantaneous freq indep conv for convolutive freq dep frdm_prior Degree of adaptability gt free for adaptive fixed for fixed params Tensor of mixing parameters e RIX for mix_type inst corresponding to A from 1 C s lt F for mix_type conv Table 2 Specification of the spatial component structure spat_comps j j 1 Jspat Field Description Value spat_comp_ind Index of the corresponding spatial component 1 Jspat factors 1 x Lj cell array of factor structures Table 3 Specification of the spectral component structure spec_comps j j 1 Jspec Field Description Value FB_frdm_prior Degree of adaptability for narrowband spectral patterns gt free for adaptive fixed for fixed FW_frdm_prior Degree of adaptability for spectral pattern weights gt free for adaptive gt fixed for fixed TW_frdm_prior Degree of adaptability for time pattern weights gt free for adaptive fixed for fixed TB_frdm_prior Degree of adaptability for t2. Cx F x Nx I x I matrix containing the spatial covariance matrices of the input signal in all time frequency bins or F x N single channel variance matrix J number of components here J_spat J_spec K number of NMF components per source transf 7 tnanstonm lsttt ion qerb fs sampling frequency in Hz wien length of the time integration window must be a power of 2 output Tan WO ivalilore SUE initialized mixture structure Trank i Pea My Wy Mi Sila a mix Str CX E mix str transi transi MIx Str TS Wes Mix Str VLON vlen miz strs pat conpsi cein iTiS mir stre Spec conps i esni FIN toe 3 lgd initialize spatial component mix_str spat_comps j time_dep eeniclicpme mix_str spat_comps j mix_type Mice mix_str spat_comps j frdm_prior free mix_str spat_comps j params randn I rank initialize single factor spectral component mix_str spec_comps j spat_comp_ind j mix_str spec_comps j factors S CEGA a factori FB factori FW factori TW factori TB 0 75 abs randn F K 0 25 ones F K diag ones 1 K 0 75 abs randn K N 0 25 ones K N factori FES fran prior freen factori FEW fram prior fixed factors TW fram prior UTTE en factor il g Ws _seieGhin_joseslore E factori TW_ constr NMF Mixes toes pecmcompsni ji tac tone ME atoni end Figure 5 Example of filling of t
3. to compute the input time frequency transform estimate the model parameters and separate the spectral components The headers of these func tions are listed in Figures 2 3 and 4 2Note that in 1 the usage of two factors excitation and filter is described The implementation presented here is more flexible since one can use any number of factors Cj and it reduces to 1 when Cj 2 This is done for convenience of usage For example if one needs to implement an excitation model only or a filter model only direct model one simply needs to choose Cj 1 without bothering to specify and to process an additional dummy factor 3In 1 only the case of four matrices is considered and the case of three matrices W5 U5 G5 is just equivalent to fixing Hy to the N x N identity matrix Since N may be quite big we fix Hy to by convention in the latter case in order to avoid storing a big identity matrix in memory 5 Examples of usage The user should also know how to fill and browse the mixture structure and how to use the above mentioned three functions An example of mixture structure filling and browsing is given in Figures 5 and 6 An example script for the separation of an instantaneous mixture of music signals is given in Figure 7 Function EXAMPLE_prof_rec_sep_drums_bass_melody m contains a more sophisticated example allowing the separation of the following four sources e drums e bass e melody singing voice or leading melod
4. j th spec comp see 1 ft ft L x K Uf Filter spectral pattern weights R ii in the j th spec comp see 1 G Filter time pattern weights goes in the j th spec comp see 1 Hf Time localized filter patterns Re in the j th spec comp see 1 R Set of real numbers Ry Set of nonnegative real numbers C Set of complex numbers 3 Mixture structure The mixture structure is a Matlab structure that is used to incorporate prior information into the framework The structure has a hierarchical organization that can be seen from the example in fig ure 1 Global parameters e g signal representation are defined on the first level of the hierarchy The second level consists of Jspat spatial components and Jspec spectral components Each source is typically modeled by one spectral component although some sources e g drums might be modeled by several spectral components e g bass drum snare etc Furthermore each spectral component must be associated with one spatial component and each spatial component must have at least one spectral component associated to it Compared to the description of the framework in 1 this implementation is more general in the sense that the number of spectral components is 1This extension makes it possible to model the fact that several sources have the same direction which is very often the case for professionally produced music recordings
5. D is updated through EM Ann_PSD_beg opt F x 1 beginning vector of annealing noise PSD def X power 100 Ann_ PSD _end opt F x 1 end vector of annealing noise PSD def X_power 10000 output Of a TIR SN estimated output mixture structure log like arr array of log likelihoods Figure 3 estim_param_a_post_model FASST function for the estimation of the model parame ters function ie separate_spec_comps x mix_str sep_cmp_inds ie separate_spec_comps x mix str sep cmp imde separate spectral components input pa x nchan x nsampl mixture signal Emi Stt input mix structure sep cmp inds opt array of indices for components to separate clei Hil 2 ban 4 Ieee a ie K_sep x nsampl x nchan estimated spectral components images where K_ sep length sep cmp inds is the number of components to separate o i cel e ot Figure 4 separate_spec_comps FASST function for the separation of the spectral component signals fonetion mix Sir lt init_mix_struct_Mult_NMF_inst Cx I ke transi hs wlen Woy asaan Ss i o Seep INEM AN a ition Cee dy IG eaa ey Ene An example of mixture structure initialization corresponding to multichannel NMF model instantaneous case Most of parameters are initialized randomly input
6. Flexible Audio Source Separation Toolbox FASST Version 1 0 User Guide Alexey Ozerov Emmanuel Vincent and Fr d ric Bimbot TNRIA Centre de Rennes Bretagne Atlantique 2 IRISA CNRS UMR 6074 Campus de Beaulieu 35042 Rennes cedex France alexey ozerov emmanuel vincent inria fr frederic bimbot irisa fr April 7 2011 1 Introduction This user guide describes how to use FASST an implementation of the general flexible source separation framework presented in 1 Before reading the user guide you are strongly encouraged to read 1 at least the two first sections This guide is organized as follows Some notations and abbreviations used throughout this document are listed in section 2 Section 3 gives a detailed specification of the mizture structure a Matlab structure used to define the available prior information The main functions the user should know about are listed in section 4 and an example of usage is given in section 5 2 Some abbreviations and notations 2 1 Abbreviations GMM Gaussian mixture model GSMM Gaussian Scaled Mixture Model HMM Hidden Markov Model NMF Nonnegative matrix factorization PSD Power Spectral Density QERB Quadratic Equivalent Rectangular Bandwidth transform S HMM Scaled Hidden Markov Model STFT Short Time Fourier Transform 2 2 Notations F Number of frequency bins in the corresponding time frequency representation N Number of time frames in the corresponding time fre
7. It is implemented by simply adding the power spectrograms of the spectral components corresponding to the same spatial component not necessarily equal to that of spatial components and more precisely Jgpec Jspat The third level of the hierarchy consists in factorizing each spectral component into one or more factors representing for instance excitation and filter structures see 1 Finally on the fourth level of the hierarchy each factor is represented as the product of three or four matrices see Table 4 which are not represented in Figure 1 For instance the factor representing excitation structure is either represented as the product of four matrices Wj U5 G5 Hj representing respectively narrowband spectral patterns spectral pattern weights time pattern weights and time localized patterns see 1 or as the product of threes matrices Wi U5 G5 when H is marked by the empty matrix 3 Almost all the fields of the mixture structure must be filled as specified in Tables 1 2 3 and 4 except those marked by the empty matrix spec _comps 1l spat_comps 1 factors 1 factors 2 spec _comps 2 factors 1 spat_comps 2 spec _comps 3 factors 1 spat_comps 3 spec _comps 4 factors 1 factors 2 Figure 1 Visualization of a mixture structure example 4 Main functions The user should know about three main functions comp_transf_Cx estim_param_a_post_model and separate_spec_comps allowing respectively
8. he mixture structure corresponding to the multichannel NMF method 2 instantaneous case SS Mir str miz str Cx 4 D double transt Mstnt fs 16000 wlen 1024 spat_comps lx1 struct spec_comps lx1i struct Noise_PSD 513x1 double les pene Mesi Bisse ilseil Grenet jie sine gt gt mix_str spat_comps 2 ans time_dep indep mix_type inst frdm prior iree params 2x1 double gt gt mix_str spec_comps 3 arse spat_comp_ind 3 Pale toms Aie struct gt gt mix_str spec_comps 3 factors 1 ans FB 513x4 double FW 4x4 double TW 4x98 double wae T FB_frdm_prior free EW_firdm_prior fixed TW_frdm_prior free Wi ecim oeio TW_constr NMEF Figure 6 Browsing in Matlab of the example mixture structure in Table 5 10 function EXAMPLE_ssep_Mult_NMF_inst data_dir example_data Tes teddy example data file prefix Shannon Hurley Sunrise inst i transf sinus a wlen 1024 nare 84 number of sources NMF_ncomp 4 number of NMF components iter_num 200 load mixture fprintf Input time frequency representation n x fs nbins wavread data_dir file_prefix _mix wav Se ee ae Me mix_nsamp size x 2 compute time frequency representation cz comp transi CL x tranar nien i TS fill in mixture structure mir str init_mix_struct_Mult_NMF_inst Cx msrc NMF_nc
9. ic instrument e remaining sounds from a stereo music recording Due to memory limits in Matlab this function cannot process sound excerpts longer than 30 seconds For full length music recording the function EXAMPLE_prof_rec_sep_drums_bass_melody_FULL m should be used This function simply cuts the full recording into small parts and applies EXAMPLE_prof_rec_sep_drums_bass_melody m to each of them References 1 A Ozerov E Vincent and F Bimbot A general flexible framework for the handling of prior information in audio source separation IEEE Transactions on Audio Speech and Signal Processing vol 20 no 4 pp 1118 1133 2012 2 A Ozerov and C F votte Multichannel nonnegative matrix factorization in convolutive mix tures for audio source separation IEEE Transactions on Audio Speech and Language Pro cessing vol 18 no 3 pp 550 563 March 2010 Field Description Value Cx F x N x I x I complex valued tensor of local c CFXNXIXI mixture covariances transf Input time frequency transform stft for STFT gt gqerb for QERB fs Sampling frequency in Hz 16000 44100 wlen Analysis window length 512 1024 used to compute STFT or QERB in samples Noise PSD F x 1 real valued nonnegative vector of additive E RXF or noise PSD e g for annealing spat_comps 1 x Jspat cell array of spatial component structures see Table 2 spec_comps
10. ime localized patterns gt free for adaptive gt fixed for fixed FB Narrowband spectral patterns Frequency Blobs FXLo FxLf ex ft R ore R corresponding to W5 or W7 FW Spectral pattern weights Frequency Weights EREE ore REES corresponding to U or U g T TW Time pattern weights Time Weights z RES XMI or E pE M corresponding to G or G F T ime i i M xN M xN TB Time localized patterns Time Blobs EREN e RM era corresponding to Hj or H TW_constr Constraint on the time pattern weights NMF no constraint note that nontrivial constraints i e GMM for GMM different from NMF are not HMM for HMM compatible with nonempty time patterns TB gt GSMM for GSMM gt SHMM for S HMM TW_DP_params Discrete probability DP parameters 1x KS 1 x K vector for the time pattern weights needed only when TW_constr NMF of Gaussian weights for GMM or GSMM ex ex ft ft K x KF Kj x K matrix of transition probabilities for HMM or S HMM TW_DP_frdm_prior Degree of adaptability for DP parameters needed only when TW_constr NMF gt free for adaptive fixed for fixed TW_all Matrix of all time weights Nonnegative real valued corresponding to G or Gf from 1 matrix of the same needed only when TW_constr NMF size as TW Table 4 Specification of the spectral comp
11. omp transf fs wlen reinitialize mixing parameters A sini pi 8 sini pi 4 simi 3 pil 3 cos p18 cosi pi 4 cos 34pi s for j Ik nsre mix str spat conpsi i k paransi A N end run parameters estimation with simulated annealing mir ine estim_param_a_post_model mix_str iter_nunm ranmi Ni source separation ie EM separate spat comps x mix Str Computation of the spatial source images fprintf Computation of the spatial source images n for j l narei wavwrite reshape ie_EM j mix_nsamp 2 fs nbins result dir filea pretax u sim int2str j wav I end Figure 7 Example of usage involving all three main functions runs the multichannel NMF method 2 in the instantaneous case 11
12. onent factor structure factors 1 1 L function Che comp_transf_Cx x transf win Len fs qerb_nbin m 0L comp transi C x transit yin lenm fs gerb mbin compute spatial covariance matrices for the corresponding transform input g ac x I x nsampl matrix containing I time domain mixture signals with nsampl samples transf transform Veter i qerb win_len window length fs opt sampling frequency Hz qerb_nbin opt number of bins for qerb transform output mioa Cx F x Nx I x I matrix containing the spatial covariance matrices of the input signal in all time frequency bins Figure 2 comp_transf_Cx FASST function for the computation of the input time frequency transform function miz str log_like_arr estin param a post model mix str inp iter num sim_ann_opt Ann_PSD_beg Ann_PSD_end mix_str log_like_arr estim param a post modelimiz str inp iter num sim ann opt Ann ESD beg Ann RSD end estimate a posteriori mixture model parameters input gt mix _str_inp input mixture structure iter_num opt number of EM iterations def 100 sim_ann_opt opt simulated annealing option def Wann Ino anmi no annealing zero noise ann annealing ann_ns_inj annealing with noise injection mpd ns priii update noise parameters Noise PS
13. quency representation I Number of channels this version is only implemented for J 1 or I 2 Jspat Number of spatial components see Section 3 S spec Number of spectral components see Section 3 Rank of the covariance matrix of the j th spatial component Number of factors in the j th spectral component C 1 direct model C 2 factored excitation filter model L amp Number of narrowband excitation spectral patterns see 1 in the j th spec comp K Number of characteristic excitation spectral patterns see 1 in the j th spec comp Ms Number of time localized excitation patterns see 1 in the j th spec comp ie Number of narrowband filter spectral patterns see 1 in the j th spec comp K Number of characteristic filter spectral patterns see 1 in the j th spec comp M Number of time localized filter patterns see 1 in the j th spec comp E CXR SERN Aj Mixing parameters in the j th spatial comp see 1 FxL W5 Narrowband excitation spectral patterns RI 7 in the j th spec comp see 1 ex EP eK U Excitation spectral pattern weights R 7 in the j th spec comp see 1 f K3 xM 3 3 G Excitation time pattern weights R 7 in the j th spec comp see 1 Mx N Hj Time localized excitation patterns Ry in the j th spec comp see 1 ft Wf Narrowband filter spectral patterns R in the
Download Pdf Manuals
Related Search