Home

Synthesizer

1. 84 609 A yep Foe e E 6 916 978 B2 7 2005 Georges etal 84 609 627 gopulos et al 6 970 822 B2 11 2005 Fayetal 704 270 5 633 985 5 1997 Severson 395 2 76 7 078 609 B2 po Gon SEHEN 84 645 5 640 590 A 6 1997 Luther 395 806 7 183482 B2 2 2007 Kobayashi 84 645 5 648 628 A 7 1997 Ngetal 84 610 7 189 915 B2 3 2007 Kobayashi 84 645 5 650 583 A 7 1997 Machover et al 84 626 7 241 947 B2 7 2007 Kobayashi 84 645 5 655 144 8 1997 Milne et al 395 807 7 319 185 1 2008 Wieder 84 609 5 675 557 A 10 1997 Hubinger 369 4 7 365 260 B2 4 2008 Kawashima 84 600 5 689 081 11 1997 Tsurumi 84 609 2001 0025561 Al 10 2001 Milburn et al 84 609 5 753 843 A 5 1998 84 609 2002 0023529 Al 2 2002 Kurakake et al 84 610 5 763 804 A 6 1998 Rigopulos et al 84 635 2002 0033090 Al 3 2002 Iwamoto et al 84 609 Rire desc Er eus 2002 0046315 Al 4 2002 Miller et al 792 imis et al 2002 0046899 Al 4 2002 Mizuno etal 181 142 5 801 694 A 9 1998 Gershen 345 339 2002 0065074 Al 5 2002 Cohn et al m o iui e Span 2002 0166440 Al 11 2002 Herberger et al 10945 OVESSON js 2002 0170415 Al 11 2002 Hruska et al 5 839 108 11 1998 Daberko et al 704 270 2002 0175665 Al 11 2002 O Grady et al 5 850 051 A 12 1998 Machover etal 81 634 2003 0013497 Al 1 2003 Yamaki et al
2. 84 611 WOS8 05200 7 1988 6 230 140 5 2001 Severson WOS89 02641 3 1989 6 245 984 6 2001 Aoki etal 097 15043 4 1997 6 281 424 BL 8 2001 Koike et al 84 636 Wo WO97 35299 9 1997 6 326 538 B1 12 2001 WO WO98 33169 7 1998 6 343 055 1 2002 Emaetal 369 531 6 WO WO01 63592 8 2001 6 347 998 2 2002 Yoshitomi et al 463 42 WO WO01 73748 10 2001 6 353 169 B1 3 2002 Juszkiewicz et al 84 600 WO WO01 86630 11 2001 6 353 172 3 2002 Fayetal 84 609 WO WO0186625 11 2001 6 353 174 3 2002 Schmidtetal 84 609 WO WO0186626 11 2001 6 390 923 Bl 5 2002 Yoshitomi et al 463 43 WO WO0186627 11 2001 6 424 944 7 2002 Hikawa 704260 WO WO0186628 11 2001 6 425 018 Bl 7 2002 Kaganas etal 710 1 WO WO0186629 11 2001 US 7 847 178 B2 Page3 WO WOO02 077585 OTHER PUBLICATIONS Beatnik Audio Engine White Paper 6 pages 2001 Beatnik AudioEngine 2 pages 2002 Beatnik mobileBAE version 02 02w 2 pages 2002 Beatnik mobileBAE version 11 02w 2 pages 2002 Combining musical theory and practice IBM Computer Music Cen ter apparently Jun 29 1995 Dream Atmel Company SAM9407 Programmer s Reference pp 1 61 Rev 11 Dec 1996 Dream an Atmel Company SAM9707 Integrated Sound Studio pp 1 20 Jan 1998 GenJam An Interactive Genetic Algorit
3. 1 further comprising providing a USB interface configured to transfer audio files 11 The method of claim 1 wherein the music algorithm comprises a composition algorithm 12 The method of claim 1 further comprising providing a user interface supporting user interactivity with the music algorithm wherein the user interface is comprised of a dis play of instrument lanes 13 The method of claim 1 further comprising providing a user interface supporting user interactivity with the music algorithm wherein the user interface is comprised of a dis play of animated sound waves or animated pulses 14 The method of claim 1 further comprising providing a user interface supporting user interactivity with the music algorithm wherein the user interface is comprised of colorful patterns having movements or shapes following the beat of specific instruments
4. 65 10 wherein these elements are interconnected as shown The memory elements 37 38 and 43 can be made of one compo nent or several physically distinct components Processor 36 is in communication with the memory elements and is able to select according to certain criteria musical tiles out of the library of musical tiles or is able to compose automatically according to the automatic composition algorithm a melody out of the database stored in memory 37 The automatic composition algorithm also utilizes the sound samples stored in memory 38 which may include some speech sentences in such a way that processor 36 delivers in synchronism on its outputs a control signal M1 connected to synthesizer 39 and a sound sample control signal S2 Output signal M2 of the synthesizer and sound sample control signal S2 are then summed and converted to analog form in circuit 43B that provides the complete audio signal MA3 for connection to a speaker or speakers not shown In a similar way the output ofradio receiver 64 can be mixed upstream as a digital signal or downstream as an analog signal of circuit 43B to add a supplementary sound source to the complete audio signal The audio signal forms the output of the inven tion that can then be played by the aforementioned speakers in a stereo system In a preferred embodiment of the invention the processor is made of a microprocessor or microcontroller linked to one or several memo
5. B2 Process Sound Sample 71 Provide Audio Sample Data to Processing Resource Synchronize Playback of Audio sample Data and MIDI Information based on Musical Bar Timing 12 US 7 847 178 B2 1 INTERACTIVE DIGITAL MUSIC RECORDER AND PLAYER This application is a continuation of U S application Ser No 11 705 555 filed Feb 10 2007 now U S Pat No 7 504 576 which is a continuation of U S application Ser No 10 634 346 filed Aug 4 2003 now U S Pat No 7 176 372 which is a continuation of U S application Ser No 09 691 314 filed Oct 17 2000 now abandoned which is a continu ation in part of U S application Ser No 09 690 911 filed Oct 17 2000 now abandoned which claims priority on French Patent Reg No 99 13036 tiled Oct 19 1999 and US Ser No 09 691 302 filed Oct 17 2000 now U S Pat No 6 392 133 which claims priority on French Patent Reg No 99 14420 tiled Nov 17 1999 BACKGROUND OF THE INVENTION a Field of the Invention The current invention relates to a device that provides for interaction with a user during recording playing composing and modifying musical selections b Description of Related Art Various devices capable of composing a musical piece automatically have been developed such as that described in U S Pat No 4 399 731 These devices are based on synthe sizers of instrumental sounds on which are applied for instance as per the MIDI Musical Ins
6. either extracted from a music library or are generated by an automatic composition function The resulting sounds are used either during the video recording or during the video playback The present invention represents an improvement of the aforementioned devices by simulating the reception of a radio station by the use of one or several of the aforementioned functions Further the invention permits selection of musical pieces by a user either from a library in which musical pieces are stored as compressed musical files MIDI files or other similar types of files from pieces recorded from the output of aradio receiver or from pieces that are composed ina pseudo random fashion using a synthesizer function to play original musical pieces Thus the present invention provides for selection accord ing to a pre selected musical style in a pseudo random fash ion or according to a pre defined criteria of audio files to be played from a speaker wherein the audio files meet the pre defined criteria and are either extracted from the library or generated by an automatic composition function Further the recording of or the automatic generation of sentences that mimic the speech of a disc jockey or of an announcer permits combination of speech passages with the musical pieces being played thereby giving the user the illusion that he is listening to an actual radio station BRIEF DESCRIPTION OF THE DRAWINGS The above objects and other
7. instrument press Effects key together with joystick up or down To change the resonance of the filter on the current instru ment press Effects key together with joystick left or right Press stop to mute the instrument Press play pause to un mute the instrument In the sample tunnel Use Joystick left right to select a sample Press Joystick up to go back to music I way To change the sample volume press Effects key together with joystick up or down To choose a different sample effect press Effects key together with joystick left or right Press play pause to play the sample If sample playback had been previously disabled see below the first press on play pause will re enable it Following presses will play the selected sample Pressing stop will disable the automatic playback of samples by the e DJ when returning to I way mode 20 30 40 45 50 55 65 12 In the microphone tunnel Use Joystick left right to select the active microphone built in or docking station If the docking station is not con nected no selection is available Press Joystick up to go back to music I way To change pitch on the microphone input press Pitch Tempo key together with joystick up or down To change the microphone volume press Effects key together with joystick up or down modify the microphone echo press Effects key together with joystick left or right Press stop to mute microphone Press play paus
8. lower it Press play pause to pause the music Press stop to stop the music and go back to style selection screen Press forward to start a new song Press backward to restart the current song Press Save Edit key to save the current song as a user song on the SmartMedia card To select a specific lane use joystick left right The image on the screen will shift left or right and present the I way seen from the new lane To change the relative volume of the current lane press Effects key together with the joystick up or down To change the reverb on the current lane press Effects key together with the joystick left or right To enter the Underground mode for the current lane and loop the current sequence press joystick down The transition to music tunnel is marked by a screen ani mation and the under ground mode is entered for the selected instrument or sound source sample or microphone The U ground screen shows the inside of a tunnel with sound waves on both sides and beat pulses coming towards the user The shape of the far side of the tunnel is modified with the sound effect applied The music will loop as long as the user remains in the Under ground mode In instrument tunnels Drums Bass Riff and Lead Use Joystick right to select a new music pattern Use joy stick left to come back to previous patterns Press Joystick up to go back to music I way To change the cutoff frequency of the filter on the current
9. memory contained within the synthesizer 39 The memory may be RAM or flash as indicated previously Various processes may optionally be used to reduce the memory requirements to store the samples The following examples are not limiting a once the recording is over the silent periods which precede and follow the useful portion of the recording can be eliminated automatically b a speech compression algorithm such as Adaptive Dif ferential Pulse Coded Modulation ADPCM 24 can then be applied to the useful portion of the recording In the process of automatic composition of music the processor can treat the sound sample signals as a special case of a musical instrument Thus the association of sound sample signals with the notes sent by the instruments is an integral part of the automatic composition process The pro cessor then extracts information from the available pool of sound samples in the same way as from the music database However the processing of sound samples has certain fea tures First ofall the invention is not restricted to using sound samples that have been pre recorded for a given type of instrument but also envisions using sound samples that have been recorded freely by the user In addition to some special effects that will be described later the processing of sound samples consists of selecting a sample in the pool of available samples according to a pseudo random sequence then select ing the moment to sta
10. program This permits easy update for the music database and the microprocessor program The synthesizer 39 and the microprocessor 36 of FIG 11 cooperate to select banks of sound samples according to predetermined processor instructions to provide the synthe sizer output S3 The sound sample banks contain sounds in digital form of predetermined instruments The processor 36 can also be used to select the sound source or sources described above Furthermore the proces sor can be used to select commands which permit activation functions such as playing a recording selecting and mixing an audio source or controlling any other functions that are commonly found in video recorder based equipment This combines the functions of blocks 57 and 59 of FIG 10 According to another feature of the present invention a device is provided that simulates a radio station including a player of musical pieces either recorded and digitized or synthesized According to the functional diagram of FIG 12 atypical embodiment ofthe invention comprises essentially a processor 36 a memory 37 containing a music database for use by an automatic composition algorithm a memory 38 storing the sound samples a musical synthesizer 39 a sum mation and digital to analog conversion circuit 43B a radio receiver 64 and a memory 43 internal or external to the invention containing a library of digitized musical files 20 25 30 35 40 45 50 55 60
11. sample Release the record button to end the recording Recording is stopped automatically if the sample duration exceeds 30 seconds 5 20 25 30 40 45 50 55 60 65 14 The record source is chosen automatically depending on the operation mode While e DJ is playing I way or U ground modes the record is inactive If no music is playing the record source is the active microphone built in or docking station If music is playing songs or radio the record source is a mix ofthe music and the microphone input if not muted To enter Virtual radio mode press the v Radio direct access key The LCD will display v Radio in the status line and a list of available station presets to chose from as well as t currently tuned frequency If no preset has been stored only the currently tuned frequency is displayed To select the desired station use Joystick Up Down To listen to the selected station press the play pause key TheLCD will display the radio screen The name ofthe radio station or frequency if it is not a stored preset is scrolling in a banner in the center right part of the LCD An animation representing radio waves is also displayed The status line shows the tuned frequency Use Joystick left right to go to previous next station is in the preset list Press play pause to mute the radio Press stop to go back to station preset selection screen Press forward to tune up Press backward
12. synchronized with the MIDI information at a predetermined time between a begin ning and an end of a musical bar In accordance with the present invention an interactive digital music device can provide multiple features for a user The interactive digital music device can include program mable memory for storing digital audio and voice samples receivers for obtaining external sound signals means for playing the stored audio and voice samples and external sound signals means for storing automatic musical compo sition instructions whereby the user is able to create unique musical compositions and means for mixing external sound US 7 847 178 B2 15 signals with the unique musical compositions and stored audio and voice samples to provide altered audio and voice presentations The interactive digital music device also can include means for providing visual effects complementing the altered audio and voice presentations The interactive digital music device also can include means for providing visual effects including visual means for providing written song lyrics The means for providing visual effects can include visual means for providing complimentary light pat terns for the altered audio and voice presentations The pro grammable memory can include flash memory and the received radio signals unique musical compositions and altered audio and voice presentations can be digitized The interactive digital music device can include a
13. the docking station has a standard MIDI interface with in and out connectors This interface is useable for communication with a personal com puter for audio file transfer or firmware upgrade The invention disclosed herein is useable by persons hav ing no appreciable musical expertise It presents a new way of listening to music wherein a user may modify existing music compose new music and save and play back music Music patterns composed by the user or taken from favorite com posers may be retained in a music database or library The an 5 20 40 45 50 55 60 65 6 composition device of the present invention utilizes a combi nation of notes rather than sounds Notes being represented digitally require less memory for storage than sounds that are usually represented in analog form and take large amounts of memory comparatively when digitized Further digital notes are easily altered while digital sounds are not readily changed As a result the instant invention may use a microprocessor operating at about 25 MHz and needs only about two 2 megabytes of storage although of course the particular clock speed and storage size are exemplary embodiments and the invention is not limited thereto This may be seen to be much less processor speed and memory than used in the least capable desktop computer for example According to the functional diagram of FIG 3 one typical embodiment of the automatic compositio
14. to tune down Pressing the Save Edit key in the station preset selection screen enters the preset edit mode Pressing the Save Edit key while a station is playing will store it in the preset list To exit v Radio mode choose any other mode by pressing the appropriate direct access key As long as no music is playing and in the v Radio mode the microphone settings are made as follows To change the pitch on the microphone input press Pitch Tempo key together with joystick up to increase the pitch or down to lower it To change the microphone volume press Effects key together with joystick up down To modify the microphone echo press Effects key together with joystick left right Reference is also made to co pending U S App Ser Nos 09 691 302 and 09 690 911 filed on Oct 17 2000 for Virtual Radio and Automatic Soundtrack Generator both of which are incorporated by reference herein and both of which may utilized alternatively with embodiments of the present invention Referring to FIG 13 in accordance with the present inven tion a method of playing music using a music algorithm can be provided As illustrated in box 70 MIDI information is processed and the processed MIDI information is provided to a processing resource As illustrated in box 71 one or more sound samples are processed and audio sample data 15 pro vided to the processing resource As illustrated in box 72 playback of the audio sample data is
15. 10 Sheet 1 of 8 US 7 847 178 B2 18 17 il FIG 1 U S Patent Dec 7 2010 Sheet 2 of 8 US 7 847 178 B2 24 22 21 ed e Musical ADPCM amp Compression Parametric Decoder USB Interface 28 Digital 3 Multi media prs Platform aphic 29 Sound FX Synthesizer CODEC Structure with MIDI Interface Keyboard 26 LCD Interface TV Video Flash FM dics Digital Interface Memory Receiver 33 volts Tuner 21 19 23 33 32 FIG 2 U S Patent Dec 7 2010 Sheet 3 of 8 US 7 847 178 B2 37 39 Music MI M poma Sound Samples 38 O Processor FIG 3 39 42A Processor DAC SA 42B FIG 4 U S Patent Dec 7 2010 Sheet 4 of 8 US 7 847 178 B2 Processor O U S Patent Dec 7 2010 Sheet 5 of 8 US 7 847 178 B2 847 178 B2 US 7 Sheet 6 of 8 Dec 7 2010 U S Patent 9 jndjngo C9 DIA IV ynduy uno 19 orsnur ans 75 ampo UOrjeJouor punog 2 KOW mpoW yoeqKe d 810 8 1 induy cA punog ode punos eu1o xq 9881016 O9pIA I9 8 9S t U S Patent Dec 7 2010 Sheet 7 of 8 US 7 847 178 B2 Sound Samples Memory 38 37 39 Music Database Digitized Musical Files U S Patent Dec 7 2010 Sheet 8 of 8 US 7 847 178
16. 28 Firmware forthe DMM is stored in the Flash Memory and can therefore be upgraded readily by a user by downloading through the MIDI PC interface 28 A digital instrument tuner for all modern and classical instruments is shown in FIG 2 at 32 The compact design obtained in the DMM to afford the portable nature of the device is obtained by housing some functions required only when using the product in a nonmo bile environment to be placed in the docking station 11 of FIG 1 These functions include the circuitry required for the insertion of Karaoke lyrics in a video source and for display on a TV screen as well as an external microphone connection for Karaoke recording A MIDI in out interface is provided in the docking station 11 for connection to a personal computer or to another MIDI device A power jack and charge control not shown is provided at the rear of the docking station to provide the DMM with a 3 3 volt power supply in a preferred embodiment as shown at 33 in FIG 2 The DMM is config ured to include two AA batteries These may be rechargeable or regular batteries Multiple functions are provided through controls 13 on the DMM such as compressed audio playback electronic DJ voice and sound samples and radio Playback and record keys are available on the face of the DMM together with general volume control keys Power video and audio connectors and microphone input connectors are provided on the docking station 11 Additionally
17. 307 A 715 727 850 851 See application file for complete search history 56 References Cited U S PATENT DOCUMENTS 8 1983 Aoki 84 1 03 3 1986 Levy etal 379 101 01 4 399 731 A 4 577 067 A Continued FOREIGN PATENT DOCUMENTS EP 484047 10 1991 Continued OTHER PUBLICATIONS Beatnik Rich Music Format 2 pages 2002 Continued Primary Examiner Jeffrey Donels 74 Attorney Agent or Firm Loudermilk amp Associates 57 ABSTRACT A digital multi media device provides features for a user unskilled in musical arts or sound handling techniques that provides automatic musical score composition in accordance with contained composition instructions Stored sound samples and interfaces for obtaining external signals provide signals for merger with visual and sound presentations to obtain altered presentations either time shifted or in real time In this fashion the user can create simulated radio stations for playback of prearranged and composed audio material Fur ther the automatically composed musical score may be mixed with synthesized digitized signals from the stored sound samples and external signals obtained through the device interfaces 14 Claims 8 Drawing Sheets 39 Synthesizer O US 7 847 178 B2 Page2 U S PATENT DOCUMENTS 6 425 822 7 2002 Hayashida et al 463
18. 455 567 5 864 868 A 1 1999 Contois 707 104 2003 0079598 Al 5 2003 Nakayama 5 877 445 A 3 1999 Hufford et al 84 602 AR 5886274 A 3 1999 lungleib Sabi 2003 0176206 Al 9 2003 Taniguchi et al 455 567 5 900 566 A 5 1999 Mino 84 610 2003 0183065 Al 10 2008 Leach 5 913 258 A 6 1999 Tamura 84 604 2003 0205125 A1 11 2003 Futamase et al 84 622 5914941 A 6 1999 Janky 370 313 2004 0039796 Al 2 2004 Watkins 5 928 330 A 7 1999 Goetzetal 709 231 2004 0064320 Al 4 2004 Chrysanthakopoulos et al 5 969 716 A 10 1999 Davis etal 345 328 2004 0088169 Al 5 2004 Smith et al 5 981 860 A 11 1999 Isozaki et al 84 603 2004 0106395 Al 6 2004 Suganuma et al 6 008 446 A 12 1999 Van Buskirk et al 84 603 2004 0109558 Al 6 2004 Koch 6 011 212 A 1 2000 Rigopulos et al 84 667 2004 0231499 Al 11 2004 Kobayashi 6 051 770 A 4 2000 Milburn et al 84 611 2005 0190199 Al 9 2005 Brown et al 6 072 480 A 6 2000 Gorbet et al 345 302 2006 0156909 Al 7 2006 Kobayashi 6 074 215 A 6 2000 Tsurumi 434 307 2006 0185504 Al 8 2006 Kobayashi 6 083 009 A 7 2000 Kim etal 434 307 a ol FOREIGN PATENT DOCUMENTS 6 121 533 A 9 2000 6143971 A 112000 kp ucc ice 6 143 973 A 11 2000 Dam aS 6 182 126 B1 1 2001 Nathan et al p 06205567 10 1904 6 192 340 B1 2 2001 Abecassis 6 225 547 BL 5 2001 Toyama etal
19. 7 6 429 863 8 2002 LoPiccolo et al 345 419 4 787 073 A 11 1988 Masaki 369 32 6 450 888 B1 9 2002 Takase etal 463 43 5 054 360 10 1991 Lisle etal 84 645 6 472 591 B2 10 2002 Aoki etal 84 611 5 099 740 A 3 1992 Minamitaka 84 649 6 482 087 11 2002 463 7 5 177 618 1 1993 Dunlap et al 358 335 6 506 969 1 2003 84 609 5 267 318 A 11 1993 Severson 381 51 6 541 691 B2 4 2003 Tolonen et al 84 616 5 281 754 A 1 1994 Farrett et al 84 609 6 576 828 B2 6 2003 Aoki etal 84 635 i pi 84 601 6 576 878 B2 6 2003 Thorpe etal 219 645 307 6 639 141 B2 10 2003 84 609 5 308 915 5 1994 6 645 067 B1 11 2003 Okita etal 2 463 7 6 657 116 B1 12 2003 Gunnerson 84 615 369 6 683 241 B2 1 2004 Wieder 84 609 pcg al usse en 6 696 631 B2 2 2004 Smith et al 84 645 425 oung Jr 6 782 299 BL 8 2004 Tsuji etal 700 94 5 451 709 A 9 1995 Minamitaka 84 609 6 815 600 B2 11 2004 Georges etal 84 609 A 3 et al n e 6 835 884 B2 12 2004 Iwamoto et al 84 609 523 urakami et al eM 6 835 887 B2 12 2004 Devecka 84 743 5 581 530 A 12 1996 Iizuka et al 369 93 6 897 368 B2 5 2005 Georges etal
20. advantages of the present invention will become more apparent by describing in detail the preferred embodiments of the present invention with ref erence to the attached drawings in which FIG 1 is a perspective of the digital multi media device of the present invention FIG 2 is a functional block diagram of the present inven tion FIG 3 is a block diagram of the automatic composer of the present invention FIG 4 is a block diagram showing one structure for sum ming outputs in the present invention FIG 5 is a block diagram showing another structure for summing outputs in the present invention FIG 6 is a block diagram showing a first variant of the diagram of FIG 3 FIG 7 is a block diagram showing a second variant of the diagram of FIG 3 FIG 8 is a block diagram of another embodiment of the automatic composer of the present invention FIG 9 is a block diagram of a variation of the present invention FIG 10 is a block diagram of another aspect of the present invention US 7 847 178 B2 3 FIG 11 is a block diagram of an additional aspect of the present invention FIG 12 is a block diagram of yet another aspect of the present invention FIG 13 is a flow chart illustrating one preferred embodi ment of the present invention DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in greater detail with reference to certain preferred and alternative embodi ments As des
21. allow a user to automatically create and play many different types of music such as dance techno rap easy listening etc Unlimited hours of random music may be generated by the musical algorithms The device also operates as a musical composer assistant An FM receiver 23 is provided in the DMM so that a user is able to tune to a favorite radio station or to a virtual radio station While music is playing a user is able to take control over automatic composition being undertaken by the musical algorithms 21 and to add the users own touches The user may interact with the music being played by way of changing the tempo and pitch Consequently interaction is available in as much as the user is able to play voice music or sound samples that can be recorded with a built in micro phone or obtained from the FM radio receiver or any other source The samples are obtained through the use of a high quality audio compression circuit 24 and are stored in the SmartMedia memory section 19 The controls 13 on the face of the DMM FIG 1 provide for introducing the sound effects such as wobbler dobbler etc and can be applied at a user s will during playback of the recorded samples Addi tionally the controls allow the user to take even more control over the musical content by selecting relative volume of an instrument or a group of instruments in a fashion normally only possible through the use of a mixing table The sound of a particular instrument may
22. at is claimed is 1 A method of playing music using a music algorithm comprising the steps of processing MIDI information wherein the processed MIDI information is provided to a processing resource processing one or more sound samples wherein audio sample data is provided to the processing resource synchronizing playback of the audio sample data with the MIDI information at a predetermined time between a beginning and an end of a musical bar 2 The method of claim 1 wherein the processing resource comprises a first portion associated with the MIDI informa tion and a second portion associated with the audio sample data 3 The method of claim 1 further comprising providing a MIDI event associated with at least a part of the audio sample data 4 The method of claim 3 wherein the MIDI event com prises a system exclusive MIDI message 5 The method of claim 1 wherein the audio sample data is provided from a writable memory resource location 10 20 25 18 6 The method of claim 5 wherein the writable memory resource location comprises a flash memory 7 The method of claim 1 wherein at least one of the sound samples is processed in response to user input 8 The method of claim 1 wherein playback of at least a portion of the audio sample data starts at the beginning of a musical bar 9 The method of claim 1 wherein the processing resource comprises a hardware digital signal processor 10 The method of claim
23. ayback module 61 that pro duces an output V2 connected to a video output module 62 and an audio output A2 connected to a sound generation module 57 Items 56 58 61 and 62 are generally contained in acommonly known camcorder device The camcorder device also includes an external sound input element 54 such as a microphone producing an audio output A1 Output AT 15 also connected to sound generation module 57 A control module 59 shown connected to items 56 58 61 and 57 in FIG 10 US 7 847 178 B2 9 selects sounds generated by sound generation module 57 to produce output S2 providing audio output 63 According to the functional diagram of FIG 11 the inven tion can typically be embodied by a module that generates music or other sounds and which comprises essentially pro cessor 36 memory 37 containing the music database of the automatic composition device musical synthesizer 39 memory 38 storing the sound samples for the musical syn thesizer a digital to analog conversion circuit 43 a summa tion circuit 58 and a memory 42 internal or external to the invention containing a library of digitized musical files The memory elements 37 38 and 43 can be made of one or several distinct physical components Processor 36 is able to select according to certain criteria that are pre defined or defined by a user musical files out of the library in memory 43 oris able to compose automatically a melody out ofthe database stored in mem
24. az United States Patent Georges US007847178B2 US 7 847 178 B2 Dec 7 2010 10 Patent No 45 Date of Patent 54 INTERACTIVE DIGITAL MUSIC RECORDER AND PLAYER 75 Inventor Alain Georges Saint Paul FR 73 Assignee Medialab Solutions Corp Marshall TX US Subject to any disclaimer the term of this patent is extended or adjusted under 35 U S C 154 b by 0 days Notice This patent is subject to a terminal dis claimer 21 Appl No 12 322 862 22 Filed Feb 8 2009 65 Prior Publication Data US 2009 0241760 A1 Oct 1 2009 Related U S Application Data 63 Continuation of application No 11 705 555 filed on Feb 10 2007 now Pat No 7 504 576 which is a continuation of application No 10 634 346 filed on Aug 4 2003 now Pat No 7 176 372 which is a continuation of application No 09 691 314 filed on Oct 17 2000 now abandoned which is a continua tion in part of application No 09 690 911 filed on Oct 17 2000 now abandoned and a continuation in part of application No 09 691 302 filed on Oct 17 2000 now Pat No 6 392 133 30 Foreign Application Priority Data Oct 19 1999 sichere etis 99 13036 Nov 17 1999 FR 99 14420 51 Int CI GI0H 7 00 2006 01 52 UIS CL 84 645 84 603 37 Music Database Processor 38 58 Field of Classification Search 84 600 84 603 625 645 611 635 434
25. be altered using filters and also may be saved to be later played over again and even to be edited later Controls 13 are intuitive using traditional player keys while the graphical display of 14 provides visual feed back in real time ofthe action the user is undertaking with the musical content As a result users with no special musical education or skill are able to alter and compose music to their own designs Audio compression circuit 24 preferably is a circuit based ona speech compression algorithm using for example adap tive differential pulse coded modulation ADPCM that can beapplied to a useful portion ofthe recording As is known in the art techniques such as PCM and ADPCM provide meth ods of compression of analog data that has been sampled and digitized typically in which the samples have been digitized and difference between samples are stored sent rather than the samples themselves or with adaptive techniques a pre dicted sample sample difference is computed and what is stored transmitted is the difference between the predicted US 7 847 178 B2 5 value and the actual value etc Such compression techniques are known in the art and used for example in various tele phony type systems built in microphone not shown in the DMM is used for Karaoke and DJ functions Further it is used with a music synthesizer 26 to provide for audio mixing and other audio sound effects The synthesizer 26 provides a MIDI interfac
26. can include an internal digital adder The synthesizer can include an additional processor for directly accessing the memory for storing sound samples and the synthesizer can include means for handling sound samples as a special case of basic instrumental sounds The automatic composer can include a microphone for recording external sound samples and means for starting and stopping recording The automatic composer can include means for automatically eliminating the silent periods that precede and follow a useful portion of a recording and means for imple menting a speech compression algorithm to compress the useful portion of the recording The processor can include means for integrating the sound samples into musical com positions means for functioning to select the sound samples according to a pseudo random sequence means for directing the sample to be played at a predetermined time between the beginning and the end ofa musical bar as governed by certain musical rules means for directing any portion of the sound sample to be played from its entirety to any part thereof and means for optionally selecting repetition ofthe sound sample The processor can include means for imparting special effects to the musical piece whereby the musical piece is optionally modified with any ones of echo vibrato distortion frequency modulation and filtering effects The processor can include a clock operating at 25 MHz maximum and the first memory and t
27. cribed below refinements and substitutions of the various embodiments are possible based on the principles and teachings herein The invention disclosed herein is an interactive digital music player that allows one or more users to listen to com pose and interact with music in any environment The device is a hand held digital music player that offers numerous unique features that until now have not existed in a single music product One of the many features the Electronic DJ e DJ automatically composes and plays in real time music in a multitude of genres such as a dance or techno which can be mixed with user supplied voice samples The e DJ offers a unique interactive way of playing with music At any moment the user can take the control of the music played by the e DJ via an attractive interface joystick graphical dis play The user can change the music patterns played by various instruments change the relative level apply effects play pre recorded samples etc Further the disclosed device allows the user to listen to create download store and interact with music and includes an FM radio receiver Users can listen to both compressed audio and MIDI karaoke music files store music on a plug in SmartMedia memory card SSFDC storage device and carry the unit to any location for playing The device is capable of storing more than 1000 MIDI karaoke file songs on a 64 MB smart media memory card and it can provide over 120 m
28. ct access key The LCD will display e Samples in the status line and a list of available samples or sample lists on the SmartMedia card to choose from Sample lists are identified by a specific icon To select the desired sample or list use Joystick Up Down To select the desired sound effect press Effect key together with joystick left right To change the sample volume press Effect key together with joystick up down To start the playback of the selected sample press the play pause key The LCD will display the play sample screen The name of the sample is scrolling in a banner in the center right part of the LCD while the audio output level is materi alized by a frame around the name which size is changing following the audio level The status line shows the current effect Press stop to stop the sample and go back to sample selec tion screen Pressing the Save Edit key in the sample select screen or while a sample is playing enters the sample edit mode Pressing the Save Edit key in the sample select screen while the current item is a sample list enters the sample list edit screen See Editing items below Pressing forward key in the sample select screen will create a new sample list To exit Sample mode choose any other mode by pressing the appropriate direct access key Recording samples is a simple operation possible in almost any operating mode of the dB1 Press record and keep record button down to record a
29. d or MIDI The DMM offers a user the capability to interact with a song being played by chang ing pitch and tempo The audio compression techniques sup ported in the DMM is Microsoft Windows Media Audio 30 35 40 45 60 4 WMA illustrated at 22 in FIG 2 having files in the active streaming format ASF WMA provides a higher compres sion ratio for equivalent sound quality as compared to previ ous compression techniques i e MP3 DMA compression techniques allow a compression ratio approximately twice that of MP3 thereby allowing for over two hours of music to be stored on a 64 megabyte SmartMedia card The WMA playback function supports the Digital Rights Management DRM technology that is used with audio files This technol ogy can be used to encrypt the compressed data which can then only be decoded correctly with a private key As a result auser can download music from the Internet that is protected with the DRM technology and receive rights to play music on a particular DMM that has been previously registered ASF and MIDI files can be copied on the SmartMedia card either via a serial PC connection offered by the DMM device or directly with an offthe shelf SmartMedia card reader Internal memory and SmartMedia memory 19 further provides the ability for the DMM to obtain firmware and sound upgrades from the Internet An electronic DJ and virtual radio functions are provided by a group of musical algorithms 21 that
30. dio output level is materi alized by a frame around the name which size is changing following the audio level The status line shows the elapsed time Ifthe song is Karaoke song the Lyrics are displayed at the bottom ofthe LCD Note ifthe song is a user song composed with the e DJ the music I way mode is entered instead of the play song mode To change music tempo press Pitch Tempo key together with joystick left to speed up or right to slow down To change music pitch press Pitch Tempo key together with joystick up to higher the pitch or down to lower it Press play pause to pause the music Press play pause again to resume playback Press stop to stop the music and go back to song selection screen Press forward to go to next song Press backward to go to previous song Pressing the Save Edit key in the song select screen or while a song is playing enters the song edit mode Depending on the type of the song user song MIDI or WMA different parameters can be edited Pressing the Save Edit key in the song select screen while the current item is a song list enters the song list edit screen See Editing items below Pressing forward key in the song select screen will create a new song list To exit Songs mode choose any other mode by pressing the appropriate direct access key In the playing samples mode the user can record or play voice music or sound samples To enter Samples mode press the Samples dire
31. e whereby due to the small size of a MIDI file a large number of songs may be made to fit in the flash memory 19 containing the SmartMedia cards The SmartMedia Flash memory pro vides for storage of compressed digital audio MIDI Karaoke files and voice samples This is currently the most compact Flash Memory form available The digital nature of the MIDI files allows tempo and pitch of music to be dynamically adapted to the users requirements Moreover when a singer s voice is taken from the built in microphone within the DMM and mixed with synthesized or stored sound samples lyrics may be displayed on a TV video interface 27 or on the display 14 of FIG 1 An external microphone may be plugged into the DMM for even more flexible usage When the DMM is connected to the docking station 11 and playing MIDI or Karaoke songs a digital light show is avail ablethrough the TV video interface 27 The resulting colorful patterns have movements and shapes following the beat of specific instruments or serve as a compliment to lyrics dis played on the screen 14 or just as a visual assist to the elec tronic DJ music A universal serial connection 28 to a personal computer allows transfer of audio files to and from the SmartMedia flash memory 19 Additionally the serial connection is avail able to perform a firmware upgrade The MIDI interface 26 in the DMM 10 converts to a synthesizer when connected to a MIDI sequencer through the USB PC interface
32. e to un mute the microphone To exit Electronic DJ mode choose any other mode by pressing the appropriate direct access key Playback of compressed audio and Karaoke is accessible in the Songs mode This mode allows to play digitally recorded WMA MP3 songs MIDI songs Karaoke songs and User songs saved during an e DJ session enter Songs mode press the Songs direct access key The LCD will display e Songs in the status line and a list of available songs or song lists on the SmartMedia card to choose from Song lists are identified by a specific icon To select the desired song or list use Joystick Up Down To start the playback of selected song or song list press the play pause key The LCD will display the play song screen The name of the song is scrolling in a banner in the center right part of the LCD while the audio output level is materi alized by a frame around the name which size is changing following the audio level The status line shows the elapsed time Ifthe song is a Karaoke song the Lyrics are displayed at the bottom of the LCD Note ifthe song is a user song composed with the e DJ the music I way mode is entered instead of the play song mode To change music tempo press Pitch Tempo key together with joystick left to speed up or right to slow down To change music pitch press Pitch Tempo key together with joystick up to higher the pitch or down to lower it Press play pause to pause the music Pr
33. ed sound with the unique musical compositions and stored audio and voice samples to provide altered audio and voice presen tations The present invention includes an improved automatic composition device that adds to instrumental sounds some previously recorded sound samples wherein the sound samples are mixed with a melody in the process of the auto matic composition The sound samples will typically but not exclusively be human voice samples The invention herein further represents an improvement over prior art devices by integrating in a video recorder or 20 25 30 35 40 45 50 55 60 65 2 player a module that generates music or other sounds which eithercan be mixed with the originally recorded sound sound mixing or can replace the originally recorded sound sound dubbing This sound mixing or dubbing can be performed either at video audio record time or at play time Additionally the invention envisions selection of musical pieces either out of a library stored for instance in the form of digitized musical files MIDI files or other types of files or from pieces that are composed in a pseudo random fashion using for example a synthesizer function to play original musical pieces The invention further envisions selection of musical pieces according to a specified musical style in a pseudo random fashion or according to predefined criteria wherein the audio files meet the predefined criteria and are
34. ental sounds and the sound samples in a sounds memory 48 as shown in FIG 8 In this embodiment the summing of digital signals M2 and S2 is naturally performed by the synthesizer according to the same process as described for the variant of FIG 7 Thus the program executed by the processor of the synthesizer 39 is modified to permit these new operations storing sound samples and summing them with instrumental sounds The program modifications performed by the processor of the synthesizer can be minimized by organizing the program in such a way that the synthesizer processes sound samples as a special case of basic instrumental sounds for example as by assigning a MIDI channel to the sound samples or as an alternative by defining the sound samples as special notes of a percussion type instrument In a preferred embodiment of the invention the invention allows a user to record sound samples with a microphone via commands that permit the user to start and stop a recording An internal microphone and or an external microphone input 20 25 30 35 40 45 50 55 60 65 8 is required As seen in FIG 9 a microphone 49 provides an output STAT connected to the input of preamplifier 51 An analog to digital converter 52 converts the microphone signal S1A1 to digital sample signals S12 The sample signals are read by processor 36 and stored either in its memory 38 or in the case of the embodiment of FIGS 7 and 8 in the
35. er Music Center date unknown Music Sketcher Section Details IBM apparently 1998 Musical Computers Miles Davis version 2 1 apparently from The Economist Dec 6 1997 p 92 Roland Corporation Personal Music Assistant Owner s Manual Dec 18 1995 Entire Manual Yamaha Mobile Audio 3 MA 3 YMU762 Yamaha LSI Yamaha Corp Preliminary May 9 2001 p 1 14 Tom Hays DirectMusic for the Masses Gamasutra com article believed to be originally published in Game Developer Magazine Sep 1998 27 pages Microsoft Microsoft DirectMusic Producer Game Development Tutorial believed to be dated Nov 12 2002 file date of electronic file or earlier 52 pages Excerpts from File History of related U S Appl No 10 634 622 Excerpts from File History of related U S Appl No 11 705 555 Excerpts from File History of U S Appl No 10 324 389 Chong John Yu Computer Generated Music Composition May 28 1996 21 pages MIT Thesis Mary Farbood A New Approach to Interactive Com puter Generated Music Sep 1 2001 78 pages MIT Thesis Alexander Rigopulos Growing Music from Seeds Parametric Gen eration and Control of Seed Based Music for Interactive Composi tion and Performance Aug 5 1994 MIT Thesis Eran Baruch Egozy Deriving Musical Control Features from a Real Time Timbre Analysis of the Clarinet Jan 20 1995 MIT Thesis cited by examiner U S Patent Dec 7 20
36. ess play pause again to resume playback Press stop to stop the music and go back to song selection screen Press forward to go to next song Press backward to go to previous song Pressing the Save Edit key in the song select screen or while a song is playing enters the song edit mode Depending on the type of the song user song MIDI or WMA different parameters can be edited Pressing the Save Edit key in the song select screen while the current item is a song list enters the song list edit screen See Editing items below Pressing forward key in the song select screen will create a new song list To exit Songs mode choose any other mode by pressing the appropriate direct access key Playback of compressed audio and Karaoke is accessible in the Songs mode This mode allows to play digitally recorded WMA MPA3 songs MIDI songs Karaoke songs and User songs saved during an e DJ session To enter Songs mode press the Songs direct access key The LCD will display e Songs in the status line and a list of available songs or song lists on the SmartMedia card to choose from Song lists are identified by a specific icon To select the desired song or list use Joystick Up Down To start the playback of selected song or song list press the play pause key The LCD will display the play song screen US 7 847 178 B2 13 The name of the song is scrolling in a banner in the center right part of the LCD while the au
37. he second memory can include a memory having a capacity of 2 MB maximum 20 25 30 35 40 45 50 55 60 65 16 Also in accordance with the present invention an auto matic soundtrack generator in a digital multi media device can be provided that is capable of mixing a signal from an external audio source in a video audio record and or play device with an internal audio source containing digitized sound files in accordance with predetermined mixing instruc tions The automatic soundtrack generator can include a pro cessor a memory element connected to the processor for storing mixing the instructions and the digitized sound files a digital to analog conversion circuit for receiving the digitized sound files and a summation circuit for combining the signal from the external audio source with the digitized sound files The automatic soundtrack generator can include a digital musical synthesizer and a store of pre recorded and auto matically composed MIDI files wherein selected ones of the store of MIDI files are connected to the digital musical syn thesizer for providing the internal audio source The auto matic soundtrack generator can include a user accessible input and means for selecting the internal audio source con nected to the user accessible input and defined by the user The automatic soundtrack generator can include an additional memory means containing preprogrammed instructions for defining the inter
38. hm Jazz Improviser John A Biles popular version of paper 4pMUI apparently presented Dec 4 1997 GenJam A Genetic Algorithm for Generating Jazz Solos John A Biles date apparently after 1993 GenJam Populi Training an IGA via Audience Mediated Perfor mance John A Biles apparently Sep 15 1995 Information on how to purchase Kid Riffs IBM date unknown Interactive GenJam Integrating Real Time Performance with a Genetic Algorithm John A Biles apparently after 1996 Hemmings Richard Scary Computer Music apparently from Avant Magazine Issue 7 Summer 1998 p 12 Kid Riffs IBM Computer Music Center date unknown Louis Duke Miles and MAC Business Week Archives appar ently Dec 18 1995 Mithic a New Generation of Music Thomson Multimedia 3 pages Jul 8 2002 Mithic the First Interactive Music Composer 2 pages 2002 Thomson Multimedia presents Mithic Technology 3 pages 2 slides per page undated Thomson Multimedia launches Mithic a unique Technology to to deliver personalized interactive Music Feb 25 2002 10 2002 Thomson to present the Mithic Composer Technology at DEMOmobile Conference Sep 19 2002 Mobile Media Analyst Music Services Might Soon Break Out of the Ring Tone Mold Mobile Media management report pp 5 12 Jan 10 2003 Motorola C350 Cellular Phone User Manual selected pages on MotoMixer Sound Editor Motorola Inc pp 1 2 129 131 2002 Music Sketcher IBM Comput
39. in utes of digital music play time In addition the disclosed device is able to create music files accept music files created by the user download music from the Internet via a PC take a music transferred from a PC or accept music added from any other smart media memory card When connected to a docking station the device offers additional features like insertion of Karaoke lyrics in a video source for display on a screen MIDI PC connection or remote control FIG 1 shows the device 10 and its docking station 11 A number of cables 12 are shown for connection to a number of ports not shown situated at the rear ofthe docking station 11 The device 10 as shown has a group of controls 13 including buttons knobs jacks etc anda display 14 on its front surface A connector within a slot 16 is shown on the device that is configured to mate with a connector not shown on a rear surface of a base 17 on the docking station 11 The docking station includes an inclined surface 18 that supports the device in the docking station when connection is made between the connector in slot 16 and the connector in the docking station The device 10 is shown in the block diagram of FIG 2 and will be referred to as a Digital Multi Media Platform DMM hereinafter A large number of features are included in the DMM one of which is the ability to playback digital audio data from a SmartMedia memory card 19 The digital audio source is either compresse
40. les in memory 38 whereas a ROM or EPROM memory can store the microprocessor pro gram and the music database in memory 37 used by the automatic composition device However a greater flexibility will be granted by non volatile memories RAM memory saved by a disposable or rechargeable battery or Flash EEPROM memory electrically erasable The non volatile memory can be used at the very least to store the sound samples in memory 38 so that they are saved when the invention is powered off It can also be used to store the music database in memory 37 for use by the automatic composition device as well as a microprocessor operating program This permits easy update of the music database and the micropro cessor operating program by means of downloading updates The summing of output signal M2 of the synthesizer and of sound sample signal S2 is obtained using various methods and circuitry described as follows Ina first scheme for performing the summing of M2 and S2 FIG 4 output signal M2 of synthesizer 39 which is a digital signal is converted to an audio signal MA2 by digital to analog converter 42A Sound sample signal S2 generated by processor 36 which is also a digital signal is separately converted to an audio signal SA2 by a second digital to analog converter 42B Then audio signals MA2 and SA2 are summed by analog adder 44 to deliver an audio output signal MA3 The analog adder 44 can consist of an operational amplifier mounted in a
41. n adder configuration US 7 847 178 B2 7 Ina second manner of performing the summing of M2 and S2 FIG 5 output signal M2 of synthesizer 39 and sound sample signals S2 generated by processor 36 are added by digital adder 46 to form a combined digital signal M3 This digital signal is then converted to an audio signal MA3 by a digital to analog converter 47 While this manner of perform ing the summing requires only one digital to analog converter 47 it nevertheless requires a digital adder 46 In a variant of the second manner of performing the sum ming of M2 and S2 FIG 6 output M2 of synthesizer 39 is sent back to processor 36 which then performs by itself the summing of digital signals M2 and S2 in this variant signal S2 only exists in a virtual form in the processor Thus the processor delivers directly on output M3 the compound digi tal signal which is converted to an audio signal MA3 by digital to analog converter 47 This variant requires only one digital to analog converter and does not require a specific digital adder circuit but conversely it requires an additional input on the processor to read the output of the synthesizer Ina second variant ofthe second manner of performing the summing of M2 and S2 FIG 7 which is applicable to the case where synthesizer 39 itself comprises a processor mi croprocessor and or digital signal processor the summing of digital signals M2 and S2 can be performed by the synthesi
42. n device of the present invention includes essentially a processor 36 a memory 37 containing a music database for use in the auto matic composition device a memory 38 for storing sound samples utilized by the automatic composition device a music synthesizer 39 and a summation and digital to analog conversion circuit 41 interconnected as shown Processor 36 is able to compose automatically a melody out of the database stored in memory 37 using an automatic composition algo rithm containing composition instructions The automatic composition algorithm also utilizes the sound samples stored in memory 38 in such a way that processor 1 delivers in synchronism on its output a control signal M1 connected to synthesizer 39 and a sound sample signal S2 An output signal 2 of the synthesizer and sound sample signal S2 are then summed and converted to analog form in circuit 41 to deliver the complete audio signal MA3 This audio signal MA3 which forms the output ofthe invention can then be presented in audio form by a stereo sound system In a preferred embodiment of the invention the processor is made of a microprocessor or microcontroller having a synchronizing clock and being linked to one or several memories Clock speed and memory capacity may be rela tively low 25 MHz and 2 MB being ample in the preferred embodiments A RAM memory volatile memory can serve as the working memory of the microprocessor and can be used to store the sound samp
43. nal audio source and means for selecting the internal audio source connected to the additional memory means The automatic soundtrack generator can include means for selecting the internal audio source at video audio record time The automatic soundtrack generator can include means for selecting the internal audio source when playing a previously recorded video audio sequence Also in accordance with the present invention a virtual radio in a digital multi media device can be provided that is capable of selecting according to given criteria musical files from predetermined sound sources for playing from a speaker The virtual radio can include a processor providing a digital output a memory element connected to the processor for storing the given criteria and the predetermined sources of musical files and a summation and digital to analog conver sion circuit connected to receive the digital output and pro viding a predetermined analog sound output The virtual radio device can include a digital musical synthesizer for adding MIDI files that are pre recorded or composed auto matically according to the given criteria to the predeter mined sound sources The virtual radio device can include a radio receiver providing a sound output used as an additional selectable sound source The virtual radio device can include means for providing voice files within the predetermined sound sources for combination with the radio receiver sound output The
44. ory 37 with an automatic composition algorithm The output signal S3 of the synthesizer after digital to analog conversion in converter 43A delivers an analog signal A3 which can optionally be mixed with the recorded audio A2 or with the external audio input A1 in summation circuit 58 to deliver the mixed complete audio signal S1 S2 This mixed audio signal S1 S2 which forms the output of the invention can then be used as a sound source at video audio record time real time S1 or at play back time time shifted S2 As a result prerecording of movie soundtracks for example is unnecessary simple way is provided for a user to change sound content pitch etc for implementation in a video soundtrack In a preferred embodiment of the invention the processor 36 is made of a microprocessor or microcontroller linked to one or several memories A RAM memory volatile memory can serve as the working memory of the microprocessor whereas a ROM or EPROM memory can store the micropro cessor program and the music database 37 of the automatic composition device However a greater flexibility will be granted by non volatile memories RAM memory saved by a disposable or rechargeable battery or Flash EEPROM memory electrically erasable The non volatile memory can be used to store the music database 37 of the automatic composition device and the digitized musical files of the library in sound samples memory 38 as well as the micro processor
45. ries A RAM memory volatile memory can serve as the working memory of the microprocessor and can be used to store the sound samples 38 whereas a ROM or EPROM memory can store the microprocessor program and the music database 37 used by the automatic composition algorithm However a greater flexibility will be granted by non volatile memories RAM memory saved by a disposable or rechargeable battery or Flash EEPROM memory electri cally erasable The non volatile memory can be used at the very least to store the sound samples in memory 38 so that they are saved when the invention is powered off It can also be used to store the music database in memory 37 for use by the automatic composition algorithm the digitized musical files of the library 43 as well as a microprocessor operating program This permits easy update of the music database and the microprocessor operating program by means of down loading updates Processor 36 can also be used to select the sound source or sources including radio receiver 64 or one of the sources in memories 37 38 and 43 described above It can also be used to select one of a number of preset radio stations including either actual radio stations or illusory radio stations Illusory stations in reality play sound samples and musical pieces that have been stored in digital form or that are composed auto matically as described herein Moreover a sound input device such as a microphone 66 is useful to inpu
46. rt playing the sample taking into account the melody that the processor is simultaneously com posing in such a way that the sample starts at the beginning of a musical bar or in specific cases at another moment that is governed by certain musical laws which may depend on the music style The sound samplecan then be played either in its entirety or in part and it can be repeated or not repeated Furthermore the sound samples may be modified to include various special effects depending on the style of music created by the automatic composition device To quote only a few of such special effects echo addition vibrato distortion frequency modulation various filtering processes to shape the sound spectrum etc One aspect ofthe current invention relates to a system such as described above that also may include or be included in an automatic soundtrack generator that operates to merge an independent sound track with a video sequence during recording or playback of the sequence aspects of the other embodiments described herein could be used to record or playback desired audio tracks in conjunction with video that being recorded and or played back etc FIG 10 represents a system in block diagram form showing an external video input 53 providing a signal V1 to a video and sound recording module 56 connected to store the signal V1 on storage unit 58 such as a tape or a digital memory The storage unit is con nected to a video and sound pl
47. t voice signals encoded in memory to be used in construction of illusory radio station sound patterns The controls 13 to which reference was made in FIG 1 include a joystick and a number of direct access keys to simplify entry into various interactive modes by users of the DMM device of the present invention Certain operating sequences for specific interactive modes will now be described To enter Electronic DJ press the e DJ direct access key The LCD will display E DJ in the status line and a selection of music styles is proposed Techno Dance Hip Hop etc To select the desired style use Joystick Up Down To start playing auto composed music ofthe selected style press the play pause key the LCD will display the music I way screen representing six instrument lanes Drums Bass US 7 847 178 B2 11 Riff Lead Samples and Microphone The screen is animated with sound waves or pulses synchronized with music beats Referring again to FIG 1 display 14 preferably displays music tunnels lanes sound waves beat pulses etc as described elsewhere herein such as is generally illustrated by display feature 14b To change music tempo press Pitch Tempo key together with joystick left to speed up or right to slow down To change music pitch press Pitch Tempo key together with joystick left to speed up or right to slow down To change music pitch press Pitch Tempo key together with joystick up to higher the pitch or down to
48. television signal interface Also in accordance with the present invention an auto matic composer in a digital multi media device is provided for composing a musical piece according to automatic com position instructions and for mixing sound samples into it The automatic composer can include a processor having an input and an output the processor operating to implement the automatic composition instructions a memory for storing a music database for use in accordance with the automatic composition instructions a second memory for storing the sound samples the first and second memories being con nected to the processor input a music synthesizer connected to the processor output for control by the processor and pro viding a synthesizer output and a summation and digital to analog conversion circuit for receiving processor and synthe sizer outputs and for providing a summed analog output The summation and digital to analog conversion circuit can include a digital to analog converter for receiving the synthe sizer and processor outputs and for providing analog signals and second a summation circuit for receiving the analog signals The summation and digital to analog circuit can include a digital adder for summing the processor and syn thesizer outputs and for providing a summed digital output and a digital to analog converter for receiving the summed digital output The processor can include an internal digital adder The synthesizer
49. trument Digital Inter face standard scores of notes that are composed automati cally Various devices capable of recording or playing video recordings have already been developed such as digital or analog camcorders and other video recorders These devices can possess one or several audio inputs which permit mixing orreplacement ofthe sound track that was recorded originally during the recording of the image with an external audio source Furthermore there also exist various devices that are capable of playing recorded pieces such as digital compact disc players players of compressed files for instance as per the MPEG level 3 standard etc Finally there exist devices incorporating a tuner which permit reception of radio broad casts via electromagnetic waves SUMMARY OF THE INVENTION An interactive digital multi media device provides a user with multiple related features wherein it is not necessary that the user have training in musical or sound handling arts A programmable memory is used to store digital audio and voice samples External sound recovers provide external sound signals such as radio and TV signals that may also be stored Playback means is provided connected to the stored audio and voice samples and the external sound signals Auto matic musical composition instructions are stored so that the user is able to create unique musical compositions Compo nents are provided that function to mix externally obtain
50. virtual radio device can include sound samples that are pre recorded in the predetermined sound sources by a user and can include means for mixing speech sound with the musical files Although the invention has been described in conjunction with specific preferred and other embodiments it is evident that many substitutions alternatives and variations will be apparent to those skilled in the art in light of the foregoing description Accordingly the invention is intended to embrace all of the alternatives and variations that fall within the spirit and scope of the appended claims For example it should be understood that in accordance with the various alternative embodiments described herein various systems and uses and methods based on such systems may be obtained The various refinements and alternative and addi tional features also described may be combined to provide additional advantageous combinations and the like in accor dance with the present invention Also as will be understood by those skilled in the art based on the foregoing description various aspects ofthe preferred embodiments may be used in US 7 847 178 B2 17 various subcombinations to achieve at least certain of the benefits and attributes described herein and such subcombi nations also are within the scope of the present invention All such refinements enhancements and further uses of the present invention are within the scope of the present inven tion Wh
51. zer modified for this purpose Indeed either the synthesizer has an additional input to read signal S2 as shown in FIG 7 or signals M1 and S2 are multiplexed on a single input of the synthesizer variant not shown on a figure In the latter case if a MIDI standard protocol is used the multiplexing of sig nals M1 and S2 will be made much easier by putting the information relevant to sound sample signals S2 in System Exclusive MIDI messages Thus the synthesizer performs internally the summing of signals M2 and S2 in this variant signal M2 only exists in a virtual form in the synthesizer and delivers directly on output M3 the compound digital signal which is converted to an audio signal MA3 by digital to analog converter 47 This second variant also requires only one digital to analog converter and does not require a specific digital adder circuit but conversely compared with a stan dard synthesizer circuit it requires a modification of the synthesizer to read and add sound sample signals S2 In another preferred embodiment of the invention appli cable to the case where synthesizer 39 itself comprises a processor memory 38 for storing sound samples is linked to synthesizer 39 instead of processor 36 Given that synthesizer 39 already possesses a memory that contains instrumental sounds that are used as basic units for music synthesis a variant ofthis embodiment consists in storing the two types of sounds i e the basic instrum

Synthesizer

Contents

Download Pdf Manuals

Related Search

Related Contents