Home

Personal media broadcasting system

1. A personal media broadcasting system enables video trans mission over a computer network allowing a user to view and control media sources over a computer network from a remote location In one embodiment the personal media broadcasting system includes a personal media broadcaster and a media player The personal media broadcaster may be configured to receive as an input virtually any media source broadcaster digitizes and compresses the received media content if necessary and streams the media over a computer network to the media player The media player may reside on any ofa wide range of client devices for viewing the media A user may send control commands e g channel up using the media player back over the network to be executed by the media source device thus affecting the media stream received by the media player In one embodiment a personal media broadcasting system includes a personal media broadcaster and media player cli ent which communicate with each other over a network The personal media broadcaster can accept an A V signal from one or more A V source devices From this A V signal the personal media broadcaster constructs a media stream that is suitable for transmission over a network to the media player client The personal media broadcaster includes a network interface for transmitting the media stream to the media player client As the media stream is received at the client it can be viewed by a user
2. For example there can be a rule that when the personal broadcaster is streaming content from ESPN the frame rate is always set at 30 frames per second Such rules can be stored on the client or on the central server which informs the client of the proper settings for a requested chan nel Moreover the rules can be learned over time by an indi vidual s own client which observes the settings chosen by the user for certain network programming In a further refinement the encoder settings may be cus tomized based on the specific programming being watched Because a major network typically has content that is both demanding high motion programming such as a major sporting event and easy low motion programming such as a newscast selecting encoder settings based solely on the channel may not be efficient for many channels Accordingly by cross referencing the current channel with the current time of day the system can determine the program that is being viewed and the encoder s settings can be selected based on the actual program being viewed In another embodiment the system constantly monitors the A V content being encoded Metrics based on pre selected criteria e g amount of motion in video are generated and this information is used to assign or adjust the encoder set tings to the personal broadcaster dynamically Alternatively the media player may receive from the user an indication of what kind of content is being watched
3. LAN 140 that couples several computing devices in a user s home According to known techniques any number of local clients 150 may be able to communicate with the home Inter net gateway 110 In this way created by the media broad caster 100 may be routed to any of these local clients 150 by way of the local network 140 either through the gateway 110 or directly The local area network 140 can be wired or wire less as the present invention is not limited to any particular US 7 877 776 B2 5 network technology or configuration The local clients 140 can be any number of device types including but not limited to desktop and notebook PCs Web tablets PDAs embedded clients built expressly for the purposes of decoding the streams of the personal broadcaster and other devices capable of receiving and or playing a media stream over a network The media streams created by the personal broadcaster 100 may also be received by remote clients 170 from a remote network 160 The remote network 160 may comprise any suitable networking technology including but not limited to wide area mobile networks e g GPRS EDGE 1X RTT 1x EvDO and FOMA 2 5G and 3G cellular networks WiFi and other public broadband access locations WiMAX net works other LANs such as at work school or a friend s home and direct connections to other Internet service pro viders As with the local clients 150 the remote clients 170 may include any number of device
4. No 11 620 707 filed Jan 7 2007 USPTO Non Final Office Action mailed Aug 7 2008 U S Appl No 11 620 711 filed Jan 7 2007 USPTO Final Office Action mailed Feb 9 2009 U S Appl No 11 620 711 filed Jan 7 2007 USPTO Non Final Office Action mailed Feb 25 2009 U S Appl No 11 683 862 filed Mar 8 2007 USPTO Non Final Office Action mailed Dec 24 2008 U S Appl No 11 147 985 filed Jun 7 2005 USPTO Non Final Office Action mailed Jun 25 2008 U S Appl No 11 428 254 filed Jun 30 2006 USPTO Final Office Action mailed Feb 6 2009 U S Appl No 11 428 254 filed Jun 30 2006 az US 7 877 776 B2 Page 5 Bluetooth News Main Future User Models Document Verification amp Qualification Bluetooth Technical Background Apr 21 1999 pp 1 of 7 and 2 of 7 http www bluetooth com v2 news show asp 1 2 Microsoft Corporation Harman Kardon Master Your Universe 1999 Matsushita Electric Corporation of America MicroCast Wireless PC Multimedia Transceiver System Nov 1998 Wireless Local Area Networks Issues in Technology and Stan dards Jan 6 1999 USPTO Final Office Action mailed Jun 25 2009 U S Appl No 11 147 985 filed Jun 7 2005 China State Intellectual Property Office First Office Action issued Jul 31 2009 for Application No 200580026825 X USPTO Non Final Office Action mailed Aug 4 2009 U S Appl No 11 734 277 filed Apr 12 2007 USPT
5. U S Appl No 12 757 697 filed Apr 9 2010 Tarra Raghuveer et al Firmware Update for Consumer Electronic Device U S Appl No 12 757 714 filed Apr 9 2010 Lee M et al Video Frame Rate Control for Non Guaranteed Net work Services with Explicit Rate Feedback Globecom 00 2000 IEEE Global Telecommunications conference San Francisco CA Nov 27 Dec 1 2000 IEEE Global Telecommunications Confer ence New York NY IEEE US vol 1 27 2000 pp 293 297 XP001195580 ISBN 978 0 7803 6452 3 lines 15 20 of sec IL on p 293 fig 1 European Patent Office International Searching Authority Interna tional Search Report and Written Opinion mailed Jun 4 2010 for International Application No PCT IN2009 000728 filed Dec 18 2009 USPTO Non Final Office Action mailed Jun 23 2010 U S Appl No 11 933 969 filed Nov 1 2007 Korean Intellectual Property Office Official Notice of Preliminary Rejection issued Jun 18 2010 Korean Patent Application No 10 2008 7021254 Japan Patent Office Notice of Grounds for Rejection Office Action mailed May 25 2010 Patent Application No 2007 0268269 Japan Patent Office Notice of Grounds for Rejection Office Action mailed May 25 2010 Patent Application No 2007 527683 USPTO Non Final Office Action mailed Aug 11 2010 U S Appl No 11 111 265 filed Apr 21 2005 cited by examiner U S Patent Jan 25 2011 Sheet 1 of 2 US 7 877 7
6. e g action music news etc which is mapped to predefined profiles in the framework optimized for that kind of content Buffering and Control of Buffer Resources In accordance with one embodiment of the invention the personal media broadcaster implements a buffering scheme to manage its buffer resources without requiring feedback from a client device As mentioned above the broadcaster and the client may communicate using TCP as a transport proto col where the broadcaster acts as a server Beneficially TCP isareliable protocol and ensures that sent data always reaches its destination in the correct order Parameters and or behav ior of the TCP stack on a server can be monitored to estimate network congestion and speed according to one or more known techniques Inaccordance with an embodiment ofthe invention a large buffer is added between the encoder which generates the data and the TCP stack on the network interface which transmits the data This additional buffer layer added above the TCP stack helps to avoid loss of data due to network congestion and the variability of data rates In one embodi ment with reference to FIG 3 the encoder functionality is performed by the processor 320 the TCP stack functionality performed by the network interface 325 and the buffer layer implemented in the general memory 330 or in a memory module dedicated for the large buffer The size of this buffer can be selected in consideration of
7. 12 United States Patent Krikorian et al US007877776B2 10 Patent No US 7 877 776 B2 45 Date of Patent Jan 25 2011 54 75 73 21 22 65 60 51 52 58 56 PERSONAL MEDIA BROADCASTING SYSTEM Inventors Jason Krikorian San Mateo CA US Blake Krikorian San Mateo CA US Assignee Sling Media Inc Foster City CA US Notice Subject to any disclaimer the term of this patent is extended or adjusted under 35 U S C 154 b by 1424 days Appl No 11 147 664 Filed Jun 7 2005 Prior Publication Data US 2006 0095471 Al May 4 2006 Related U S Application Data Provisional application No 60 577 833 filed on Jun 7 2004 Int HOAN 7 18 2006 01 H04N 7 173 2006 01 US CL 725 95 725 80 725 133 725 141 Field of Classification Search 725 80 725 133 141 93 97 See application file for complete search history References Cited U S PATENT DOCUMENTS 3 416 043 4 254 303 A 12 1968 Jorgensen A 3 1981 Takizawa 5 161 021 A 11 1992 Tsai 5 237 648 A 8 1993 Mills et al 5 386 493 A 1 1995 Degen et al 5 434 590 A 7 1995 Dinwiddie Jr et al 5 434 678 A 7 1995 Abecassis 5 438 423 A 8 1995 Lynch et al 5 493 638 A 2 1996 Hooper et al Central Server 180 Home Internet Personal Gateway Broadcaster On Network 110 100 Storage Device 5 602 589 A 2 1997 Vishwanath et al 5 610
8. 653 A 3 1997 Abecassis 5 661 516 A 8 1997 Carles 5 666 426 A 9 1997 Helms Continued FOREIGN PATENT DOCUMENTS CN 1464685 12 2003 Continued OTHER PUBLICATIONS ReplayTV 5000 User s Guide 2002 entire document Continued Primary Examiner Hunter B Lonsberry 74 Attorney Agent or Firm Ingrassia Fisher amp Lorenz PC 57 ABSTRACT A personal media broadcasting system enables video distri bution over a computer network and allows a user to view and control media sources over a computer network from a remote location A personal broadcaster receives an input from one or more types of media sources digitizes and com presses the content and streams the compressed media over a computer network to a media player running on any of a wide range of client devices for viewing the media The system may allow the user to issue control commands e g channel up from the media player to the broadcaster causing the source device to execute the commands The broadcaster and the media player may employ several techniques for buffer ing transmitting and viewing the content to improve the user s experience 52 Claims 2 Drawing Sheets AN Source Devices 120 Remote Clients 170 130 Local Clients 150 US 7 877 776 B2 Page2 5 682 195 5 684 918 5 696 869 5 706 290 5 708 961 5 710 605 5 722 041 5 757 416 5 774 170 5 778 077 5 794 116 5 822 537 5
9. No 12 426 103 filed Apr 17 2009 Thiyagarajan Venkatesan et al Always On Top Media Player Launched From a Web Browser U S Appl No 12 617 271 filed Nov 12 2009 Paul John Michael et al Systems and Methods for Delivering Messages Over a Network U S Appl No 12 619 192 filed Nov 16 2009 Rao Padmanabha et al Methods and Apparatus for Establishing Network Connections Using an Inter Mediating Device U S Appl No 12 642 368 filed Dec 18 2009 Dham Vikramet al Systems and Methods for Establishing Network Connections Using Local Mediation Services U S Appl No 12 644 918 filed Dec 22 2009 Paul John et al Systems and Methods for Remotely Controlling Media Server Via a Network U S Appl No 12 645 870 filed Dec 23 2009 Bajpal Parimal et al Method and Node for Transmitting Data Over a Communication Network using Negative Ackhowledgement U S Appl No 12 404 920 filed Mar 16 2009 Bajpal Parimal et al Method and Note for Employing Network connections Over a Connectinoless Transport Layer Protocol U S Appl No 12 405 062 filed Mar 16 2009 Asnis Ilya et al Mediated Network address Translation Traversal U S Appl No 12 405 039 filed Mar 16 2009 Australian Government Office Action Australian Patent Applica tion No 2006240518 mailed Nov 12 2009 Jain Vikal Kumar Systems and Methods for Coordinating Data Communication Between Two Device
10. U S Appl No 12 699 280 filed Feb 3 2010 Gangotri Arun L et al Systems and Methods and Program Appli cations for Selectively Restructuring the Placeshiftnig of Copy Pro tected Digital Media Content U S Appl No 12 623 955 filed Nov 23 2009 Paul John et al Systems and Methods for Searching Media Con tent U S Appl No 12 648 024 filed Dec 28 2009 Carlson T Mule 2 x Getting Started Guide Apr 15 2008 134 pages Curtis Sean Systems and Methods for Formatting Media Content for Distribution U S Appl No 12 635 138 filed Dec 10 2009 Newton s Telcom Dictionary 20th ed Mar 2004 The Authoritative Dictionary of IEEE Standard Terms 7th ed 2000 Gurzhi Alexander et al Systems Methods for Emulating Net work Enabled Media Components U S Appl No 12 711 830 filed Feb 24 2010 US 7 877 776 B2 Page 6 Bajpai Parimal et al Systems and Methods of Controlling the Encoding of a Media Stream U S Appl No 12 339 878 filed Dec 19 2008 Malone Edward D et al Systems and Methods for Controlling Media Devices U S Appl No 12 256 344 filed Oct 22 2008 Banger Shashidhar et al Systems and Methods for Determining Attributes of Media Items Accessed Via a Personal Media Broad caster U S Appl No 12 334 959 filed Dec 15 2008 Kulkarni Anant Madhava Systems and Methods for Creating Logi cal Media Streams for Media Storage an
11. a bit rate that is low enough for effec tive distribution across local area networks and upstream through a home Internet gateway and out to the public Inter net for connection by remote clients In variations that include digital inputs analog inputs and a digitization and encoding function can still be present Despite the specific network topology illustrated in FIG 1 there are many variations on the present invention that have the personal broadcaster placed in different positions relative to the other components For example one variation of one embodiment has the personal broadcaster incorporated into a home Internet gateway By integrating with the router func tionality the broadcaster can completely automate the pro cess of port configuration e g port forwarding Alterna tively if the home Internet gateway is actually two or more devices e g a cable modem anda stand along router switch the personal broadcaster can be located between the cable modem and the router Both of these variations of the present invention provide a unique quality of service opportunity Because the personal broadcaster is upstream from all net worked clients and can talk with the network router the video streams from the personal broadcaster can be priori tized ahead of other less time critical traffic The control can be applied both to traffic moving within the local area network as well as to traffic moving out from the local area netw
12. area network WAN 12 The personal media broadcasting system of claim 1 wherein the network comprises the Internet 13 The personal media broadcasting system of claim 1 wherein the encoder setting comprises a frame rate 14 The personal media broadcasting system of claim 1 wherein the personal media broadcaster is configured to encode the media stream by digitizing and compressing the uncompressed signal received from the audio visual source device 15 The personal media broadcasting system of claim 1 wherein the media stream is encoded by compressing the media stream using the encoder setting 16 The personal media broadcasting system of claim 1 wherein the uncompressed signal is a live stream from the audio visual source device 17 The personal media broadcasting system of claim 1 wherein the media player client communicates with the per sonal broadcaster through a Web browser interface 18 A device comprising an input interface configured to receive an uncompressed signal from one or more source devices a processor configured to construct a media stream by encoding the uncompressed signal using an encoder set US 7 877 776 B2 27 ting and to adjust the encoder setting based upon varia tions in an available bandwidth of a network and an embedded network interface configured to transmit the media stream to a client over the network 19 The device of claim 18 wherein the network interface is configured t
13. enable a user to access and control television and other A V content from any number of remote networks where a broadband connection is available to the user e g at an airport at work at school ina hotel ina cafe at an acquaintance s house Yet another application would be to enable a user to access and control television and other A V content from a mobile phone or other computing devices that can be connected to a wide area network e g GPRS W CDMA CDMA 2000 1XEVDO and the like In various applications users likely to want to access their media content stored on devices such as personal computers and other devices having storage from remote networks Nevertheless network bandwidth and other limitations have made it difficult to provide an effective and enjoyable remote media experience for the user But traditional streaming media solutions do not enable these applications in any effective way moreover they suffer from technical limitations that would prevent their use in personal media broadcasting applications like those described above Accordingly it would be desirable to enable 20 25 30 35 40 45 50 55 65 2 users to access their A V content from any of a variety of remote locations inside or outside the home as such content is currently available only from locations in the home that traditionally receive and play it e g a television set SUMMARY OF THE INVENTION
14. for each watermark respectively As described the system intelligently exploits stack behavior to estimate network status and reacts to provide optimal user experience in presence of bandwidth variations This may offer improved performance as compared to using client server interaction which can be complicated react more slowly and may not make correct decisions if the encoder s output bit rate is expected to vary Fast Start Streaming Conventionally when a streaming media player receives a command to play a media stream the media player fills its audio video buffers before starting the playback A buffer of five seconds or longer is typically maintained to ensure smooth playback ofthe media since the time taken to transfer the media stream over a network typically varies while the media stream is meant to be played at a constant rate Once the required amount of data is accumulated to fill the buffer sufficiently the media player starts playing the requested content at normal play speed 1 1 0x Disadvantageously US 7 877 776 B2 15 the user must wait the amount of time required to fill the buffer before viewing the requested content This time may be very small in applications that send the stream faster than real time e g a media stream from a storage device but it is noticeable when the received media stream is at normal play back speed 1 1 0x In such a case filling a five second buffer would
15. low bit rate encoding the following Gaussian filter of size 5x5 is used ee G x ty 202 For high bit rate encoding a low pass filter having an average mask of size 3x3 is used as shown in the table below eee eee eee The parameters o and yp are varied based on the edge estimate motion level estimate and the encoding bit rate Increasing o or decreasing 1 increases the smoothness of the video while the opposite decreases the smoothness of the video The edge estimate is operated at pixel level and is obtained by taking the gradient at all eight directions This is given by XA fo 610 45 90 135 180 225 270 315 where Afo amp y Ly Af 4s amp y amp y D Pe If is greater than a threshold T no pre filtering is per formed on the current pixel thereby preserving the image details The motion level estimate gives a classification of whether the motion is high medium or low motion The estimate is based on the bits required for the current frame and the recursive average ofthe bits per frame Ifthe current frame bits are greater than a always 71 times the recursive average the current video content is declared as high motion and if the current frame bits are less than B lt 1 times the recursive average the video content is classified as low motion Other wise it is classified as medium motion Based on the type of motion the filter strength is va
16. maintain a single session limit for remote clients FIG 2 illustrates an embodiment of the personal media broadcaster 100 having an interface for receiving a video signal from a collection of possible A V source devices 120 The personal broadcaster 100 can thus support a number of input types and possibly may include a number of outputs types according to one embodiment of the present invention In the embodiment illustrated in FIG 2 the personal broad caster 100 can support a composite video input 210 an S video video input 200 a coaxial cable input 250 and left and right audio inputs 220 The personal broadcaster 100 may also have a coaxial cable output 260 where the input cable signal is split inside the personal broadcaster 100 to allow a pass through of that signal for local viewing Pass through outputs for A V S video and any ofthe other inputs may also be provided for the same purpose A wide variety of video and audio inputs are possible in addition to those shown in FIG 2 Inputs and outputs can be either analog e g component video or digital e g DVI IEEE 1394 USB SDI Toslink optical or composite coax digital audio and there may be multiple connectors of a single type FIG 2 also includes an IR output 270 and or an RS 232 output 280 These outputs are intended to provide the final leg of backchannel control that originates from the client device Depending on whether the A V source device 120 is con trolled v
17. of claim 1 wherein the personal media broadcaster is configured to adjust the encoder setting in real time as the media stream is transmitted on the network 6 The personal media broadcasting system of claim 1 wherein the encoder setting comprises at least one of the following a bit rate a resolution or a frame rate 7 The personal media broadcasting system of claim 1 wherein the media player client and the personal media broadcaster are configured to communicate over the network to mutually establish capabilities of the media player client and wherein the personal media broadcaster is configured to initially establish the encoder setting based upon the capa bilities of the media player client 8 The personal media broadcasting system of claim 1 wherein the media player client and the personal media broadcaster are configured to communicate over the network to mutually establish a throughput of a connection between the media player client and the personal media broadcaster over the network and wherein the personal media broad caster is configured to initially establish the encoder setting based upon the throughput of the connection 9 The personal media broadcasting system of claim 1 wherein the encoder setting comprises a bit rate 10 The personal media broadcasting system of claim 1 wherein the encoder setting comprises a resolution 11 The personal media broadcasting system of claim 1 wherein the network comprises a wide
18. plugging in the third party remote control skins is provided to the user To allow third parties to create skins for the media player an API is provided to allow access to some of the features and func tionality within the media player client Third party skins allow users to develop content as well as third party device manufacturers and service providers to make virtual remote controls that closely resemble the physical remote control associated with an A V source device 120 The virtual remote on the client application can thus be made easier to use since the user is used to its layout Operation of Personal Media Broadcasting System As described above with reference to FIGS 1 through 3 the personal media broadcaster 100 can receive an input video signal from any of a number of A V source devices 120 The broadcaster 100 then prepares the received video signal as a media stream for being transmitted over a network to a remote or local client where the media stream is viewed by the user Additions alternatives and refinements to this general pro cess are described below Control of Audio Visual Source Devices As stated above embodiments of the personal media broadcaster allow a userto control an A V source device from clients connected to the remote or local networks The client may allow for control of the user s specific model of A V source device Upon initial configuration of the personal broadcaster and a client the user i
19. rate frame rate resolution and other properties remotely con trolling commands for the IR blaster and any other command that a user may desire for viewing content from the A V source device 120 broadcaster 100 may further comprise a controller interface 335 for interfacing with an output for controlling an A V source device 120 As mentioned above control of an A V source device 120 may be performed using an RS 232 serial controller or an IR emitter 285 The controller interface 335 thus receives the appropriate output signals from the processor 320 and provides the corresponding interface for controlling an operation of an A V source device 120 While FIG 3 shows an embodiment of the broadcaster 100 that only takes analog inputs other embodiments may accept digital inputs as well For example embodiments of the present invention can be incorporated into service offerings from cable MSOs or DBS providers In these variations the personal broadcaster can have direct access to the digital bit stream being broadcast This can be through incorporation of the personal broadcaster into a set top box or home Internet gateway by the service provider or through some digital interconnect such as IEEE 1394 or USB 2 0 With access to the digital bit streams digitizing and encoding compression of the streams can be entirely unnecessary For these inputs however the personal broadcaster can be asked to transrate or transcode the media to
20. the media broadcaster includes A V pass throughs for all of the inputs where there is an output that corresponds to each of the inputs This saves the user from using multiple outputs on an A V source device and may provide a complete method for prohibiting two sessions Moreover this embodiment can be used to prohibit the simul taneous occurrence of a client connection to a personal broad caster and a local viewing of the A V source which is plugged into the personal broadcaster For example if someone were watching a pay per view movie at home and a user connected to the personal broadcaster tunes into the same movie the personal broadcaster can disable its A V outputs Whether the A V outputs are disabled can depend on the content For example in one embodiment only pay per view content trig gers a blocking whereas regular programming does not This discrimination scheme can be integrated with an EPG as discussed above Summary The foregoing description of the embodiments of the invention has been presented for the purpose of illustration it is not intended to be exhaustive or to limit the invention to the precise forms disclosed Persons skilled in the relevant art can appreciate that many modifications and variations are pos sible in light of the above teachings It is therefore intended that the scope of the invention be limited not by this detailed description but rather by the claims appended hereto What is claimed is
21. types but not limited to desktop and notebook PCs Web tablets PDAs embedded clients built expressly for the purposes of decoding the streams of the personal broadcaster and other devices capable of receiving and or playing a media stream over a network In one embodiment the local clients 150 and or the remote clients 170 execute a client software application that includes a user interface for requesting content from the broadcaster 100 and for viewing that content In another embodiment the client functionality is provided by a Web site and is accessible by the local clients 150 and or the remote clients 170 via a Web browser When a remote client 170 wishes to connect to the stream of the personal broadcaster 100 using the client application or via a Web browser interface it may specify the home IP address of the user to access and pull the media stream from the personal broadcaster This action sends a request to the personal broadcaster and the request travels across the public Internet to the network ofthe user s Internet service provider ISP into the home via the telephony or cable infrastructure or wirelessly in the case of the fixed wireless or satellite broadband ISP to the home Internet gateway 110 and finally to the personal broadcaster 100 In one embodiment a central server 180 is coupled to the remote network 160 and provides one or more roles includ ing that ofDNS server Because most residential ISPs allocate dy
22. variability of network bandwidth over time calls for a system that is dynamic in nature and capable of real time changes to the encoder settings To address this variability one embodiment of the present invention implements a feed back loop between the client and personal broadcaster to maintain the proper encoder settings over time This feedback loop can be implemented in a number of ways In one embodi ment the client gives notification to the personal broadcaster when it experiences frame drops Alternatively the client communicates from time to time the size ofthe buffer or the total measured throughput The client may communicate any or all of these statistics or any other data that reflects on the need to adjust the settings Given this feedback from the client the personal broad caster alters the encoder settings For example the personal broadcaster can reduce the encoding bit rate from 350 kbps to 280 kbps if the size of the buffer becomes reduced or an unacceptable number of frame drops are observed The time between measurements and feedback received from the client can vary and the figure depends on a balance between react ing quickly to significant changes and overcorrecting based on temporary blips Based on the feedback the personal broadcaster can adjust the settings upward e g increase encoding bit rate as well as downward While one embodiment provides for a method of automatic adjustment of encoder settings t
23. video samples may be changed according to the existing playback speed to reduce jerks in the video stream As a result the change in the audio stream is also not perceptible and the user is less likely to perceive the difference between the normal streaming speed and the initial fast start playback speed Control Mode For Low Latency One of the most used features in TV viewing is channel control and users expect such control operations to take at most one second to execute But streaming video over IP networks performs best when some delay is added between 0 40 45 16 the server and the client This delay is needed to fill the A V buffers in the media player client The two requirements of low delay for user interactions and smooth audio video dis play are conflicting in nature To deal with both of these requirements in one embodiment of the invention two modes of operations are introduced normal mode and control mode In the normal mode the system performs conventional audio video streaming wherein a buffer of five or more sec onds is maintained by the media player to ensure smooth playback In this mode the media broadcaster may also start buffering data if the network bit rate drops below encoder bit rate i e the encoder at the broadcaster runs ahead of the media stream transmission The total delay between video input to the media broadcaster and its viewing on the media player is thus the sum of three param
24. 006 Logan 2002 0012530 Al 1 2002 Bruls 2006 0031381 Al 2 2006 Van Luijt et al 2002 0031333 Al 3 2002 Mano etal 2006 0050970 Al 3 2006 Gunatilake 2002 0046404 Al 4 2002 Mizutani 2006 0051055 Al 3 2006 Ohkawa 2002 0053053 Al 5 2002 Nagai et al 2006 0095401 Al 5 2006 Krikorian et al 2002 0080753 Al 6 2002 Lee 2006 0095472 Al 5 2006 Krikorian et al 2002 0090029 Al 7 2002 Kim 2006 0095942 Al 5 2006 Van Beek 2002 0105529 Al 8 2002 Bowser et al 2006 0095943 Al 5 2006 Demircin et al 2002 0112247 Al 8 2002 Horner et al 2006 0107226 A1 5 2006 Matthews et al 2002 0120925 Al 8 2002 Logan 2006 0117371 Al 6 2006 Margulis 2002 0122137 Al 9 2002 Chen et al 2006 0146174 Al 7 2006 Hagino 2002 0131497 Al 9 2002 Jang 2006 0280157 A1 12 2006 Karaoguz et al 2002 0138843 Al 9 2002 Samaan et al 2006 0280437 Al 12 2006 Logan etal 2002 0143973 Al 10 2002 Price 2007 0003224 Al 1 2007 Krikorian et al 2002 0147634 Al 10 2002 Jacoby et al 2007 0005783 Al 1 2007 Saint Hillaire et al 2002 0147687 Al 10 2002 Breiter et al 2007 0022328 Al 1 2007 Tarra et al 2002 0167458 Al 11 2002 Baudisch et al 2007 0074115 Al 3 2007 Patten et al 2002 0188818 Al 12 2002 Nimura et al 2007 0076604 Al 4 2007 Litwack 2002 0191575 Al 12 2002 Kalavade et al 2007 0113250 Al 5 2007 Logan etal 2003 0001880 Al 1 2003 Holtz et al 2007 0168543 Al 7 2007 Krikorian et al 2003 0028873 Al 2 2003 Lemmons 2007 0180485 Al 8 2007 Dua 2003 0065915 Al 4 2003 Yu etal 2007 0198532 Al 8 2007 Krikorian e
25. 1 system comprising a personal media broadcaster configured to receive an uncompressed signal from an audio video source device to encode a media stream from the uncom pressed signal using an encoder setting to transmit the media stream over a network and to adjust the encoder setting based upon variations in an available bandwidth ofthe network and a media player client configured to communicate with the personal media broadcaster over the network to receive the media stream and to play the media stream for a user 20 25 30 35 40 45 50 55 60 65 26 2 The personal media broadcasting system of claim 1 wherein the media player client is configured to receive a user command to control the media stream and to communicate the user command to the personal media broadcaster and wherein the personal media broadcaster is configured to adjust the encoder setting responsive to the user command 3 The personal media broadcasting system of claim 1 wherein the personal media broadcaster includes an interface to receive the uncompressed signal from a plurality of types of audio video source devices 4 The personal media broadcasting system of claim 1 wherein the personal media broadcaster is configured to dynamically adjust the encoder setting in response to the variations in the available bandwidth of the network as the media stream is transmitted over the network 5 personal media broadcasting system
26. 4 Horoszowski et al EP 0838945 4 1998 2004 0236844 Al 11 2004 Kocherlakota EP 1077407 Al 2 2001 2004 0255249 Al 12 2004 Chang et al EP 1443766 A2 8 2004 2004 0255330 Al 12 2004 Logan EP 1691550 A 8 2006 2004 0255334 Al 12 2004 Logan EP 1830558 Al 9 2007 2004 0255336 Al 12 2004 Logan etal GB 2307151 5 1997 2005 0005308 Al 1 2005 Logan etal JP 2003046582 A 2 2003 2005 0021398 Al 1 2005 McCleskey et al JP 2003114845 A 4 2003 2005 0027821 A1 2 2005 Alexander et al JP 2004015111 A 1 2004 2005 0038981 Al 2 2005 Connor et al KR 19990082855 A 11 1999 2005 0044058 Al 2 2005 Matthews et al KR 20010211410 A 8 2001 2005 0050462 Al 3 2005 Whittle et al wo 0133839 Al 5 2001 2005 0053356 A1 3 2005 Mate etal WO WO 01 33839 5 2001 US 7 877 776 B2 Page4 WO 0147248 A2 6 2001 WO 0193161 A1 12 2001 WO 03026232 1 3 2003 WO 03052552 A2 6 2003 WO WO 03 052552 6 2003 WO 03098897 A 11 2003 WO 2004032511 Al 4 2004 WO 2005050898 A2 6 2005 WO 2006064454 1 6 2006 WO 2006074110 A 7 2006 WO 2007027891 A2 3 2007 WO 2007051156 A2 5 2007 WO 2007141555 A2 12 2007 WO 2007149466 A2 12 2007 WO 2008024723 A 2 2008 OTHER PUBLICATIONS http web archive org web 2000081505275 l http www myreplaytv com Aug 15 2000 entire document Faucon Brick Opens up Copyright Can of Worms Finan cial Review Jul 1 2003 online Retrieved on Apr 12 2006 Retrieved from the Internet lt URL http afr com cgi bin newtextversions
27. 7 filed Apr 12 2007 Goma International Search Report for PCT US2008 069914 mailed Dec 19 2008 PCT Partial International Search PCT US2009 054893 mailed Dec 23 2009 European Patent Office European Search Report mailed Sep 28 2009 for European Application No EP 06 78 6175 Ditze M et all Resource Adaptation for Audio Visual Devices in the UPnP QoS Architecture Advanced Networking and Applications 2006 AINA 2006 20 H International conference on Vienna Aus tria Apr 18 20 2006 Joonbok Lee et al Compressed High Definition Television HDTV Over IPv6 Applications and the Internet Workshops 2006 Saint Workshops 2006 International Symposium Phoenix AZ USA Jan 23 27 2006 Lowekamp B et al A Hierarchy of Network Performance Charac teristics for Grid Applications and Services GGF Network Mea surements Working Group pp 1 29 May 24 2004 Newton s Telcom Dictionary 21st ed Mar 2005 Sling Media Sling Media Unveils Top of Line Slingbox PRO HD online Jan 4 2008 XP002560049 retrieved from the Internet URL www slingmedia com get pr slingbox pro hd html retrieved on Oct 12 2009 Srisuresh P et al Traditional IP Network Address Translator Tra ditional NAT Network Working Group The Internet Society Jan 2001 Lucas Brian et al Systems and Methods for Establishing Connec tions Between Devices Communicating Over a Network U S Appl
28. 76 B2 Central Server 180 A N Source Devices 120 Home Internet Personal Gateway Broadcaster On Network 110 100 Storage Device 130 Local Clients 150 Remote Clients 170 FIG 1 Analog Input Cable Feed Control Home Internet Gateway 110 Network OUT 290 AN Source Device 120 IR Emitter FIG 2 US 7 877 776 B2 Sheet 2 of 2 Jan 25 2011 U S Patent cee eoeyelul 2 See Oct ooejiaju WOMAN A 40SS820Jd oipny 1u51 91 EP 2 506 indul 5 2 JH US 7 877 776 B2 1 PERSONAL MEDIA BROADCASTING SYSTEM CROSS REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U S Provisional Application No 60 577 833 filed Jun 7 2004 which is incorporated by reference in its entirety This application is also related to co pending U S Application entitled Fast Start Streaming and Buffering of Streaming Content for Per sonal Media Player to Krikorian et al filed Jun 7 2005 and co pending U S Application entitled Personal Media Broadcasting System With Output Buffer to Krikorian et al filed Jun 7 2005 each of which is incorporated by reference in its entirety BACKGROUND 1 Field of the Invention This invention relates generally to personal streaming media broadcasters and in particular to streaming media from a media source in
29. 8 152 XP010519373 ISBN 978 0 7695 0253 3 abstract paragraph 03 1 paragraph 05 2 figures 1 2 USPTO Non Final Office Action mailed Dec 17 2004 U S Appl No 09 809 868 filed Mar 15 2001 USPTO Final Office Action mailed Jul 28 2005 U S Appl No 09 809 868 filed Mar 15 2001 USPTO Non Final Office Action mailed Jan 30 2006 U S Appl No 09 809 868 filed Mar 15 2001 USPTO Final Office Action mailed Aug 10 2006 U S Appl No 09 809 868 filed Mar 15 2001 USPTO Non Final Office Action mailed Jun 19 2007 U S Appl No 09 809 868 filed Mar 15 2001 USPTO Non Final Office Action mailed Apr 16 2008 U S Appl No 09 809 868 filed Mar 15 2001 USPTO Final Office Action mailed Sep 18 2008 U S Appl No 09 809 868 filed Mar 15 2001 USPTO Non Final Office Action mailed Mar 31 2009 U S Appl No 09 809 868 filed Mar 15 2001 USPTO Non Final Office Action mailed May 1 2008 U S Appl No 11 111 265 filed April 21 2005 USPTO Final Office Action mailed Dec 29 2008 U S Appl No 11 111 265 filed Apr 21 2005 USPTO Non Final Office Action mailed Jun 8 2009 U S Appl No 11 111 265 filed Apr 21 2005 USPTO Non Final Office Action mailed Jun 26 2008 U S Appl o 11 620 707 filed Jan 7 2007 SPTO Final Office Action mailed Oct 21 2008 U S Appl No 11 620 707 filed Jan 7 2007 USPTO Non Final Office Action mailed Mar 25 2009 U S Appl
30. 83 6 889 385 6 892 359 6 898 583 6 907 602 6 927 685 6 930 661 6 931 451 6 941 575 6 944 880 6 952 595 6 981 050 7 016 337 7 020 892 7 032 000 7 047 305 7 055 166 7 058 376 7 110 558 10 2002 11 2002 11 2002 12 2002 12 2002 1 2003 1 2003 3 2003 4 2003 4 2003 4 2003 5 2003 5 2003 6 2003 6 2003 7 2003 7 2003 7 2003 8 2003 8 2003 9 2003 11 2003 11 2003 12 2003 12 2003 12 2003 2 2004 3 2004 3 2004 3 2004 3 2004 4 2004 6 2004 6 2004 6 2004 6 2004 7 2004 7 2004 8 2004 8 2004 8 2004 8 2004 9 2004 9 2004 10 2004 10 2004 11 2004 11 2004 11 2004 12 2004 1 2005 1 2005 2 2005 2 2005 3 2005 5 2005 5 2005 5 2005 6 2005 8 2005 8 2005 8 2005 9 2005 9 2005 10 2005 12 2005 3 2006 3 2006 4 2006 5 2006 5 2006 6 2006 9 2006 Tracton et al Plotkin et al Chai Humpleman Sampsell Bhagavath et al De Bonet et al Yamamoto et al Chai et al Abecassis Mimura et al Kadono Allport Konstantinou et al Huh et al Yawitz McAlister et al Chui Swix et al Apostolopoulos Tan et al Vallone et al Malkemes et al Chen et al Chen et al Forsman et al Kretschmer et al Schneider et al Minke et al Six et al Kitagawa Swix et al Bahl et al Hensley et al Park et al Look et al Price Wen et al Malvar Ahmed et al Cheung Masud et al Skelley Jr Ngo Radha et al Chui et al Zhang et al Coden et al Ojard et al Acharya et al Barlow et al Ferriere Tard
31. 831 664 5 850 482 5 852 437 5 880 721 5 892 536 5 898 679 5 909 518 5 911 582 5 922 072 5 936 968 5 953 485 5 968 132 5 987 501 6 002 450 6 008 777 6 014 694 6 020 880 6 031 940 6 036 601 6 040 829 6 043 837 6 049 671 6 075 906 6 088 455 6 088 777 6 091 886 6 097 441 6 104 334 6 108 041 6 115 420 6 117 126 6 141 059 6 141 447 6 151 444 6 160 544 6 201 536 6 208 805 6 212 282 6 222 885 6 223 211 6 240 459 6 240 531 6 243 596 6 256 019 6 263 503 6 279 029 6 282 714 6 286 142 6 340 994 6 353 885 6 356 945 6 357 021 6 370 688 6 389 467 6 408 128 6 434 113 6 442 067 6 456 340 6 466 623 17 5 PATENT DOCUMENTS gt gt gt gt gt gt gt gt gt gt gt lt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt 1 10 1997 11 1997 12 1997 1 1998 1 1998 1 1998 2 1998 5 1998 6 1998 7 1998 8 1998 10 1998 11 1998 12 1998 12 1998 3 1999 4 1999 4 1999 6 1999 6 1999 7 1999 8 1999 9 1999 10 1999 11 1999 12 1999 12 1999 1 2000 2 2000 2 2000 3 2000 3 2000 3 2000 4 2000 6 2000 7 2000 7 2000 7 2000 8 2000 8 2000 8 2000 9 2000 9 2000 10 2000 10 2000 11 2000 12 2000 3 2001 3 2001 4 2001 4 2001 4 2001 5 2001 5 2001 6 2001 7 2001 7 2001 8 2001 8 2001 9 2001 1 2002 3 2002 3 2002 3 2002 4 2002 5 2002 6 2002 8 2002 8 2002 9 2002 10 2002 Hendricks et al
32. Abecassis Abecassis Shaw et al Hylton et al Nelson Freadman Birch et al Hite et al Davidson Matsuda et al Katseff et al Wharton et al Meany et al Wugofski et al Yen Logan et al Brederveld et al Chui Redford et al Hutchinson et al Lyons Abecassis Tokunaga Hamilton et al Darbee et al Yiu Aharoni et al Naimpally Chui et al Heckel Croy et al Driscoll Jr et al Slivka et al Fenwick et al Logan et al Sorber Abecassis Allport Allport Faroudja et al Wang Appelbaum et al Boyce et al Linzer et al Abecassis Hayashi et al Hendricks et al Abecassis Mershon Chaddha et al Hamilton et al Roberts et al Spilo et al Kikinis Allport Margulis Sampat et al Ghori et al Ehreth Margulis et al Herzi et al Shaw et al Kitagawa et al Hejna Jr Eyal Abecassis Gubbi Chawla et al Margulis Youn et al 6 470 378 6 476 826 6 487 319 6 493 874 6 496 122 6 505 169 6 510 177 6 529 506 6 553 147 6 553 178 6 557 031 6 564 004 6 567 984 6 584 201 6 584 559 6 597 375 6 598 159 6 600 838 6 609 253 6 611 530 6 628 716 6 642 939 6 647 015 6 658 019 6 665 751 6 665 813 6 697 356 6 701 380 6 704 678 6 704 847 6 708 231 6 718 551 6 754 266 6 754 439 6 757 851 6 757 906 6 766 376 6 768 775 6 771 828 6 774 912 6 781 601 6 785 700 6 795 638 6 798 838 6 806 909 6 807 308 6 816 194 6 816 858 6 826 242 6 834 123 6 839 079 6 847 468 6 850 571 6 850 649 6 868 0
33. CT International Search Report and Written Opinion PCT US2005 020105 Feb 15 2007 6 pages Roe Kevin Third Party Observation Under EPC Article 115 On The Patentability Of An Invention Dec 21 2007 Roe Kevin Third Party Submission For Published Application Under CFR 1 99 Mar 26 2008 Skodras et al JPEG2000 The Upcoming Still Image Compression Standard May 11 2000 14 pages Kessler Gary C An Overview of TCP IP Protocols and the Internet Jan 16 2007 retrieved from the Internet on Jun 12 2008 at http www garykessler net library tcpip html originally submitted to the InterNIC and posted on their Gopher site on Aug 5 1994 Taubman et al Embedded Block Coding In JPEG2000 Feb 23 2001 pp 1 8 of 36 Balster Eric J Video Compression and Rate Control Methods Based on the Wavelet Transform The Ohio State University 2004 pp 1 24 Kulapala et al Comparison of Traffic and Quality Characteristics of Rate Controlled Wavelet and DCT Video Arizona State University Oct 11 2004 International Search Report and Written Opinion for PCT US2006 04382 mailed Apr 27 2007 International Search Report for International Application No PCT US2007 076337 mailed Oct 20 2008 Rao Padmanabha R U S Appl No 12 347 465 filed Dec 31 2008 European Patent Office European Search Report for European Application No EP 08 16 7880 mailed Mar 4 2009 MythTV Wiki MythTV User Ma
34. O Final Office Action mailed Jul 31 2009 U S Appl No 1 683 862 filed Mar 8 2007 USPTO Non Final Office Action mailed Aug 5 2009 U S Appl No 11 147 663 filed Jun 7 2005 USPTO Non Final Office Action mailed Sep 3 2009 U S Appl No 11 620 711 filed Jan 7 2007 Einaudi Andrew E et al Systems and Methods for Selecting Media Content Obtained from Multiple Sources U S Appl No 12 543 278 filed Aug 18 2009 Malode Deepak Ravi Remote Control and Method for Automati cally Adjusting the Volume Output of an Audio Device U S Appl No 12 550 145 filed Aug 28 2009 Akella Aparna Sarma Systems and Methods for Event Program ming Via a Remote Media Player U S Appl No 12 537 057 filed Aug 6 2009 Shah Bhupendra Natwerlan et al Systems and Methods for Transcoding and Place Shifting Media Content U S Appl No 12 548 130 filed Aug 26 2009 Banger Shashidhar et al Systems and Methods for Automatically Controlling the Resolution of Streaming Video Content U S Appl No 12 537 785 filed Aug 7 2009 Panigrahi Biswaranjan Home Media Aggregator System and Method U S Appl No 12 538 681 filed Aug 10 2009 Nandury Venkata Kishore Adaptive Gain Control for Digital Audio Samples Media Stream U S Appl No 12 507 971 filed Jul 23 2009 Shirali Amey Systems and Methods for Providing Programming Content U S Appl No 12 538 676
35. TO Non Final Office Action mailed Mar 31 2010 U S Appl No 11 620 707 filed Jan 7 2007 USPTO Non Final Office Action mailed Apr 1 2010 U S Appl No 12 237 103 filed Sep 24 2008 Qiong Liu et al Digital Rights Management for Content Distribu tion Proceedings of the Australasian Information Security Work shop Conference ACSW Frontiers 2003 vol 21 2003 XP002571073 Adelaide Australia ISSN 1445 1336 ISBN 1 920682 00 7 sections 2 and 2 1 1 China State Intellectual Property Office Office Action issued Mar 18 2010 for Application No 200680022520 6 Canadian Intellectual Property Office Office Action mailed Feb 18 2010 for Application No 2569610 European Patent Office European Search Report mailed May 7 2010 for Application No 06786174 0 China State Intellectual Property Office Office Action issued Apr 13 2010 for Application No 200580026825 X Margulis Neal Apparatus and Method for Effectively Implementing Wireless Television System U S Appl No 12 758 193 filed Apr 12 2010 Margulis Neal Apparatus and Method for Effectively Implementing Wireless Television System U S Appl No 12 758 194 filed Apr 12 2010 Margulis Neal Apparatus and Method for Effectively Implementing Wireless Television System U S Appl No 12 758 196 filed Apr 12 2010 Kirkorian Jason Gary et al Personal Media Broadcasting System with Output Buffer
36. The encoding bit rate can be decreased each time the free space dips below a watermark while it can be increased when the free space rises above a watermark or if the free space never drops below the watermark the first place In this way the encoding bit rate can be dynamically adjusted based oncurrent performance ofthe system and this dynamic adjustment can occur at the broadcaster without requiring feedback from the client In another embodiment the media player client imple ments a fast start mechanism by which the media player US 7 877 776 B2 3 receives a media stream in real time from the personal media broadcaster and stores the media stream in a buffer The media player client plays the received media stream from the buffer at a decreased playback speed without waiting for the bufferto fill to a predetermined threshold While the playback speedis keptata slower rate than the streaming media content is received the buffer fills albeit at a rate slower than the media stream is received This allows the media player client to play a received media stream without waiting for its buffer to fills which improves the user s experience dramatically in situations such as where the user is changing channels or operating a menu on the remote A V source device In addi tion the media player client may further enhance this expe rience by switching into a control mode when the client receives a user command to control content in the
37. a message to another external machine If the personal broad caster sits behind a full cone NAT the following is possible personal broadcaster from time to time sends a message to the central server and the central server makes a note ofthe IP address and port from which it was received When a remote client wishes to connect to the personal broadcaster it asks the central server for the address and port recently used by the personal broadcaster to send its message The remote client can use the same IP address and port to locate and connect to the personal broadcaster If the NAT is not a full cone NAT another method is used A third possible method to use comprises a UDP hole punching technique This technique which is known to those of ordinary skill in the relevant art would involve using the central sever as a way to introduce the remote client and the personal broadcaster The method works for all NAT types except for symmetric NATs so the STUN test would be useful for determining if this is a viable option and uses the central server to cause both devices to send messages to one another simultaneously Because both devices believe they are initiating the conversation the return packets are per mitted to flow through the NAT to the destination device If none of the above or other methods are viable solutions the present invention can walk the user through the steps for manual configuration of the NAT on t
38. a remote network This is because incoming requests are rejected by many NATs unless the NAT has been explicitly instructed how to forward the incoming packets There are many methods for solving this problem some more desirable and automated than others Some embodiments of the inven tion include a multi step process for determining and imple menting the possible solutions The system may first attempt to determine whether the home Internet gateway supports UPnP Universal Plug N Play If so the personal broadcaster may be able to control the home Internet gateway using UPnP The personal broad 20 25 30 35 40 45 50 55 60 65 24 caster can thus instruct the NAT to forward a specified port to the internal IP address dedicated to the personal broadcaster If the home Internet gateway does not support UPnP the system may then attempt to determine the type of NAT in the home Internet gateway and specifically whether the NAT is a full cone NAT This detection may be performed by using the central server as a STUN server which runs various tests to determine what the type of the NAT behind which the per sonal broadcaster sits There are four basic types of NATs full cone restricted cone port restricted cone and symmetric full cone NAT is a NAT that allows a client behind it to receive messages from one external machine that are addressed to an IP address and port that the internal client used in sending
39. at least two parameters the minimum data that can be generated by the encoder and the maximum network down time that has to be supported Although the system cannot prevent data loss when available 30 40 45 50 55 65 14 bandwidth goes below minimum bandwidth required by the broadcaster for extended period of time a larger buffer helps to reduce this risk As the data for the media stream are being generated by the broadcaster the intermediate buffer acts as a FIFO queue When the available network bandwidth is more than the encoder s bandwidth the broadcaster is able to send data as soon as it is generated The intermediate buffer will begin to empty When the available network bandwidth is less than the encoder s bandwidth the broadcaster will generate the data faster than it can be transmitted This will begin to fill the buffer The buffer may therefore vary between being com pletely full and completely empty To classify the occupancy ofthe buffer a number of watermarks are defined to indicate the amount of free space left in the buffer While the number of watermarks can vary in one embodiment four watermarks are used at the 90 75 50 and 30 levels As data are added to and taken from the buffer the amount of data filling the buffer can change over time When this level reaches one of the watermarks various actions are taken depending on which watermark has been reached amount of free space lef
40. ate of the source media signal is reduced and its encoding format is possibility changed The personal broadcaster 100 may perform this transrating and transcoding functionality In this situation the personal broadcaster 100 acts as a networked attached transrater and transcoder The broadcaster receives the media stream from the on network storage device 130 transrates and possibly transcodes the media and outputs a media stream with a sufficiently low bit rate so the media can be effectively streamed upstream from theuser s broadband service The method for determining the proper transrating or transcoding settings e g the bit rate to which the source content is to be reduced can be accom plished within the framework for determining the throughput currently supported between the personal broadcaster and the local or remote client outlined below The use of the personal broadcaster 100 as an agent to transrate and or transcode the material residing on one or more on network storage devices 130 has the benefit of cre ating a system where only one device the personal broad caster streams media upstream through the home Internet gateway 110 and out to the Internet 160 This is beneficial because the user does not have to make further configurations to the NAT or firewall of the home Internet gateway which can include manually forwarding a port to allow direct access to each on network storage device In addition to requiring further c
41. atency on the client side In one embodiment upon a change into control mode the media broadcaster stops buffering data and flushes data cur rently contained in its buffers The media broadcaster then immediately generates an I frame also known as key frame to send to the media player The I frame allows the media player to reconstruct an image whereas other types of frames that encode the frame based on previous frames would not allow the media player to reconstruct the frame due to the discontinuity in the media stream Given the dependency of modern audio video encoding standards however it may not be desirable for the media broadcaster to stop buffering data completely This is because without buffering there maybe too much data loss leading to an extremely poor user experience A compromise can there fore be made to balance low delay and a reasonable user experience Based on a set of empirical values for achieving good balance an amount of data is allowed to be buffered by the media broadcaster during control mode as a function of bit rate of content For example in control mode buffered data can be reduced by flushing the data if it crosses a limit of about one second however other empirical values may be used for various applications In one embodiment the media broadcaster keeps track of effective transmission bit rate for example by periodically calculating the average transmission bit rate for the last few second
42. cally at first connection or can be manually ordered by the user to recalibrate initial optimization Based on the time it takes for the client to receive those bits the client has an idea of the actual data throughput between the personal broadcaster and the client With this information in hand the client instructs the personal broadcaster to begin streaming at a rate compatible with this throughput This is not likely to be the full rate of throughput but some rate less for example 80 of throughput to allow for inevitable variation in network bandwidth In choosing the proper reso lution setting the application residing on the client notes its current capabilities 1 resolution of its display and pairs the appropriate bit rate setting with the appropriate resolution setting in its command to the personal broadcaster The client can learn of its capabilities in several ways One can be related to the version of the application itself For example the application for a Pocket PC can know that the device best supports streams at or below a certain resolution and frame rate Another way is to take inventory of system resources before it sends the request For example the client can identify its display resolution and incorporate this infor mation in the streaming request to the personal broadcaster The initial optimization process represents a starting point that can very well provide for the proper encoder settings However the
43. ce 120 in this example is controlled via IR controls called for by the remote client 170 or local client 150 are relayed from the personal broadcaster 100 to the A V source device 120 via an IR emitter 285 e g an IR blaster assembly One end of the IR emitter 285 is plugged into the personal broadcaster 100 which blasts appropriate IR codes through a wire and out an IR LED of the IR emitter 285 Accordingly the IR emitter 285 is placed directly in front of the IR receiver of the A V source device 120 The personal broadcaster 100 connects to the home Inter net gateway 110 from its Ethernet port 290 using e g Cat5 cable which connection can be direct or via an Ethernet wall jack located near the personal broadcaster 100 which in turn connects to the home Internet gateway 110 In other embodi ments the connection between the personal broadcaster 100 and the home Internet gateway 110 is wireless where the broadcaster 100 may include built in wireless or power line networking capability FIG 3 is a block diagram showing the internal components of the personal media broadcaster 100 according to one embodiment of the invention As shown the broadcaster 100 includes an input interface 305 for receiving any of a variety of input types including an RF signal from analog cable or antenna an S video signal a composite video signal and a left right audio signal Because an RF signal includes a num ber of video signals modulat
44. crosoft Outlook Lotus Notes and others as well as any Internet based messaging services such as MSN Messenger and AOL Instant Messenger Personal Video Recorder PVR Functionality For Mobile Devices As users get more accustomed to video experiences on mobile devices and mobile storage solutions e g removable flash media and small hard disk drives become more robust and inexpensive the desire to add personal video recorder functionality to these devices will emerge One embodiment of the present invention adds the functionality of a personal video recorder to a mobile phone communicator PDA or other device connecting to a Wide Area Network or other remote network outside a user s local area network One variation that accomplishes this can feature a sched uler as part ofthe remote client application which coordinates with an electronic programming guide When the user launches the remote client application he can search for and select the shows he would like to have recorded to his remote client When the time for the scheduled recording occurs the remote client application initiates a recording The applica tion can call the connection manager on the remote client which in turn opens an Internet connection The remote client application then connects to the personal broadcaster at the user s home Next the remote client application can issue the right set of commands related to channel selection and encoder settings on the se
45. d Playback U S Appl No 12 323 907 filed Nov 26 2008 Rao Padmanabha R Systems and Methods for Linking Media Content U S Appl No 12 359 784 filed Jan 26 2009 Krikorian Blake Gary et al Systems and Methods for Presenting Media Content Obtained From Multiple Sources U S Appl No 12 408 456 filed Mar 20 2009 Krikorian Blake Gary et al Systems and Methods for Projecting Images From a Computer System U S Appl No 12 408 460 filed Mar 20 2009 International Search Report and Written Opinion for International Application No PCT US2006 025911 mailed Jan 3 2007 International Search Report for International Application No PCT US2007 063599 mailed Dec 12 2007 International Search Report and Written Opinion for International Application No PCT US2008 080910 mailed Feb 16 2009 International Search Report and Written Opinion for International Application No PCT US2006 025912 mailed Jul 17 2008 International Search Report for International Application No PCT US2008 059613 mailed Jul 21 2008 Wikipedia LocationFree Player Online Sep 22 2007 XP002512400 retrieved from the Internet URL http en wikipedia org w index php title LocationFree Player amp oldid 159683564 gt retrieved on Jan 28 2009 Wikipedia Slingbox Online Oct 21 2007 XP002512399 retrieved from the Internet lt URL http en wikipedia org w index php title Slingbox amp oldid 166080570 gt retrie
46. deo Clips From a Stream ing Source Embodiments of the invention also allow people to send video clips to friends and acquaintances In accordance with one embodiment the media player device continually cap tures and caches the last five minutes or some other desig nated time period of video and audio received The media player automatically discards the earliest recorded content when the cached stream hits five minutes or some other designated time period so that the five minutes being cached is always the five most recent minutes of media streamed When a user sees something of interest that the user would like to capture and save or share the user clicks on or selects a designated button or command using the media player s graphical interface Once selected the media player brings up a simple video editing interface which enables the user to select the beginning and end of the clip that the user would like to capture Once the bounds of the clip have been selected the user selects a command to save the clip or send the clip to another person In response to a command to send the captured video clip the media player calls the email client resident on the user s client device creates a new message and attaches the video clip to the message The user can then select the intended recipients of the message and send the message The address book databases can be those available from email clients such US 7 877 776 B2 19 as Mi
47. digital converter configured to convert the uncompressed signal from the analog format to a digital format 31 The device of claim 18 wherein the uncompressed signal is received in a digital format 32 The device of claim 18 wherein the processor is con figured to adjust the encoder setting while the media stream is transmitted on the network 33 method comprising receiving an uncompressed signal encoding the uncompressed signal to construct a media stream adjusting the encoding based upon variations in an avail able bandwidth of a network and transmitting the media stream to a client over the network 34 The method of claim 33 wherein the uncompressed signal is a live stream provided by a source device 35 method of claim 33 wherein the adjusting com prises dynamically adjusting the encoder setting based on the variations in the available bandwidth of the network as the media stream is transmitted over the network 36 The method of claim 33 wherein the adjusting com prises adjusting the encoder setting in real time as the media stream is transmitted over the network 20 25 30 35 40 45 50 55 28 37 The method of claim 33 wherein the encoding com prises compressing the uncompressed digital signal 38 The method of claim 33 wherein the encoding com prises compressing the uncompressed digital signal using an encoder setting 39 The method of claim 38 wherein the adjusting com pr
48. ed therein the input interface 305 is coupled to provide the RF input to a tuner 310 The tuner 310 filters the RF signal for a selected channel demodulates the channel and converts the signal into separate analog video and audio for further processing by an audio video decoder 315 The input interface 305 is coupled to provide the S video signal a composite video signal and a left right audio signal directly to the A V decoder 315 as those signals need not be processed by a tuner 310 In one embodiment the A V decoder 315 converts the analog video input into YUV video and applies various types of filters and color corrections to the signal The A V decoder 315 also extracts vertical blanking interval VBI data such as close caption tele text and copy protection bits The A V decoder 315 also decodes the tuner audio and converts it into stereo or mono digital audio depending on the broadcast signal The analog signal is further converted into a digital signal in the A V decoder 315 The digital video and audio from the A V decoder 315 is then sent to a processor 320 for further processing The personal broadcaster 100 may include memory 330 such as flash memory or SDRAM used by the processor 320 for performing its associated processing tasks The memory 330 may also be used as a buffer for the outgoing media stream as described herein for various embodiments In one embodiment the processor 320 performs pre pro cessing on the digita
49. el 1 as long as i is not 0 SM for level i 1 is checked after time interval of T If the SM is greater than 21 it indicates that the current content be encoded at level 1 1 so the encoder moves to level i 1 This method thus tries to maintain the inter frame distances constant at each level thereby improving the overall video quality Dynamic Parameter Control For Video Encoder In areal time video streaming environment network char acteristics change dynamically To improve the use of the network resources a video encoder should be able to adapt to these network changes however most of the standard video encoders do not support such adaptation In accordance with one embodiment of the invention a scheme for the Windows Media Video WMV9 encoder is provided wherein the parameters can be changed dynamically during streaming The following parameters can be changed dynamically while streaming is in progress bit rate frame rate video smoothness and I frame interval The bit rate can adapt to the rate that can be supported by the network at a given time and rate control buffer delay bits per frame and quantization step size vary according to changes in bit rate Frame rate depends on the dynamically changed bit rate At lower bit rates high frame rates cannot be sustained which creates the need in US 7 877 776 B2 23 some circumstances for a dynamic frame rate The video smoothness parameter indicates the encoded vid
50. eld remote control In one embodiment therefore a virtual remote control is pro vided by the client application that includes either a generic image representing the A V source device s remote control or animage or likeness ofthe actual A V source device s remote control Moreover the client application may support a num ber of virtual remote controls one customized for each A V source device 120 Interfaces on the client application are thus selected by the user to resemble each particular A V source device 120 found in the user s home For example if a TiVo personal video recorder is connected to the personal broad caster the user can decide to use the TiVo skin which modifies the virtual remote control on the client application to resemble the TiVo remote control Remote control commands are mapped to the graphical image in such a way that a press ofthe button triggers the action suggested by the image ofthe button e g pressing on CH button turns the channel from 3 to 4 In one embodiment the client application contains a data base of remote control skins from which to choose In this case a central database maintained on the central server is likely to update the client application upon configuration to ensure that the latest remote control skins are available to the users In addition third parties may be allowed to create and share images with commands mapped to particular regions of the image In this case a method for
51. eo buffers This buffer is maintained to ensure smooth playback When going into control mode the media player flushes the data that is present in the A V buffers and the network buffer In one embodiment when going into control mode the media player makes the source filter flush all the data buffered in the A V buffers and also in all the filters downstream decoders and renderers which may be holding 2 3 seconds worth of content SP then flushes all the data present in the network buffer Thereafter the media player sends a notification to the media broadcaster to go into control mode and waits for the next I frame received from the media broadcaster After the flush operation on the source filter and the net work buffer discontinuities occur in the media stream The filter has intelligence built in so that if there is any packet discontinuity in the media stream the filter waits for the next I frame Any incoming data that is not an I frame is discarded by the source filter until a valid I frame is detected When the media broadcaster goes into control mode it sends an I frame immediately As soon as this I frame is detected by the source filter it is sent downstream for rendering While the media player remains in control mode no buffering occurs in the source filter the samples are sent downstream for rendering US 7 877 776 B2 17 as soon as they are read from the network buffer In this manner the media player reduces the l
52. eo quality Lower bit rates supporting high values of video smoothness can cause jerky video while low values of video smoothness at higher bit rates leads to under utilization of the available resources Depending on the video smoothness parameter the quantization upper bound and lower bound step values are changed which affects the quality of the video Because the bits required to encode I frames are greater than bits required for P frames I frame intervals are larger for lower bit rates I frame intervals can be reduced at higher bit rates where more bits can be afforded Accordingly the encoder in the media broadcaster can dynamically adjust these parameters to adapt to the changing network characteristics and optimize use of system resources This allows for improved performance than the standard encoder which does not dynamically change the above parameters in a real time streaming environment Although described in the context of video encoding with Windows Media Video this method can be applied to other video compression formats including MPEG 4 H 263 H 264 and any other compression formats that use the same or similar parameters mentioned herein Point and Click Interaction With Traditional CE Menus Because the personal broadcaster digitizes encodes and streams the analog output of the A V source devices being used the client presents the user with the full interface of the A V source device The default pa
53. er frame distance constant preserving temporal video quality This results in better motion estimation and accept able spatial video quality levels Frame rate regulation is performed in one embodiment according to the following algorithm The target frame rate defined by the encoder is F sustainability measure SM determines whether the frame rate can be sustained over the time interval T SM can be defined as the ratio of the bit pool available to the estimated bits required for the next T seconds The bit estimate is a product of frames per T seconds with the recursive average of the bits per frame The SM may be calculated based on the estimate and the target bit rate The motion estimate is based on the recursive average ofthe encoded bits per frame In one embodiment the frame rates at subsequent levels are 2 3 5 If F denotes the assigned frame rate for level i based on SM a decision is made every T whether the encoder should continue in the same level or move to a level higher or a level lower If SM is less than a indicating that that the current frame rate F cannot be sustained for the time interval T the encoder moves to level 1 1 If the encoder is already at the lowest frame rate it stays there This operation is again performed after every T The level thus goes down when bit rate is not sufficient enough to cater high motion content bit rate need While in lev
54. eters the time taken to buffer on the media broadcaster the network transmission time taken for the stream to move from the media broadcaster to the media player and the time taken to buffer on the media player The network transmission time taken for the stream to move from the media broadcaster to the media player cannot be directly controlled however both the media broadcaster and the media player can minimize delays caused by their buffers The system enters control mode when the user starts inter acting with the media broadcaster by way ofthe media player In one embodiment user interaction is defined as when the user requests an IR command or tuner command to control the operation of the A V source device through the media player user interface In one embodiment the system returns to normal mode from control mode after a predetermined amount of time has passed since the last action that could cause the system to start control mode In another embodi ment the system returns to normal mode immediately after it performs the operations associated with going into control mode as described below On entering control mode the media player and the media broadcaster change their behavior relative to that in normal mode in a number of ways As mentioned in the normal mode the media player per forms normal A V streaming wherein the media player reads the media stream from the network buffer parses the stream and fills its audio vid
55. filed Aug 10 2009 Thiyagarajan Venkatesan Systems and Methods for Virtual Remote Control of Streamed Media U S Appl No 12 538 664 filed Aug 10 2009 Thiyagarajan Venkatesan et al Localization Systems and Method U S Appl No 12 538 783 filed Aug 10 2009 Shirali Amey et al Methods and Apparatus for Seeking Within a Media Stream Using Scene Detection U S Appl No 12 538 784 filed Aug 10 2009 Thiyagarajan Venkatesan Systems and Methods for Updating Firmware Over a Network U S Appl No 12 538 661 filed Aug 10 2009 Iyer Satish Methods and Apparatus for Fast Seeking Within a Media Stream Buffer U S Appl No 12 538 659 filed Aug 10 2009 European Patent Office International Searching Authority Interna tional Search Report for International Application No PCT US2009 049006 mailed Sep 11 2009 China State Intellectual Property Office First Office Action issued Jan 8 2010 for Application No 200810126554 0 SPTO Final Office Action mailed Nov 6 2009 U S Appl No 9 809 868 filed Mar 15 2001 SPTO Final Office Action mailed Nov 12 2009 U S Appl No 1 620 707 filed Jan 7 2007 SPTO Non Final Office Action mailed Oct 1 2009 U S Appl No 11 778 287 filed Jul 16 2007 USPTO Non Final Office Action mailed Nov 23 2009 U S Appl No 11 683 862 filed Mar 8 2007 USPTO Final Office action mailed Jan 25 2010 U S Appl No 11 734 27
56. g text with news or other information that varies based on the content the user is currently viewing For example if the user is currently watching a San Francisco Giants baseball game the user can be presented with a news ticker that details other baseball scores around the league or perhaps a Web page with statistics and facts about the San Francisco Giants The content can be embedded in the interface of the client application itselfor can be presented through the launching of another application such as a Web browser Similarly the user can be presented with content and context specific advertisements An example in the context of the same base ball game would be an advertisement from the San Francisco Giants ticket office which hopes to attract viewers to pur chase tickets to a future game broadcasting system can determine the content cur rently being viewed by the user in a number of ways In one embodiment the personal broadcaster or client sends infor mation including the current time the channel lineup being used e g location and cable TV provider and the current channel being tuned to to the central server The central server then takes the information received and examines the electronic programming guide applicable to the user s ser vice From information it is determined which show the user is currently viewing Alternatively the client can perform the programming guide lookup itself and transmit this info
57. h originates from the user s own home The media stream transmitted from the personal media broadcaster to the media player client networks can take advantage of a combination of any number of networking protocols including HTTP over TCP IP as well as UDP RTSP RTP RSVP and the like Because embodiments of the invention can accept digitize encode and stream any analog A V source there are a vast number of uses for the personal broadcaster from a security camera system to a method for extending a user s satellite radio to the user s cell phone System Architecture FIG 1 is a block diagram ofthe media broadcasting system in accordance with one embodiment of the invention As illustrated a personal media broadcaster 100 is configured to receive an input video signal from a wide variety of A V source devices 120 For example any component or device having analog A V outputs can be connected to the personal broadcaster 100 Upon receiving the video and or audio feed from a connected A V source device 120 the personal broad caster digitizes encodes and streams the digitally com pressed media to the home Internet gateway 110 The gateway 110 may comprise one or more separate devices including a router a switch or hub and or an analog DSL cable or other type of broadband modem or the gateway 110 may comprise a single device that encompasses one or more of these func tions The gateway 110 may be coupled to a local area network
58. he home Internet gate way To make this easy and integrated into the set up process for the personal broadcaster the configuration screens for the home Internet gateway can be embedded in the window that houses the set up application This provides the user with a greater sense of seamlessness On Device EPG One variation of the present invention features an elec tronic programming guide EPG that resides locally on a client The EPG 1s configured at initial configuration of the personal broadcaster The user is asked for a zip code and the service and package to which the user subscribes Based on this information the client application downloads an EPG that covers the next several days From time to time the EPG is updated via download from the central server or another server from a third party provider The EPG can be stored locally on the client Beneficially the EPG can be made interactive Among the many features enabled by an on device EPG a user could search and sort programming content by a number of vari ables and a single click a channel can automatically tune the A V source device and media player to the desired channel Community or Buddy List One embodiment of the media player incorporates a buddy list Using the buddy list a user can connect to personal broadcasters that reside at different locations For example if Charlie declares Amy as a buddy Charlie s personal broadcaster appears o
59. he user may also have the ability to set the encoder settings manually These manually adjustable settings include frame rate bit rate resolution quality and time between key frames for video bit rate and sampling rate for audio as well as client side settings that can impact performance such as buffer size and smoothing Adjustment of Encoder Settings Based on Programming Type While the data throughput between the personal broad caster and the client may be one important determinant ofthe proper encoding settings the type of content being viewed may also be an important criterion For example fast motion video from a sports program requires a higher frame rate than a talk show which features much less motion Likewise video with little movement can require comparatively lower bit rates or resolution than fast motion video to achieve an acceptable quality This is because video with slower move US 7 877 776 B2 13 ment tends to be encoded much more efficiently Accordingly the encoder settings may be selected based on the type of programming being encoded There are various methods for determining the type of content being viewed and hence the proper range of encoder settings First there can be settings that apply generally to all content on a given channel By identifying the programming channel or network to which the personal broadcaster is tuned e g HBO or NBC the client can request suitable encoder settings
60. ia IR or serial commands the user connects an emit ter cable from the appropriate output on the personal broad caster 100 to a serial input or the IR receiver on the A V source device 120 This provides the communication means that allows the client to control the A V source device 120 e g to change the channels In FIG 2 the personal broadcaster also includes an Ether net port 290 that provides a communication interface to the home Internet gateway 110 In some embodiments of the invention the personal broadcaster 100 also supports wire less networking e g through a built in 802 11 capability and the broadcaster 100 may even be built as an access point AP or router for a wireless network according to a wireless networking standard such as 802 11 The personal broad caster 100 can also include a power connector 230 a hard reset button 240 and a number of indicator lights e g on a front panel that show the state of the personal broadcaster US 7 877 776 B2 7 100 Many other inputs and outputs are also possible For example the personal broadcaster 100 can have video and analog outputs for a local display and sound FIG 2 also shows the connections possible between an A V source device 130 and one embodiment of the personal broadcaster 100 As illustrated an S video cable and left and right composite audio cables connect the personal broad caster 100 and an example A V source device 120 Because the A V source devi
61. if Malvar Apostolopoulos et al Rakib et al Nason et al Rising III Tsai et al Wathen Uchida et al Logan et al Allen Allen Ikedo et al Tobias et al Wu et al Levesque et al Tripp Brooks et al Logan et al Logan et al Elliott US 7 877 776 B2 Page3 7 124 366 B2 10 2006 Foreman et al 2005 0055595 Al 3 2005 Frazer et al 7 151 575 BL 12 2006 Landry et al 2005 0060759 Al 3 2005 Rowe etal 7 155 734 Bl 12 2006 Shimomura et al 2005 0097542 Al 5 2005 Lee 7 155 735 B1 12 2006 Ngo et al 2005 0114852 Al 5 2005 Chen et al 7 184 433 Bl 2 2007 Oz 2005 0132351 Al 6 2005 Randall et al 7 224 323 B2 5 2007 Uchida et al 2005 0138560 Al 6 2005 Lee etal 7 239 800 B2 7 2007 Bilbrey 2005 0198584 Al 9 2005 Matthews et al 7 344 084 B2 3 2008 DaCosta 2005 0204046 Al 9 2005 Watanabe 7 430 360 B2 9 2008 Abecassis 2005 0216851 Al 9 2005 Hull et al 7 430 686 Bl 9 2008 Wang et al 2005 0227621 Al 10 2005 Katoh 7 464 396 B2 12 2008 Hejna Jr 2005 0229118 Al 10 2005 Chiu etal 7 502 733 B2 3 2009 Andrsen et al 2005 0246369 Al 11 2005 Oreizy et al 7 505 480 BL 3 2009 Zhang et al 2005 0251833 Al 11 2005 Schedivy 7 565 681 B2 7 2009 Ngo et al 2005 0262539 Al 11 2005 Barton et al 7 661 121 B2 2 2010 Smith et al 2005 0283791 Al 12 2005 McCarthy et al 2001 0021998 Al 9 2001 Margulis 2005 0288999 Al 12 2005 Lerner etal 2002 0004839 Al 1 2002 Wine et al 2006 0011371 A1 1 2006 Fahey 2002 0010925 Al 1 2002 Kikinis 2006 0015925 Al 1 2
62. ises adjusting the encoder setting as the available band width of the network changes 40 The method of claim 33 wherein the uncompressed signal is received in an analog format and wherein the method comprises digitizing the analog signal 41 The method of claim 33 wherein the uncompressed signal is received in a digital format 42 The method of claim 33 wherein the client is a web browser 43 The method of claim 42 wherein the uncompressed signal is a live stream from a source device and wherein the method comprises receiving an instruction from the web browser and controlling the source device in response to the instruction received from the web browser to adjust the live stream 44 A method comprising receiving an uncompressed signal from an audio visual source device encoding the uncompressed signal to construct a digitally compressed media stream transmitting the digitally compressed media stream to a web browser via an IP network and adjusting the encoding of the uncompressed signal in responseto variations in an available bandwidth ofthe IP network 45 The method of claim 44 wherein the encoding is per formed in real time as the uncompressed signal is received 46 The method of claim 44 wherein the adjusting com prises dynamically adjusting the encoding based on the varia tions in the available bandwidth of the IP network as the digitally compressed media stream is transmitted on the IP network 47 The
63. l audio and video before compression The pre processing can be performed based on the input type compression properties and target bit rate After pre process ing the processor compresses the audio and video to a desired bit rate using any suitable compression technique such as 9 4 H 263 and H 264 The compressed audio and video are multiplexed into a single media stream together along with other user data such as close caption tele text parental control and Macrovision In one embodiment the 20 25 30 35 40 45 50 55 60 65 8 processor 320 is capable of statically and or dynamically adjusting the compression bit rate frame rate resolution and filtering depending on a user request the input content avail able network bandwidth or on any other data known to the processor 320 The compressed media stream is then con verted into network packets for transmission over the local network 140 or a remote network 160 via the network inter face 325 The network interface 325 may be a wireless or a wired interface or may have provisions for both types As mentioned above the personal broadcaster 100 may also receive and process commands received from a client over the network interface 325 Some examples of these commands include selecting a particular channel automatic scanning of channels switching between RF input and base band input changing compression properties compression type bit
64. lable data throughput present between the personal broadcaster on one end and the local clients and remote clients on another can vary considerably based on network topology There is also likely to be considerable throughput variation in a given con nection due to competing traffic and general network con gestion In one embodiment of the invention a method for optimizing the audio e g bit rate and sampling rate and video e g bit rate resolution and frame rate compression based on available network bandwidth and capabilities of client device is implemented Because various embodiments of the present invention encompass both the personal broadcaster and the client devices and these elements can operate in a 1 1 relationship 1 each broadcaster may accept only one client connection at a time the two components are able to act in concert to optimize the experience for the user In one embodiment the optimization process includes an initial optimization stage and an ongoing optimization stage In the initial optimization process the client and personal broadcaster communicate to mutually establish the capabili ties of the client device as well as the throughput of the connection between the personal broadcaster and the client device The client device first requests the personal broad 20 25 30 35 40 45 50 55 60 65 12 caster to send a set number of bits to the client this can happen automati
65. media stream When the client enters the control mode the buffer is flushed and the received media stream is played In this way the content in the buffer need not be displayed on the client device and the user can then almost immediately receive visual feedback confirming the user s command to control content in the media stream BRIEF DESCRIPTION OF THE DRAWINGS FIG 1 is a block diagram of a media broadcasting system in accordance with an embodiment of the invention FIG 2 is a block diagram of a set of input and output connections for a personal media broadcaster in accordance with an embodiment of the invention FIG 3 is a block diagram of a personal media broadcaster in accordance with an embodiment of the invention DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Overview Embodiments ofthe invention allow for distribution ofA V content from a variety of A V source devices over a computer network to a client device As used herein A V content may include any type of media content including audio or video content or both In one embodiment a personal media broad caster takes as an input an audio or video signal digitizes and compresses the content e g in Windows Media Video 4 or 264 and streams the content over an IP network such as IP4 or IP6 to a client device for viewing and or listening to the content The personal broadcaster which may be implemented as an embedded solution may allow the
66. method of claim 44 wherein uncompressed signal represents a live stream from the audio visual source device wherein the encoding and transmitting are performed as the uncompressed signal representing the live stream 18 received 48 The method of claim 44 wherein the encoding com prises compressing the uncompressed signal using an encoder setting 49 The method of claim 44 further comprising receiving an instruction from the web browser via the IP network 50 The method of claim 49 further comprising controlling the audio visual source device in response to the instruction received from the web browser 51 The method of claim 44 further comprising converting the digitally compressed media stream to network packets compatible with the IP network 52 method of claim 51 wherein the transmitting com prises transmitting the network packets on the IP network
67. n pre fil tering is employed to simplify the image content before com pression The pre filters often comprise low pass filters which not only smooth blocking artifacts but also undesirably reduce image details Moreover low pass filters are not adap US 7 877 776 B2 21 tive to video content which makes them ill suited for percep tual video encoding To avoid the limitations of previous pre filters one embodiment of media player performs a con tent specific edge estimation algorithm on the media stream received from the media broadcaster In accordance with this embodiment an edge estimate gives the location ofthe details in the image that should be preserved while a motion estimate gives a classification of whether the content is in high medium or low motion Inone embodiment the media player applies a pre filter on the received stream The pre filter comprises a low pass filter having pass band characteristics that are changed based on an edge estimate a motion level estimate and an encoding bit rate Based on the motion level estimate and the encoding bit rate a filter characteristic is defined for a frame The filter characteristic is then fine tuned at the pixel level based on the edge estimate Using this approach a higher degree of smoothing is used for high motion contents and lower degree for low motion content while leaving the details intact Inone embodiment two sets ofadaptive low pass filters are defined For
68. n Amy s buddy list By choos US 7 877 776 B2 25 ing Charlie s personal broadcaster Amy connects to Char lie s device All of the necessary settings e g IP address port password and any other required settings are automati cally provided to the buddy Last Come Last Served Another embodiment of the invention allows only a single client to be connected to the personal broadcaster at any given time The broadcaster may implement a number of priority schemes one of which is last come last served In this scheme if a client is connected and client B attempts to connect priority is given to the client B One embodiment can provide a message to client informing the user of client A that the client is about to be disconnected The user of client A may be provided the opportunity to override this rule and remain connected This priority system is especially useful in the situation where a person logs in at home then leaves the house without disconnecting and attempts to log in from a remote client Encryption and Security Various known security mechanisms can be used in differ ent embodiments of the present invention Example of secu rity mechanisms that can be used with the personal media broadcasting system described herein include but are not limited to password protection communication over a secure link encrypting the content sent over the remote network Blocking Out the Local Display In another embodiment
69. namic IP addresses via DHCP as opposed to providing static IP addresses there is a need for a system that provides a consistent method for accessing the user s home network from remote networks In the embodiment illustrated in FIG 1 the central server 180 assigns the personal media broad caster 100 a DNS name e g username slingbox com and correlates that DNS name to the user s IP address To account for the dynamic nature ofthe IP address in one embodiment a dynamic DNS client application resides on the personal broadcaster 100 The dynamic DNS client application reports to the central server 180 any change to the IP address leased by the ISP When a remote client 170 needs to communicate with the broadcaster 100 the client 170 first obtains the associated IP address from the central server 180 In this way the DNS address called by a user on a remote client 170 is the current IP address ofthe gateway 110 even when that address changes over time make this process even easier for the user so that the user need not manage a constantly changing IP address or enter a DNS name much of the process for connecting to a personal broadcaster 100 can be abstracted from the user For example in one embodiment the user need only to enter the name of the personal broadcaster or select an icon represent ing the personal broadcaster and then enter the correspond 20 25 30 35 40 45 50 55 60 65 6 ing passwo
70. ndicates which make and model of A V source device the user would like to control e g TiVo Personal Video Recorder Series 2 The configu ration software on the client then identifies the group of IR or serial codes that correspond to the specific A V source in question This can be performed by searching a database that ships with the included software an updated database resid ing on the central server or a database that resides on the personal broadcaster US 7 877 776 B2 11 In one embodiment of the invention the IR codes is then stored on the client device When a user wishes to invoke a code the user selects the given command on the control panel or virtual remote control in the client application The client application then sends the corresponding IR or serial com mand over the IP network Once the IR or serial command reaches the personal broadcaster the personal broadcaster processes the code and sends it out the IR or serial output triggering the requested action in the A V source device The connection between personal broadcaster and the A V source device can be an IR emitter in the case that the A V source device was to be controlled via IR or an RS 232 port in the case that the A V source device was to be controlled by serial commands In another embodiment the IR and serial codes are stored on the personal broadcaster rather than the clients In this case a client device that requests a given command sends a n
71. nual Online Aug 27 2007 XP0025 15046 retrieved from the Internet URL http www mythtv org wiki title User_Manual Introduction amp oldid 25549 gt International Searching Authority Written Opinion and International Search Report for International Application No PCT US2008 077733 mailed Mar 18 2009 International Searching Authority Written Opinion and International Search Report for International Application No PCT US2008 087005 mailed Mar 20 2009 Watanabe Y et al Multimedia Database System for TV Newscasts and Newspapers Lecture Notes in Computer Science Springer Verlag Berlin Germany vol 1554 Nov 1 1998 pp 208 220 XP002402824 ISSN 0302 9743 Yasuhiko Watanabe et al Aligning Articles in TV Newscasts and Newspapers Proceedings of the International Conference on Computationallinguistics XX XX Jan 1 1998 pp 1381 1387 002402825 Sodergard C et al Integrated Multimedia Publishing Combining TV and Newspaper Content on Personal Channels Computer Net works Elsevier Science Publishers B V Amsterdam Netherlands vol 31 No 11 16 May 17 1999 pp 1111 1128 XP004304543 ISSN 1389 1286 Ariki Y et al Automatic Classification of TV News Articles Based on Telop Character Recognition Multimedia Computing and Sys tems 1999 IEEE International Conference on Florence Italy Jun 7 11 1999 Los Alamitos CA USA IEEE Comput Soc US vol 2 Jun 7 1999 pp 14
72. o receive a user command to control content in the media stream from the client 20 The device of claim 18 wherein the input interface includes an interface for receiving the analog signal from a plurality of types of source devices 21 The device of claim 18 wherein the device is config ured to dynamically adjust the encoder setting based on the variations in the available bandwidth of the network as the media stream is transmitted over the network 22 The device of claim 18 wherein the processor is con figured to adjust the encoder setting in real time 23 The device of claim 18 wherein the encoder setting comprises at least one of the following a bit rate a resolution or a frame rate 24 The device of claim 18 wherein the analog signal is a live stream from the one or more source devices 25 The device of claim 18 wherein the analog audio visual signal is a live stream from a personal video recorder PVR 26 The device of claim 18 wherein the analog audio visual signal is a live stream from a cable set top box 27 The device of claim 18 wherein the analog audio visual signal is a live stream from a video camera 28 The device of claim 18 wherein the network comprises a wide area network WAN 29 The device of claim 18 wherein the network comprises the Internet 30 The device of claim 18 wherein the uncompressed signalis received in an analog format and wherein the device further comprises an analog to
73. odiment is a very desirable characteristic for systems in which the users interacts with the media broadcaster through the media player The addition of a control mode achieves this low latency while the return to normal mode once user interaction has stopped achieves the smooth streaming desired for normal viewing Moreover in combi nation with the fast start streaming described above the con trol mode function provides a low latency control interaction experience for the user with a seamless transition from con trol mode to normal viewing 20 30 35 45 50 55 60 65 18 Trick Play of Received IP Stream In one embodiment a user viewing a live stream being encoded in real time by the personal broadcaster and dis played on the media player can replay the last several seconds or minutes of content just viewed or it can pause the live stream for resumption at some point in the near future The personal broadcaster receives and processes the analog input signal converts it into digital format encodes the video in a compression algorithm such as MPEG 4 H 264 Windows Media Video Series 9 or another appropriate format and then streams the encoded via stream over TCP IP oran alternative protocol such as RTSP to the media player To enable this functionality the media player caches the last five minutes or some other fixed period oftime as chosen by user or limited by product manufacturer of the a
74. onfiguration streaming content directly from the on network storage device 130 can create security concerns for the user especially if the on network storage device is a PC Because allowing a PC to stream directly upstream to the public Internet involves open ing a port on the firewall NAT that forwards to the PC a user can be concerned that other personal or private information is at risk for being exposed By relying on the personal broad caster to be a gateway for streaming media to the public Internet the PC or on network storage device on which the media is stored need not have a port forwarded to it In such an embodiment the personal broadcaster is the only device for which NAT port configuration is required In addition using the personal broadcaster instead of the on network storage device to transcode and transrate content prevents the CPU on the on network storage device 130 from being unnecessarily taxed This is especially important if the on network storage device 130 is a PC because the increase 20 30 40 50 55 60 65 10 in CPU utilization would detrimentally impact the perfor mance of the PC for accomplishing other tasks Because the personal broadcaster 100 provides the user with access to the same A V source devices 120 available at home it makes sense to provide the user with an interface to the A V source device 120 similar to the one used in the living room setting Most often this is a handh
75. ork to the Internet For example a video stream coming from the personal broadcaster can be granted higher priority than a US 7 877 776 B2 9 print job request over the local network or an email download from aremote POP3 server In either case the personal broad caster preferably incorporates a network switch as part of its architecture As shown in FIG 1 the personal broadcaster 100 may also be able to receive a digital audio or video stream or other digital media from an on network storage device 130 The on network storage device 130 may be a personal computer a networked attached storage device or a dedicated media server For example a user could have a collection of audio and video clips stored on a personal computer or media server that resides on the same home network as the personal broad caster The user could then access the media on a remote client 170 over a remote network 160 by logging into the personal broadcaster 100 When wishing to stream media stored on an on network storage device 130 to a remote location a challenge arises The bit rate of the media clips present on the on network storage device can be higher than the bit rate supported by the upstream link ofthe user s broadband service For example video clip on an on network storage device 130 can have a bit rate of 800 kbps whereas very few broadband connections currently have an uplink speed equal to or greater than that In such cases the bit r
76. otation representing that command e g UP if the user wants to change the channel from 4 to 5 rather than the or serial control code itself When the notation represent ing the command reaches the personal broadcaster the per sonal broadcaster performs a simple lookup and outputs the appropriate IR or serial code Certain commands or selections made on the client device can activate a series of commands also known as a macro For example by clicking on the CNN button which can possibly be represented by a logo of the network the combination of commands that tune the A V source device to CNN would be triggered For example if CNN was channel 202 on a user s DirecTV system a press ofthe CNN button can trigger the following commands in succession 2 0 2 and Enter To make the set up of multiple client devices easier profiles of the AV source devices used can be stored in the personal broadcaster during the configuration of the initial client device This enables easy configuration for subsequent client devices as the personal broadcaster informs the subsequent client device which A V source devices it is able to connect to and which IR or serial codes it uses Adjustment of Encoder Settings Based on Throughput and Device Capability Because the broadcaster enables access of a media stream by a variety of client device types connected to the local area network as well a various remote networks the avai
77. pl storyid 1056825330084 amp date 2003 07 01 amp pagetype printer amp section 1053801318705 amp path articles 2003 06 30 1056825330084 html gt Archive of TV Brick Home Server www tvbrick com online Archived by http archive org on Jun 3 2004 Retrieved on Apr 12 2006 Retrieved from the Internet lt URL http web archive org web 20041 107111024 www tvbrick com en affiliate tvbs tvbrick tvbrick document18 print gt PCT International Search Report and Written Opinion 17505 20105 Feb 15 2007 6 pages Krikorian Jason U S Appl No 11 734 277 filed Apr 12 2007 Tarra Raghuveer et al U S Appl No 60 975 239 filed Sep 26 2007 Rao Padmanabha R U S Appl No 12 166 039 filed Jul 1 2008 Williams George Edward U S Appl No 12 167 041 filed Jul 2 2008 Faucon B Brick Opens up Copyright Can of Worms Finan cial Review Jul 1 2003 online Retrieved on Apr 12 2006 Retrieved from the Internet URL http afr com cgi bin newtextver sions pl storyid 1056825330084 amp 3ate 2003 07 01 amp pagetype printer amp section 1053801318705 amp path tarticles 2003 06 30 0156825330084 html Archive of TV Brick Home Server www tvbrick com online Archived by http archive org on Jun 3 2004 Retrieved on Apr 12 2006 retrieved from the Internet URL http web archive org web 20041107111024 www tvbrick com en affiliate tvbs tvbrick docu ment 18 print gt P
78. put to a client device over a network 2 Background of the Invention While people spend a great deal of time watching televi sion programming and other forms of audio visual A V content they are also spending an increasing amount of time interfacing with computing devices such as personal comput ers personal digital assistants mobile phones dedicated multimedia devices and other devices that like the tradi tional television include a display These types of computing devices allow people to be increasingly mobile but this mobility reduces the time people spend at home in front of their televisions It would therefore be beneficial to enable people to enjoy their television programming and other forms of A V content they now receive at home on these computing devices as well regardless of location and without depen dence on physical connections This ability would enable several desirable applications For example a user might want to access and control televi sion and other regularly consumed A V content from a per sonal computer desktop as well as notebook computers or other computing devices around the home via the user s local network in the home Since cable satellite and other sources of television content typically enter the house at a few discrete locations allowing access to the content over a home network gives the user more freedom to enjoy the content throughout the home Another possible application would be to
79. radigm for navigating menus ofthe A V source device that are rendered by the client is exactly the same as it would be for a user viewing the A V source device in a more traditional fashion That is the mode of navigation typically involves Up Down Right Left and Select as its key components However alter native methods for interacting with menus and other lists are possible For example instead of pressing the Down com mand four times to highlight an item four spaces down from what is currently highlighted an embodiment of the media player allows a user simply to point and click directly on the desired menu item This speeds up the interaction and takes advantage of the rich human interface tools e g keyboard and mouse that are available in many computing devices but not generally used with televisions Hot Spot Finder In one embodiment of the invention the media player includes a directory of hotspots wireless broadband net works available in public locations stored on the client device and accessible when the client is not connected to a network Such a directory allows the user more easily to find a location where the user can gain access to the personal media broadcaster Testing and Configuring a User s NAT For users that have a home Internet gateway that includes a Network Address Translation NAT some configuration can be required to allow a user to access the personal broadcaster from
80. rd before being automatically directed to their personal broadcaster 100 This can be accomplished by tying aunique device name to the DNS name assigned to the user s dynamic IP address The translation between the device user name and the DNS name can take place within the remote client 170 itself or it can be accomplished through a directory maintained on the central server 180 Connections from a local client 140 or a remote client 170 can be accomplished either by using a client application designed specifically for the purpose of accessing the per sonal broadcaster stream or via a traditional Web browser option of using a Web browser provides for wide range of client compatibility without the need for configuration while the client application provides for an optimized experience The client application or the Web interface may prompt the user for a password before allowing communication with the broadcaster 100 as a security measure As an additional mea sure of security the media stream can be encrypted In one embodiment of the invention there is a limit of one connected client applies equally to remote clients and local clients per device That is only one client at a given point in time can be connected to and streaming from the personal broadcaster Other variations ofthis embodiment can provide for multiple simultaneous sessions Still other variations can allow for multiple simultaneous sessions from local clients but
81. ried For high motion content for example a relatively heavy smoothing filter can be applied as compared to low motion content This pre filtering method not only retains the image details but also is content specific That is the filters are adapted to the motion type high medium or low This pre filtering method provides non blocky constant quality encoded video 0 5 25 35 40 45 50 55 65 22 for all motion types Unlike conventional pre filters this reduces the frame drops without compromising quality Frame Rate Regulation and Quality Control For Encoder Most video software encoders at low bit rates dynamically drop video frames to meet some specific target bit rate often during high motion This dropping of frames can lead to jerky video and fluctuating quality levels As potentially large sec tions of frames are dropped the motion estimation process becomes ineffective To avoid this problem one embodiment of the media broadcaster regulates frame rate using a multi level approach designed to enhance the viewing experience of the user by sustaining both frame rate and quality at each level In one embodiment the encoder in the media broadcaster estimates a sustainable frame rate based on a Sustainability measure SM Four levels of frame rate regulation are defined Based on SM the frame rate is selected and an appropriate quality level is defined Each stage of the algorithm keeps the int
82. rma tion to the central server After determining what show the user is currently viewing overlaid information and or advertisements can be transmit ted from the central server or other server on the public Internetto the client media player Alternatively applications such as a Web browser can be launched simply sending the user to a specified URL The client can also contain the information necessary to queue any relevant information or advertisement for display to the user For example the client can have stored in its memory certain advertising banners or URLs of relevant Web sites that are queued depending on what type of content the user views This method alleviates much of the need for transmission of content and or adver tisements from a central server or other remote server Embodiments that include personal video recorder func tionality using local storage on the personal broadcaster or using storage of the on network storage device can provide for context specific ads and information even when playing a previously recorded show When recorded the content is tagged with information indicating the programming being captured On play back this information can be used to prompt content specific information and ads Content Specific Edge Preserving Pre Filter Low bit rate coding involving block based video coders produce strong blocking artifacts To reduce the severity of these artifacts 1n one embodiment of the inventio
83. rver and begins to capture the incoming stream onto the local storage medium e g hard disk drive or flash memory This can happen in the background so a user can be on phone call or listening to music depending on other capa bilities of the device while the remote client is recording If the recording is interrupted because a network connec tion is lost the live streaming content can be temporarily stored on the local built in or removable storage that exists in one variation of the personal broadcaster Once a connec tion between the personal broadcaster and the remote client is reestablished the content resumes the stream from the per sonal broadcaster to the remote client taking into consider ation the point at which the previously interrupted transfer left off In another variation an on network storage device such as a PC can be used to store content temporarily after a connection between the personal broadcaster and the remote client has been interrupted In another embodiment an entire show or media program 15 encoded and stored on the personal broadcaster or in a storage device and then transferred to the mobile device in the background Beneficially the trans fer can be performed at a more efficient time for data trans fer e g the file downloaded to a cellular phone overnight during non peak hours so that the content is available to the user for offline viewing the following day In the case where the original
84. s Based on the bit rate it calculates permissible buffer usage If usage goes beyond the current limit then all data is removed and an I frame is forced This also means encoder output rate is more than network transmission rate therefore encoder output is reduced to half of observed transmission rate Due to the resulting discontinuity in the media stream an I frame is also forced Too many user control commands in quick succession will force many I frames which will adversely affect the encoder Therefore all buffer flushes and forcing of I frames are pref erably timed and spaced out by at least one second When returning to normal mode from control mode the source filter ofthe media player pauses the playback buffers e g up to five or more seconds content in the A V buffers and then sends the data downstream for rendering To operate effectively in a low latency low buffer environ ment a command can be immediately sent to the personal broadcaster to reduce the bit rate when in control mode Without any additional adjustments this would result in a reduction in image quality however because the menu screens being navigated typically feature very little motion the frame rate ofthe video can be significantly reduced With less frames to process the personal broadcaster can output a stream with image quality good enough to effectively read the on screen text The low latency achieved by the dual mode operation of this emb
85. set top box from any desktop PC notebook PC PDA or other network devices For example this application allows a user to convert a wireless notebook PC situated in the kitchen into a wireless LCD TV As another example of in home use young parents can pair a personal broadcaster with a simple analog camera and place them in the baby s room to provide a real time video and audio stream oftheir child displayed on a device in another location in the home Another usage scenario category relates to users who are away from home but have access to a broadband connection This access can be in a variety of locations such as at work at school at a friend s house in a cafe at the airport or in a plane or in a hotel From these locations users can connect to their personal broadcaster over the public Internet and enjoy the same live television experience that they have at home The present invention allows users that are away from home to access the full breadth of programming they are accus tomed to and even view content that they previously captured on their personal video recorder Yet another usage scenario category relates to individuals with mobile phones communicators or other wide area net work devices On emerging networks the bit rate is now high enough for users to enjoy streaming video on their devices This opens up new possibilities for the personal broadcaster to deliver a wealth of programming to the mobile user all of whic
86. source of the media being streamed is an on network storage device using the personal broadcaster as a transcoder transrater transfers between the personal broadcaster and the remote client can be accom plished over time That is if a remote client loses network connectivity the download pick ups and continues later at the proper spot in the content Another embodiment enabling a PVR on a remote device a variation of the embodiment described above allows the personal broadcaster to work with an on network storage device such as a PC ora NAS or SAN device to provide for the user a PVR that can be accessed remotely The personal broadcaster takes in the analog content digitizes and encodes the content and then streams the resulting media stream to the on network storage device for storage When a user wants to 20 25 30 35 40 45 50 55 60 65 20 access the content from a remote client the personal broad caster acts as an arbiter between the remote client which wants to view the content and the on network storage device personal broadcaster receives the stream from the on network storage device and transcodes or transrates the con tent if necessary before repackaging and streaming it to the remote client Pairing Programming Being Viewed With Context Specific Content Advertisements One embodiment of the invention provides the ability for the client to present the user with Web pages scrollin
87. t al 2003 0093260 Al 5 2003 Dagtas et al 2007 0234213 Al 10 2007 Krikorian et al 2003 0093790 Al 5 2003 Logan et al 2007 0286596 A1 12 2007 Lonn 2003 0095791 Al 5 2003 Barton et al 2007 0300258 A1 12 2007 O Connor et al 2003 0115167 Al 6 2003 Sharif et al 2008 0019276 A1 1 2008 Takatsuji et al 2003 0159143 Al 8 2003 Chan 2008 0036917 A1 2 2008 Pascarella et al 2003 0187657 Al 10 2003 Erhart et al 2008 0037573 Al 2 2008 Cohen 2003 0192054 A1 10 2003 Birks et al 2008 0059533 Al 3 2008 Krikorian 2003 0208612 A1 11 2003 Harris et al 2008 0134267 A1 6 2008 Moghe et al 2003 0231621 Al 12 2003 Gubbi et al 2008 0195744 Al 8 2008 Bowra et al 2004 0003406 Al 1 2004 Billmaier 2008 0199150 A1 8 2008 Candelore 2004 0052216 Al 3 2004 Roh 2008 0294759 Al 11 2008 Biswas et al 2004 0068334 Al 4 2004 Tsai et al 2008 0307456 A1 12 2008 Beetcher et al 2004 0083301 Al 4 2004 Murase et al 2008 0307462 Al 12 2008 Beetcher et al 2004 0100486 Al 5 2004 Flamini et al 2008 0307463 Al 12 2008 Beetcher et al 2004 0103340 Al 5 2004 Sundareson et al 2009 0074380 Al 3 2009 Boston et al 2004 0139047 Al 7 2004 Rechsteiner et al 2009 0199248 1 8 2009 Ngo etal 725 90 2004 0162845 Al 8 2004 Kim etal 2010 0100915 Al 4 2010 Krikorian et al 2004 0162903 Al 8 2004 Oh 2004 0172410 1 9 2004 Shimojima et al FOREIGN PATENT DOCUMENTS 2004 0205830 Al 10 2004 Kaneko 2004 0212640 Al 10 2004 Mann et al DE 4407319 Al 9 1994 2004 0216173 Al 10 200
88. t in the intermediate buffer is observed for a period of time e g one minute If the amount of free space in the buffer remains above the 9096 watermark during the last observation period the encoder s output bit rate may be increased Although any increase can be imple mented based on the application in one embodiment the increment is about 10 of the bit rate then being used Because network bandwidth varies over time sudden drops in available bandwidth may take place frequently In such cases the TCP stack transmit rate will go down and the occupancy of the intermediate buffer will increase If this occurs for a long enough period the free space in the buffer will decrease so that the 9096 watermark may be breached In response the broadcaster reduces the encoder s bit rate by a small percentage for example about 1596 of the bit rate then being used If this network problem is a temporary one the TCP stack will again be able to send the backlog of data in the buffer so that the amount of free space in the buffer will rises again above the 9096 mark The encoder s bit rate can then be increased On the other hand if the network problem persists the amount of free space will continue to decline Over a period of time the buffer will fill and the other watermarks will be breached As each watermark is breached the encoder s bit rate is further reduced In one embodiment these subsequent reductions may be larger e g 33 50 50
89. take five seconds assuming no network com munication issues While this delay may betolerated in some streaming media applications it becomes unbearable where the media streams are changing such as in a personal media broadcaster in which the viewer is changing channels In such a case the user would have to wait for the buffer to refill each time the channel and thus the media stream is changed To avoid this disadvantage in one embodiment of the invention the media player performs a fast start when a new media stream is selected With the fast start the media player client application plays the video immediately as it is received from the media broadcaster while still filling its buffer In this way the user does not have to wait for the media player s A V buffers to fill up e g upon channels changes and the media player can still build its buffers to provide smooth playback The media player is able to fill its buffers even though it plays the media stream immediately because the media player plays the A V stream slightly slower than normal play speed By playing the stream slightly slower than normal playback speed the part ofthe received media stream that runs ahead of the stream played back is added to the buffer resulting in accumulation of the A V buffer albeit more slowly Because the user need not wait for content to be buffered and can immediately start watching the content the experience is much more similar to that wi
90. th a normal television Inone example the media player begins in fast start mode playing back a received media stream immediately but at slightly slower speed than normal play speed e g 0 85x speed Since the broadcaster is streaming at normal speed the media player slowly accumulates media stream data in its A V buffer For example if the playback speed is 8596 nor mal the data accumulates in the A V buffer at a rate of 1596 or 1 5 second for every 10 seconds of received streaming data Once the media player s buffer is full or otherwise reaches an acceptable level the media player begins to play the media stream at a normal rate and the A V buffer stops accumulating data When the play speed of stream is changed slightly the change in the video stream is generally not noticeable How ever the change in audio becomes immediately perceptible To take care of this the media player may use time stretching on the audio stream while maintaining the pitch in the audio stream One software tool that can be used to time stretch the audio stream is Sound Touch an open source audio process ing library one embodiment the playback speed is increased gradually from the low threshold e g 0 85x to the real time playback speed e g 1 0x The rate at which the playback speed is increased may be a function of the buffer level so the user does not perceive any drastic change in the playback speed The timestamps of the audio and
91. udio and video content to a local storage medium while receiving a video stream The media player may store the data on a local storage medium such as a hard drive in the case of a PC or some removable media including but not limited to compact flash smart media a memory stick or a micro drive When the user wishes to pause or replay the content the user instructs the media player to do so by selecting an appro priate labeled button in the media player user interface When the user instructs the media player to rewind the media player accesses the content cached in its storage allowing the user to scan through it and play as desired Once the user has rewound content the user can then fast forward the view ing of the stream at faster than real time speed until the current most point in the video stream is being displayed When the user instructs the media player to pause the media player pauses the video stream being displayed but continues to receive the incoming stream which it caches to its local storage medium The media player continues to cache the stream as it is received until the point when the maximum number of minutes or size of data allowed to be cached when in pause mode is reached If the maximum number of minutes or size of data is reached the media player resumes playback Otherwise the media player resumes playback when the user instructs the media player to do so Capturing Editing and Sending Vi
92. user to issue control commands such as channel up via an IR or serial command back over the network to be executed by the original source device Using various embodiments of the present invention a user can connect to the personal broadcaster a cable or satellite set top box a personal video recorder a DVD player a DVD jukebox a music server a satellite radio receiver a camcorder a digital video recorder DVR or any other A V source component This way the user can view and control the live output of these sources from any networked device Various embodi ments of the present invention may include different compo nents including a personal broadcaster and media player The media player may be a software application that runs on a client device which is configured to receive the media stream created by the personal broadcaster Different client software applications can exist for different classes of client devices such as personal computers and cell phones The client may be able to connect to the personal broadcaster through a Web browser interface 20 25 30 35 40 45 50 55 60 65 4 system described herein may be applied in a number of applications or usage scenarios For example there are a variety of uses in the home over a wired or wireless home network for the system In the home users can view and control their A V source components such as a personal video recorder PVR ora cable
93. using the media player client In one embodiment the personal media broadcaster is a dedicated appliance not a general purpose computer In this way a general purpose computer need not be powered on and con nected to the A V source devices for the user to receive media content remotely In another embodiment the media player client can run on any general or multi purpose device such as a personal computer or cellular phone beneficially avoiding the need for the user to carry special equipment to use the broadcasting system The computer network over which the broadcaster and media player client communicate may com prise a wide area network such as the Internet allowing the user to receive media content from the home to anywhere in the world where a connection to the network is available To improve the user s experience the person media broad casting system may employ any of a number oftechniques for buffering transmitting and viewing the content In one embodiment for example the media stream is constructed by encoding the audio visual signal using a bit rate selected at least in part based on an amount of free space remaining in an intermediate output buffer used to temporarily store the media stream before it is transmitted to the media player client While various measures of occupancy of the interme diate output buffer can be used one involves the use of mul tiple watermarks to measure the free space available in the buffer
94. ved on Jan 28 2009 Capable Networks LLC Keyspan Remote Control Controlling Your Computer With a Remote Online Feb 21 2006 XP002512495 retrieved from the Internet lt URL http www slingcommunity com article 1179 1 Keyspan Remote Control Controlling Your Computer With a Remote highlight remote control gt retrieved on Jan 28 2009 Sony Corporation LocationFree Player Pak LocationFree Base Station LocationFree Player Online 2005 002512401 retrieved from the Internet lt URL http www docs sony com re lease LFPK1 pdf gt retrieved on Jan 28 2009 Sling Media Inc Slingbox User Guide Online 2006 002512553 retrieved from the Internet URL http www slingmedia hk attach en US_ Slingbox_User_Guide_v1 2 pdf gt retrieved on Jan 29 2009 Sony Corporation LocationFree TV Online 2004 SP0025 12410 retrieved from the Internet lt URL http www docs sony com releaselLFX 1_ XSrevision pdf gt retrieved Jan 28 2009 note document uploaded in two parts as file exceeds the 25MB size limit European Patent Office International Searching Authority Interna tional Search Report mailed Mar 30 2010 International Applica tion PCT US2009 068468 filed Dec 27 2009 USPTO Final Office Action mailed Mar 12 2010 U S Appl No 11 620 711 filed Jan 7 2007 USPTO Final Office Action mailed Mar 3 2010 U S Appl No 11 111 265 filed Apr 21 2005 USP

Personal media broadcasting system

Contents

Download Pdf Manuals

Related Search

Related Contents