With the advent of technology characterized by heterogeneity and the increasing offers of services and applications, a user's Quality of Experience (QoE) has become a crucial factor for the success or failure of new applications and services. Content adaptation is the process of selecting, generating, or modifying content to suit the user's preferences, consumption style, computing and communications environment, and usage context. This can be applied within media types, such as reducing image size or resolution, and across media types, like converting video from one codec to another. Content adaptation can be implemented through three general approaches: Client-side approach (user); server-side approach (media source); or, intermediate-proxy approach between the client and the server.10
The client-based approach is performed by the client's device, therefore providers can create one version of content and the same content can always be displayed on different devices. The terminal selects the content, which is matching the most of its capacities and the available bandwidth. Examples are Scalable Video Coding (SVC), where the delivered content is coded in different versions in the form of different layers and the client terminal selects the layers that best match the terminal capacity and network status; and HTTP Adaptive Streaming (HAS), where content is coded considering different quality levels and the terminal requests the best matching content based on device capacity and available bandwidth.
In the server-based approach, the content is adapted before being delivered to the client, thus reducing the transmission time and the bandwidth consumption since the delivered content is in an adapted form (only the content users need will be sent). This also reduces the processing time at the client side.11 Examples include Windows Media Services, Adobe Flash Media Server, and QuickTime Streaming Server, where they store multiple variants of the same content in the server and the content best matching to the client content is selected through the client identification sent in the client request to the client profile.1
In the intermediate proxy approach, content adaptation is done by an intermediate element. For example, in Wegner et al.,19 the Media Aware Network Element (MANE) is a proxy between the server and the client that gathers the terminal capacity in the client side and the network characteristics and selects the best adapted content based on the terminal and network status. On the other hand, several research contributions exist on content adaptation methods that adapt the content and its delivery based on information related to network congestion, terminal capacity, measured QoE. However, to our knowledge, there is no method that combines all these factors for both content adaptation and delivery. In our work, we consider a multidimensional approach that adapts the content and its delivery means for enhancing the user's experience. Adapting the content is mainly in terms of changing bit rate, changing codec, while adapting the content delivery means is related to how content will be delivered (unicast, multicast, through which access network). The adaptation process should consider the measured QoE coupled with context information on the user, devices, network, and the content itself to take the adequate adaptation decision. Our notion of QoE is explained in Diallo et al.,4 where the user experience is evaluated as a function of different context concerning the user (preferences, content consumption style, level of interest, location), his environment including the terminal used (for example, capacity or screen size) and network (available bandwidth, delay, jitter, packet loss rate).
The adaptation process should consider the measured QoE coupled with context information on the user, devices, network, and the content itself to take the adequate adaptation decision.
In this article, we examine the existing video streaming techniques and compare the HTTP-based techniques with other techniques. We then focus on the content delivery means, present the content adaptation, and discuss the research contributions for content adaptation and the adaptation of content delivery means.
Today there are several streaming methods to deliver audiovisual content, including: HTTP streaming, Real Time Streaming Protocol (RTSP, developed by IETF), Microsoft Media Services (MMS), and Real Time Messaging Protocol (RTMP) for Adobe Systems. Here, we review these different techniques.
HTTP-based streaming techniques carry out a dynamic content adaptation before and/or during the session following a client-based approach. The adaptation is managed by the player. The different HTTP-based streaming techniques are:
Adaptive streaming based on HTTP (HAS)13,15 allows a multi-rate client-based streaming for multimedia content, where different bitrates and resolutions are available for the same content and the streaming is managed by the client. As explained in Oyman and Singh,16 HAS deployment presents opportunities for services and content providers when compared with traditional adaptive streaming techniques. The server sends a manifest file containing the description of content pieces namely chunks (supported codec, minimum bandwidth, resolution, bitrates, URL). The Media Player then selects the most adequate fragment based on the available bandwidth and device capacity (CPU load, supported codec, bitrate). The following are examples of Adaptive Streaming methods based on HTTP.
The native codec chosen for HLS is MPEG H.264 for video and Advanced Audio Coding (AAC) for audio. HLS is implemented as follows: Encoding video in H.264/TS format at different bitrates. For each encoding profile, a Stream Segmenter cuts the content into pieces called chunks, typically 10 seconds each, and generates a play list file (m3u or m3u8) with a URL for each chunk of this encoding profile. Generating a general index file (manifest) indicating each available encoding profiles (bitrate, codec) and the URL of the corresponding playlist files. Distributing content chunks, playlists, and manifest through an HTTP server (origin or cache). Looking for the most suitable bitrate by the receiving device, which can be a laptop, set-top boxes, or mobile phone and selects the most suitable chunk based on the device capabilities and CPU status.
Once the client receives the manifest, it is able to request some indexed fragments according to its environment (available bandwidth, screen resolution, supported codec). Like HLS, Smooth Streaming leaves the client the choice of the relevant bitrate.
Like other adaptive streaming techniques, this manifest identifies alternative streams, their respective URLs, network bandwidth, and CPU utilization. On this basis, the player chooses the most adapted stream. Two types of file segment types are allowed in DASH: MPEG2 TS (currently used by HLS), and ISO Base media file format (ISO BMFF, currently used by Smooth Streaming and HDS). This simplifies potential migration of existing adaptive streaming platforms to MPEG DASH, as the media segments can often remain the same, and only the index files need to be migrated to the Media Presentation Description (MPD) format.
Adaptive streaming techniques not based on HTTP. RTSP,b RTMP,c and MMSd streaming techniques are not HTTP based and the adaptation (if it is enabled) is managed by the server following a server-centric approach. In RTSP, the content adaptation can take place making use of the Real Time Control Protocol (RTCP) reports sent between the clients to the server. These reports contain information such as packet loss, jitter, RTT (measured/estimated at the client side) that can help the server adapt the content to network conditions. For example, if the connection deteriorates and the transfer rate decreases, the content is streamed with a lower quality so that playback interruptions are avoided, and stream quality is increased if the connection becomes more fluid.
An advantage of this type of streaming technique is the fast start ability, that is, to start content streaming without delay. On the other hand, the drawback mainly lies in the need of a dedicated server with non-negligible license cost (example are Xiph, Icecast, Real Helix Streaming Server, Windows Media Services, Adobe Flash Media Server, QuickTime Streaming Server, and so on) and in difficulties of passing through firewalls. Consequently, this type of streaming is more adapted for users who are connected in managed networks where operators/content providers have a mastery of end-to-end links and application needs in terms of resource consumption. Then they can anticipate the network congestion because most of these streaming techniques use the User Datagram Protocol (UDP) and this transport protocol does not retransmit lost data. Here, we discuss these streaming techniques.
Audiovisual delivery based on RTSP: RTSP was developed by Real Networks, Netscape, and Columbia University. It is available on the products of these companies. The RTSP is a network control protocol designed for use in entertainment and communications systems to control media streaming. The stream consists of cutting the data into packets (fragments) whose size is adapted to the available bandwidth between the client and the server. When the client receives enough packets (buffering), the client application begins to play a package.
Audiovisual delivery based on RTMP: Initially a proprietary protocol developed by Macromedia, RTMP is now Adobe proprietary and free to use. This protocol is used for streaming audio, video, and data over the Internet, between Adobe player (Flash) and a server. Audiovisuals delivery based on MMS, Microsoft's proprietary network streaming protocol used to transfer data in Windows Media Services. MMS can be transported via UDP or TCP.
Table 1 compares the advantages and disadvantages of streaming techniques through the standard player and the origin server.
Here, we describe some means of performing content adaptation for mobile TV, IPTV, and WebTV services.14
Content delivery adaptation in mobile TV. The most common way to adapt content in mobile TV relies on using different encoding profiles according to the Radio Access Technology (or RAT, such as EDGE, UMTS, HSPA, LTE, or Wi-Fi) and to the client's device (form factor, OS, hardware specificities, screen resolution). While device information can be obtained indirectly through the standard User-Agent HTTP header, an enrichment process adds an extra header containing the client's identity (MSISDN) and its RAT information. Generally, most of the operators are constrained to use different video profiles for addressing software, hardware, and legal limitations (profiles for 3GPP mobiles, HLS, Windows media). Each TV channel is coded in a different profile and each profile has its own characteristic (video encoding rate, resolution, video codec, frame rate, audio encoding rate, audio codec, which corresponds to different quality level. For example the following video profiles could be proposed as a function of the device and the access type.
Table 2 illustrates fictive profiles that depend on the terminal screen and the access type.
Content delivery adaptation in WebTV and IPTV. For IPTV Live and Video on Demand (VoD) service, the user generally has two choices: Standard Definition (SD) streams and High-Definition (HD) streams. Nevertheless, the operator may enforce a given content quality based on the available network bandwidth and on the end user's subscription type. WebTV contains TV and/or VoD services offered by a third party available from any Internet access. This method is thus by default available on unmanaged networks, where the best-effort is the unique possible QoS traffic class. Some content providers like French TF1 or France 24 adapt content to the network conditions by using HTTP Adaptive Streaming for their live TV channels. With this technology, users do not care about whatever video quality to choose in order to match their available bandwidth: the video player will automatically request content pieces that are the most adapted to its network status and the device capacity. On the contrary, some OTT content providers for the VoD let the client choose the type of delivery as follows: Streamed mode available in SD (for example, 620kbps stream) for "Instant Viewing;" Progressive Download mode available for both HD (for example, 1500kbps/1GB/WMV file); and in SD (for example, 620kbps/450MB/WMV file), with a possible non-negligible start-up delay for HD that depends on the client's bandwidth.
Limitation of content adaptation means in current deployment. The current content adaptation for mobile TV, IPTV, and Web TV does not consider a sufficiently large set of parameters to fully enable optimal QoS and QoE. The user context is considered in a limited manner through mainly considering the characteristics of the device used and the network used. The user's localization (physical/geographical location, but also location in terms of proximity to the access network entities) and the user's consumption style are not considered. In addition, network context is considered only in terms of bandwidth availability while ignoring the cost of using this bandwidth instead of allocating it to monetized services. Neither is considered the matching degree of the content to the users' preferences. Consequently, context awareness and user centricity need more consideration in content adaptation through studying users' localization including their physical location and in terms of proximity to the access network entities; or considering the users' consumption style for a given type of video service through, for instance, a trade-off between start-up delay and quality content. For example, a fast start is the most important factor for news and live streams, while content quality is the most important one for a movie.
The current content adaptation for mobile TV, IPTV, and Web TV does not consider a sufficiently large set of parameters to fully enable optimal QoS and QoE.
The classification of existing research contributions on content adaptation show three main categories: Content adaptation: Which version of a given content shall we transmit? (codec, bit rate, video resolution). This aspect is related to the encoding of the content information. Content delivery adaptation: How the content is delivered through the network (unicast, multicast, from which server(s), from a cloud, through which access network)? This aspect is related to the service and network aspect of the content transmission. And adaptation of content and delivery: Integrating adaptation of both content and its delivery.
Content adaptation. Several research contributions exist about adapting content based on terminal capacity, network congestion, user profile, and service requirements. For instance, the method in Kim et al.6 provides a QoE-guaranteed service that maximizes the visual expectation of the viewer by considering the size of the LCD panel size on his device. In Mohan and Agarwal,12 users are allowed to define their preferences (user profile) during service subscription, according to some categories based on QoS requirements (streaming, conversational, interactive, background). For example, the streaming traffic class is sensitive to packets losses. The user can also select different types of subscription (Bronze, Silver, Gold, Platinum) for each profile and traffic class. There are maximum and minimum QoS parameters where an adaptation is needed for each type of service and user profile.
The method introduced in Koo et al.8 adjusts the quality level and transmission rate of video streaming on the basis of the wireless channel status (Modulation Coding Scheme, Signal to Interference-Ratio level), location, and client buffer status. The transmission rate is determined as a function of the network context (for example, packet loss, jitter) and some player buffer ratios. In Guo et al.,8 the adaptation of the transmission rate is done on the basis of the pre-buffering time and of the available bandwidth (network status/context), so that QoE is maximized even in case of network congestion.
Khan et al.7 adapts content using two parameters: the "congestion" (C) and the "degradation" (D). The congestion is defined as the fraction of the number of video blocks lost (BL) divided by the total number of video blocks sent (BS) within an interval of time. After predicting the estimated quality of experience (denoted MOSt), the degradation is defined as the difference between the maximum achievable Mean Opinion Score (MOS) and the estimated MOS (MOSt). The Sender Bit Rate (SBR) is computed by on an algorithm using congestion and degradation. W3C proposed by Bakhtiar,1 content adaptation techniques within the Composites Capabilities Preferences Profiles (CC/PP) for Web content and User Agent Profiles (UAProf) for mobile phones. These frameworks can be used to deliver devices contexts (screen size, audio/video capabilities) and users preferences (language, type of content) and allow devices to communicate their capabilities and preferences to servers. The server can then accurately adapt content according to this information.
Adaptation of content delivery. Research contributions on the adaptation of content delivery adaptation focus on two main approaches: the network-centric approach, in which decisions are made at the network side (mainly by network operators) and principally based on the network operator's benefits; and the user-centric approach, where the decision is based on the user's benefit, generally, without considering network load-balancing or other users. It should be noted that the user-centric approach has the main drawback from a load balancing perspective, since users generally consider only their own benefit while making decisions. This could result in bad performance of the overall network and service.2 Ksentini et al.9 considers QoE measurements over different access types. After predicting a MOS with Pseudo Subjective Quality Assessment (PSQA), a vertical handover (change in access network and/or technology) is carried out toward the network offering the best MOS; that is, the best QoE.
A new algorithm is called Smooth Adaptive Soft-Handover Algorithm. Its goal is to improve the user-perceived quality while roaming through a heterogeneous wireless network environment.
A new algorithm is defined in Ciubotaru and Muntean3 called Smooth Adaptive Soft-Handover Algorithm (SASHA). Its goal is to improve the user-perceived quality while roaming through a heterogeneous wireless network environment. The score of each connection is evaluated based on a comprehensive Quality of Multimedia Streaming (QMS) score including the following metrics: QoS, QoE, cost, power efficiency, and user preferences. The idea is to adapt delivery in the network that has the best QMS score. Each metric is weighted with a proper coefficient.
Adaptation of content and delivery means. Pichon et al.17 examine the most suitable content to be delivered to the user and selects the best delivery mean. Two decision entities are considered, namely the service manager responsible for the service delivery, and the mobility manager responsible for the network connectivity. For content adaptation, the service management entity will be notified when a terminal requests the streaming of a new video (contents encoding is done with SVC), and decides which version should be sent according to the user rights, to his preferences, to his terminal capabilities, and to the network congestion. The service management entity then provides its decision to the service execution entity that sends the corresponding signalization. For content delivery adaptation, the mobility manager is notified about the network-related events and service requirements and retrieves network-related information and decides which possible network connection(s) must be used for every service based on information such as cost, network load, and user preferences.
Comparison of content adaptation and content delivery adaptation techniques. Here, we provide a comparative study regarding different issues that should be addressed in content adaptation techniques, mainly considering user context, user satisfaction, network congestion, and required resources. There are a lot of research contributions on content adaptation and its delivery. We have therefore classified them into several categories:
Terminal capacity has a lot of advantages among which we can mention the consideration of user context by using the terminal capacity. The disadvantage of this method is not considering the dynamic variation of user's needs, network resources optimization, and user satisfaction.
Network congestion. The main advantage of this method is the network resources optimization. The lack is no consideration of others context information like user context (his location, his preferences, his profile), user measured quality of experience.
User profile and service requirements. The advantage is the consideration of user context by reviewing his profile. This adaptation technique is easy to implement because the user profile and service needs are known by the server platform. The disadvantage is the no consideration of user context, network congestion in the adaptation process.
Network congestion and terminal capacity considers some context information like terminal capacity and aids resource optimization. The needed resource is a centralized server for gathering network status and terminal feedback. The user location, the measured QoE, is not considered in the adaptation technique.
Network congestion and measure QoE. This type of adaptation considers the user satisfaction (QoE) and optimizes the network use. Some context information is missed in the adaptation technique like user location or terminal capacity. In the literature we notice some limitations in the existing work as follows: The method in Kim et al.6 does not consider the dynamic variation of users' needs and the network resources optimization. The solution presented by Mohan and Agarwal12 could not adequately enhance the user's experience since the media source is not aware of the user context (terminal capacity, user preferences, and localization). The method presented by Koo and Chung8 can be difficult to execute because it is not easy to ask each user to implement his profile when he subscribes to a service. Some important context information is missed in the proposed method by Guo et al.,5 like the user localization and terminal capacity.
A general notice is the lack of contributions considering the adaptation of both the content and their delivery means.
This article surveyed content adaptation techniques considering the application-level adaptation and the adaptation of the delivery methods. Firstly, a review on video streaming techniques was given comparing them through several operational, performance, and user-centric criteria. Secondly, the content delivery means were reviewed and compared considering both operational solutions and research contributions.
We notice several research contributions on content adaptation based on terminal capacity, user preferences, network congestion, and so on; however there is a lack of fully contextual methods making use of context information on the terminals, network, and users. In addition, there are limited contributions considering adaptation of both contents and their delivery means. Consequently, we solicit the need to consider the user context (environment, location), terminal context, network context, content context for better content adaptation, and optimized content delivery aiming to improve the user's satisfaction.
2. Ciubotaru, B. and Muntean, G. Quality of multimedia streaming-oriented handover management solution for mobile application. IEEE International Symposium on Broadband Multimedia Systems and Broadcasting. (May 2009).
8. Koo, J. and Chung, K. MARC: Adaptive rate control scheme for improving the QoE of streaming services in mobile broadband networks. In Proceedings of the 2010 International Symposium on Communications and Information Technologies (Oct. 2629, 2010). IEEE.
9. Ksentini, A., Viho, C. and Bonnin, J.-M. QoE-aware vertical handover in wireless heterogeneous networks. In Proceedings of the 7th Annual Wireless Communications and Mobile Computing Conference (2011). IEEE.
12. Mohan, S. and Agarwal, N. A convergent framework for QoS-driven social media content delivery over mobile networks. In Proceedings of the 2nd Annual Conference on Wireless VITAE (Feb. 28-Mar. 3, 2011). IEEE.
15. OTT Streaming, 2nd Edition. White Paper. (Sept. 2011); http://www.anevia.com/IMG/pdf/Anevia_White-Paper_OTT-Streaming_2nd_Edition.pdf/
17. Pichon, D. Rieublandou, G., Bonnin, J.M. and Seite, P. Coordinated service and mobility management to maximize user QoE and optimize network resource use. In Proceedings of the IEEE 20th International Symposium on Personal, Indoor and Mobile Radio Communications (2009).
©2013 ACM 0001-0782/13/11
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or fee. Request permission to publish from firstname.lastname@example.org or fax (212) 869-0481.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2013 ACM, Inc.
No entries found