How to optimize content delivery with segmented download?

1 Star2 Stars3 Stars4 Stars5 Stars (1 votes, average: 5.00 out of 5)

Large files and heavy web content can make pages load for too long. We keep generating new and new content while the number of users is constantly growing. So as a content provider you need to find the right tools to optimize content delivery to keep users engaged. How to optimize the delivery of files and boost performance of your content delivery network? Segmented download is the secret of battling downtime and lags.

Monolithic Downloads

Initially, when a user made a request for some content, it was streamed from the origin. It could be simultaneously cached on a local server. If users request different byte ranges, another stream would be opened to serve the content. That results in a higher load than expected while a file is pulled into the cache. There are two main issues with monolithic download used in traditional CDN solutions:

  1. Although we can transfer many cached files in and out of memory, the increasing volume of content takes up a lot of resources, because such download process requires the full file to be in memory during delivery.
  2. Downloading of the entire file puts extra draw on the origin server, particularly when the file is large.

Even if the general cache-hit-ratio is high for the origin server, there will be times when it’s not in cache because it’s been purged or rarely requested.

Segmented Downloads

Large files are rarely consumed, or are partially consumed, and segmented download pulls the ranges from the origin that end users do consume, which helps to lower the total outbound traffic from the origin. For instance, if there is a two-minute fragment of a video that users usually watch, a movie that no one watches to the end, the network will only pull the data required to fulfill the request. This minor change can greatly improve cache efficiency. When this model is used, a large file can be segmented into 8-megabyte chunks with only some of them cached, which will optimize inventory space, download speed and other values.

However, segmented downloading has some challenges. For instance, if you use a TTL (time to live) system that also works in your origin, you will need to adjust TTL to be long enough for all segments to be transferred. To maintain the function of instant global purge, the inventory should be life-aware and segment-aware. Segmented downloading model is only being developed and improved, but it will definitely become a must in advanced CDN networks.

Vadim Kolchev

52 publications

Vadim has graduated from Moscow Institute of Entrepreneurship and Law as finance and credit specialist. Prior to starting to work in hosting business directly, he occupied various roles in several companies, including but not limited to banking sphere and sports. As of 2015 he works for INXY Holding, with SpaceCDN being a vital part of the hosting branch of its business. Being tech enthusiast, he has started writing articles about dedicated servers, CDN, storage solutions and other hosting services long ago, and since then accumulated a lot of experience and knowledge in the field. Building hosting sales and support departments from scratch has added even more experience and knowledge and allowed to see the business from the inside and build required expertise. Now Vadim is CPO and COO of a successful hosting business. Having several important interviews and publications at platforms such as Hosting Journalist and Forbes, he continues to share knowledge about this branch of technology that has become not only his job but also a passion.

Learn more about us
in social networks