How to Organize CDN Origin Storage Efficiently?

1 Star2 Stars3 Stars4 Stars5 Stars (45 votes, average: 5.00 out of 5)
Loading...

Since CDN is a scalable and flexible hosting solution, it processes most requests from the edge cache without connecting with asset distributor’s origin or application. But in order to be available in cache, the content should also be stored in the origin source: in case of cache miss, it will be retrieved from the origin. It concerns all types of assets including images, files, videos, etc. Therefore, the origin storage system is a critical component for application functioning.

The vast majority of CDNs historically feature file-based solutions designed to store content on disk and serve as the origin server and destination for CDN cache. Alternatively, object stores, cloud storage services, and CDN cloud-based solutions, as well as objects-based architectures. The main peculiarity of origin storage service in CDN is that it should be more beneficial than monolithic cloud environment. CDN is a huge infrastructure, and the origin storage is one of the most crucial applications in this system.

In fact, origin storage is totally different from caching applications provided by CDNs. Storage requires durability and reliability, while objects in cache are removed and renewed when they expire or become less popular. CDN operates as a distributed architecture that is connected with telco providers. If storage is spread over CDN in several locations, requests from visitors for the content that is not cached yet will be delivered much quicker from nearby location. But the speed of request processing will be much slower, if it has to pass through CDN and potentially access the origin storage to retrieve the content.

Some content distributors are okay with the fact that the availability of content and speed of loading are good enough for single locations, when they’re stored on cloud storage (the applications will continue to work, even if the provider experiences some technical issues). However, others are not satisfied with such solution. This is why some companies offer using proprietary tools to create your own CDN solution that uses multiple storage locations. However, the cost of creating one’s own infrastructure is significant, especially when it concerns multi-terabyte libraries.

Besides, when clients put focus on the price of storage they actually don’t consider extra costs of replicating content, or overall storage access fees. Access to the content in conventional cloud storage can appear to be much more expensive than the cost of storing content. Therefore, as a content owner you need to choose a CDN provider that charges a stable fee for storing multiple copies of your content. There shouldn’t be extra charges for uploading content to additional storage or accessing it from storage or cloud. Note that if you store content in several places, it is delivered faster, if it’s not in the cache. In this case, storage costs can be absolutely justified by customer satisfaction and increased conversion.

Realizing that every company needs specific object storage integrated with content delivery network, and challenges of content migration can be different, some providers start introducing intelligent storage technologies. Thus, the movement of objects into storage is automated and based either on audience demands, or manifest of files. Additionally, content can also be stored in provider’s storage services. The parameters of migration can be specified, as well.

When searching for the best CDN solution and origin storage option, users should find the one that performs automatic replication to several locations depending on regional policies. Policies can be applied to separate regions, or be used globally. This way, the content is automatically distributed across locations to be close to the target audience, and the content absent in the cache will be served from the best origin storage location according to the request.

That can be useful in a number of cases. For example, in case of new content production its movement to CDN storage can be automated: it will be transferred after the first request. This option is great, if you migrate a library from your current storage to a CDN origin storage. Automatic uploading may be helpful for pre-caching: CDN service will distribute files over their origin storage, and when the content becomes available online, CDN network will handle traffic spikes offloading demands from user’s infrastructure.

CDN solutions are not about high loading speed only: high quality of delivery also matters a lot. Now you’re aware that there’s the whole range of services helping to manage content production workflows and to improve user experience. With smart solutions for request handling you will see significant elevation of performance.

Vadim Kolchev

52 publications

Vadim has graduated from Moscow Institute of Entrepreneurship and Law as finance and credit specialist. Prior to starting to work in hosting business directly, he occupied various roles in several companies, including but not limited to banking sphere and sports. As of 2015 he works for INXY Holding, with SpaceCDN being a vital part of the hosting branch of its business. Being tech enthusiast, he has started writing articles about dedicated servers, CDN, storage solutions and other hosting services long ago, and since then accumulated a lot of experience and knowledge in the field. Building hosting sales and support departments from scratch has added even more experience and knowledge and allowed to see the business from the inside and build required expertise. Now Vadim is CPO and COO of a successful hosting business. Having several important interviews and publications at platforms such as Hosting Journalist and Forbes, he continues to share knowledge about this branch of technology that has become not only his job but also a passion.

All Comments - 0

Submit Your Comment Here
Learn more about us
in social networks