Comparing stream counts for video editing servers – Part 1

Can your system play x number of streams of my favorite codec?

When you’re searching for a new shared storage system it’s common to compare a system’s published stream counts against those from different manufacturers, but there’s much more behind these numbers than meets the eye.

The purpose of this article is to explain how stream counts can be generated, and highlight some of the less obvious things that can affect a shared storage system’s ability to do (or not do!) a certain number of streams of video playback. We think it’s important when making comparisons to understand how stream counts are generated, and to know what other factors can affect streaming performance and playback. We will also explain some of the ways we generate our own video stream counts and discuss why we get them that way.

Part 1

(This is a two-part article. Part 2 explores the stream count topic more deeply and can be found here.)

Getting stream counts

There are several ways to estimate stream counts, and it won’t always be clear to see how a system’s stream counts were measured just by looking at “the chart,” so trying to use these metrics to compare one system to another can be kind of tricky at best, and misleading at worst. The process a storage manufacturer must go through to determine a system’s stream count is challenging to do accurately and repeatably; it requires testing in a methodical way, with real applications and a vast array of different kinds of hardware and software.

Stream count charts are common, and they often leave out — intentionally or not — some important details!

You frequently won’t know which method was used, but understanding the following methods for generating stream counts (ranging from shortcut to legitimate testing) will help you have a clearer picture of what’s behind the numbers you’re seeing.

1. Benchmark utilities: These kinds of utilities are usually just run on one workstation, and they work by analyzing how long it takes to write or read a file to/from storage. These tools have their place and can be really useful for quick “in the ballpark” tests, but what these tools are best at measuring is the ability of a single workstation to generate a synthetic workload, particularly for a single-user storage device. The trouble with benchmark utilities is that they don’t take into account many critical elements that can occur in a shared storage environment (particularly those that we’ll cover in part two of this article). Further, they may not even produce enough of a workload to sufficiently tax a storage system, in turn, giving misleading results.

2. Simple calculation: bandwidth available / throughput required = streams: By taking the available system bandwidth (which may also be an inflated number!) divided by the throughput required by your codec of choice, you will get a theoretical approximation of a stream count. This method of extrapolating a stream count is tolerable for very rough estimates. But in exchange for the convenience of this method it ignores a host of variables—even more so than with benchmark tools—that can dramatically overestimate a shared media storage system’s ability to handle a real-world workload. This method is essentially “cooking the books.”

3. Application testing: This is by far the hardest way to measure a shared storage system’s ability to reliably handle a workload for a sustained duration. It’s a very tedious process, but the results are worth the effort. When done properly it’s very time consuming and requires a lot of expensive hardware and software and a methodical level of control with things like workstation hardware/specs, OSes and configuration, NICs and drivers, firmware, clips/clip length, etc. It’s also the only way to really know how many streams a system can play. Application testing is the method we use to measure stream counts in our SNS labs, and this is the only way to really know what any shared storage system can do, reliably.

So when we talk about stream counts we’re talking about a number that is derived by using a network of real workstations each playing real streams of separate video files using real applications like Avid Media Composer, Final Cut Pro 7, FCPX, and Adobe Premiere Pro. (In other words, we test our systems just like how you’d interact with a shared storage system in daily production workflow.)

The Obvious Variables

So, aside from the codec itself and the method used to calculate the number of streams, what are some of the major differences that come into play when determining or comparing how many streams of video a storage system can reliably handle? Here are four big variables:

• Number of drives used — The number of disks used for the stream count test is one of the most important things to know, yet sometimes it’s not published.

• Frame rate — It can be really easy to overlook frame rates when comparing stream numbers, but frame rates are actually really important when you’re measuring performance. For example, all else being equal it’s 20% less work to handle a stream at 23.976 fps than 29.97 fps for the same codec. So, if a stream count for a codec at 23.976 fps is 20 streams, the same stream count at 29.97 fps would be approximately 20% less, or about 16 streams.

• RAID level used — RAID 5? RAID 6? RAID 0? RAID 50? RAID-Z? Each has its own strengths, and you should know which is used to derive a stream count. Generally speaking, it’s easiest for a system to handle more streams with RAID 0 than any other level, though RAID 0 is rarely used in real world shared storage deployments.

• Model of drives in use — All drives are certainly not created equally, and this goes for varying capacities even within the same family of drives from a given drive manufacturer. For example, a storage system may be able to play 30 streams of a codec with an array of 6TB drives, but simply replacing those drives with a 2TB model from the very same disk manufacturer will most likely provide fewer streams.

We hope these points have helped you understand the various ways stream counts can be generated and what you should know when looking at charts. Look out for our next article where in part 2 we will uncover and take a deeper dive into some of the elements of “counting streams” for shared storage systems.If you’d like to get a look at our stream counts you can do that here.

Want to know more? Contact us today — we’d love to hear from you!