Understanding the bandwidth myths.
Don’t believe everything your ISP may tell you about bandwidth. What you see is not necessarily what you get.
Anyone involved with the construction and management of multi dwelling units (MDUs) should have some familiarity with the importance of amenities like Internet access. Unfortunately getting reliable Internet access is one of the most misunderstood and potentially expensive property utilities, especially if you can’t deliver reliable service. In some cases Internet access is purchased in bulk by owners and is provided as part of the lease agreement. In the current economic environment, owner/managers are looking for ways of improving the competitive posture of properties and are considering bulk Internet service agreements.
Recently published bandwidth utilization studies indicate a rapidly increasing trend towards real-time entertainment traffic. This includes streaming audio and video, peercasting, place shifting and flash video. One report from Sandvine, for example, claims real time entertainment “now accounts for 26.6 percent of total traffic in 2009, up from 12.6 per cent in 2008”.
These findings track with what we are seeing from student housing at Korcett Holdings and they will have a significant impact on network design and performance in the next 24 months. “During the past two service years we have seen bandwidth requirements grow by an average of 25% per year from our student housing properties,” notes KHI Direct of Business Intelligence Charles Flynn. “One of the more recent and significant increases in bandwidth demand occurred with the introduction of NetFlix streaming on Xbox and Playstation during the holiday season in 2009. This resulted in a 20% bump in bandwidth consumption between December 09 and February 1, 2010.” As you might expect, the increase in complaints from residents and subsequent cost of increasing bandwidth has caught property owners/managers completely by surprise.
One of the communication issues facing service providers and owner/manager customers is setting and managing expectations when it comes to Internet services. For example, the average non-technical customer understands that when the service provider tells you that you are getting 5 Mbps of download bandwidth; they will assume that 5 Mbps is available anytime of day or night for as long as they need it. This is not the case.
Service providers tell customers that they provide a 5Mbps download by 1 Mbps upload (5 x 1) capacity and then bury the specifics in the fine print. In point of fact the customer can only get 5 x 1 services as long as the network is not crowded. If there are other customers using the network at the same time chances are good that the customer will not get 5 x 1. The fine print in the service provider agreement typically stipulates that the provider must make it’s “best effort” to provide the customer with a 5 x 1 service. In all fairness to service providers, this is not a problem as long Internet usage patterns remained stable. Traditional, product-centric network design typically used by service providers was never intended to deal with the rapid increase in demand for Internet-based content.
Oversubscription Ratio (OSR)
The OSR is an engineering metric that tells the ISP how your network has been designed. If, for example, there were 10 subscribers on a network with an advertised service cap of 5 Mbps x 5 Mbps, you would need 50 Mbps of bandwidth to “guarantee” bandwidth to all subscribers. This is an OSR of 1 to 1 (1:1) and a synchronous 50 Mbps pipe will cost the service provider something north of $2,500 per month. Because until recently Internet traffic has been very sporadic, most service providers could get by with providing one tenth of the bandwidth required to deliver advertised data rates. This requires an OSR of 10:1. The “market price” on a synchronous “best efforts” 5 Mbps circuit is about $800 per month, much less expensive than $2,500. So you can see why the practice of oversubscription has been a wonderful source of revenue for ISPs and is something that they don’t typically discuss. It is the foundation upon which much of the industry is built. For this example, assuming a 60-month service agreement for 10 subscribers paying market rates, the ISP can make as much as $72,000 in profit. Do the math for much larger groups of subscribers.
The real problem here is the slippery definition of “best efforts” and the lack any reference to OSR in the sales process and service agreements. As the demand for better performance from Internet amenities increase the practice of oversubscription will create significant problems for ISPs and owners. Service providers cannot continue advertising fixed prices for a “maximum possible bandwidth” delivered on “shared bandwidth circuits”.
For student housing developers, access to the Internet has evolved into a mission critical amenity. The ability for residents to quickly and reliably connect to the Internet is directly tied to occupancy. There are typically two ways that developers can make sure residents have reliable Internet access; leave it up to the regional Internet service providers (ISP) or demand bulk “managed services” for all your residents. If you get it right you can attract and retain residents. If you get it wrong it will cost you residents and may ultimately kill your long-term return on investment.
Even when working with very reputable ISPs who are delivering dedicated bandwidth, developers struggle to deploy and maintain reliable Internet access. The underlying problem has to do with the fact that providing reliable Internet access is not a static problem. The range of new Internet-based services and content is growing exponentially and it is driving the demand for bandwidth through the roof. This means that regardless of how stable your premise network is on day one, things will change. Most of the technology available to build and manage networks is static in nature. In all fairness to ISPs and network equipment manufacturers, static solutions have worked for years. As the rate of demand for new services has risen, however, static solutions must be updated more frequently or networks will fail.
The good news is that there are a few cable companies who offer ubiquitous managed services. They have standardized the premise network design and will work closely with developers to tune services to their business needs. These include Time Warner Cable, Comcast, Suddenlink, Charter, Insight and a few others.
The Ultimate Bandwidth Hog
Bandwidth consumption is primarily being driven by the consumption of Internet distributed video. An encoded 720p video stream, for example, consumes between 1 – 3 Mbps. A 1080p video stream consumes between 2 and 6 Mbps. As we have shown above, the fine print on an Internet service agreement usually includes the term “best effort” and means that the service provider probably used oversubscription and will not guarantee bandwidth. No one wants to discuss what happens when users run out of bandwidth, something akin to running out of hot water when too many people want to take a shower at the same time.
What does this mean for MDU owners and managers? If a property has 1,000 residents and a property bandwidth cap of 100 Mbps, it can support a 1 Mbps x 1 Mbps connection on a 10-to-1 oversubscription basis. Adding video to the mix changes things dramatically. Each video stream, assuming minimum resolution, consumes 1 percent of the total available bandwidth. Translation: Less than 10 percent of the population watching minimum-resolution 720p streams will crater the network. With higher resolution 1080 flat-screen HDTVs, 14 to 16 residents streaming video can crash the network.
How likely is this to happen? It’s only a matter of time. The real question is, are you ready for it? Is your finance department prepared for the cost of unexpected network and bandwidth upgrades.
Currently the total burden of video streaming and downloads are typically spread out among many providers throughout the day. There is a very important distinction, however, between “library, pre-encoded video” and “live streaming day and date video”. Students downloading or streaming their favorite shows throughout a 24-hour period, using a cloud server storage system put a certain load on the system. But the far greater burden and threat will come when a live event or program is streamed over the broadband connection at a particular time and 80% of the users on the network decide to watch (i.e. think Presidential inauguration or the Superbowl in HD). To address this kind of load will require multicasting to properly “managed” local switches on fast internal networks. This kind of managed Internet access becomes the ONLY way to insure against looming network failure.
In most cases neither the ISP nor the developer discriminate between managing bandwidth intelligently and simply throwing more bandwidth at the problem, but the distinction is crucial. If you are fortunate enough to have a mature managed service environment, you will know when to add bandwidth. Without intelligent management, you WILL end up paying for more bandwidth whether you need it or not. So the advise to developers is this, “whether you are building or buying an MDU make sure to get some help in selecting your ISP and designing your Internet amenity”. Don’t pay for services that can be easily compromised and force you into even more costly upgrades.
For those who fail to choose wisely the financial impact will be significant. Lets say you are servicing 500 student residents with 50 Mbps pipe (minimum suggested provisioning for this number of students) and the advertised Internet amenity claims 1 x 1 Mbps service for each resident. Traditionally ISPs will use an OSR of 10:1 and provide you with a 50 Mbps pipe at an internal cost of roughly $2,500 per month. The cost of providing the total aggregate price for a 500 Mbps circuit for an OSR of 1:1 is roughly $11,000 per month. When the residents start complaining about poor Internet access your ISP will recommend more bandwidth, although they will probably not mention OSR. You can expect a recommendation of somewhere in the neighborhood of twice the original bandwidth and an extension of service term. In most cases this kind of alteration to your Internet service agreement will represent a significant and unexpected increase in recurring monthly expenses.
When dealing with ISPs it is important to understand that bandwidth is just one of a complex set of network design variables that dictate stability and performance. Unless developers assume control of network design, they will have to rely upon the competence and goodwill of their service providers. However, there is a simple fix. Instead of paying for a new network design for each new property, developers can find a competent network design firm, pay to have the design done once and use this design as the standard for all future projects.
For example, one fairly important component of network design has to do with the network equipment selected. The primary piece of equipment used to deliver Internet access to residents is called a “managed switch”. Because no two managed switch vendors have the same product functionality and command line interface (CLI) and most ISPs have their own preferences when selecting network equipment, network design from one property to the next is always dissimilar. This makes achieving any kind of consistency in how networks behave nearly impossible. So developers who want to deploy and support a large number of geographically remote networks can reduce costs and improve consistency by limiting the number of different managed switches you are willing to deploy. Don’t let the ISPs make this decision for you. Establish your own design standards and require ISPs to comply.
In terms of design costs, the benefit of using design standards is straightforward and significant. If the design for a single property network costs $12,000 in engineering fees and a developer has 50 properties, then designing each property network individually will result in a total cost of $600,000. In most cases retaining an engineering firm to develop a set of design standards that can be used for the networks at all 50 properties can save over half a million dollars in upfront capital costs.
From a problem resolution perspective the benefit of using design standards is harder to calculate, but it’s still real. For example, suppose some new, wildly popular Internet-based service or malware is introduced that consumes a great deal of bandwidth. The resulting spike in bandwidth consumption creates performance problems for your residents. Let’s assume your service provider is responsive and that the leasing office spends only eight hours to deal with irate residents while the problem is identified and corrected. Without design standards, the developer ends up paying to resolve the same problem 50 times, for a total of 400 hours of leasing office time. If you assume an hourly rate of $20 for front office staff, this translates into $8,000 in operational expense. Finally, let’s be optimistic and assume that you only see one such problem a quarter. This translates in to an unexpected and ongoing operational expense of $32,000 per year.
Hopefully we have shed some light on the complexities of designing and maintaining data networks in a geographically diverse portfolio. Remember, the property data network is not simply another utility infrastructure; it is a constantly changing living environment that must be managed in real time. Don’t allow the service provider to use non-standard, product-centric design practices. Find competent engineering to help you develop network design standards that meet your needs. You will be far less likely to keep hearing that “you need more bandwidth”.