From The Sip Trunking Experts

TMCNet:  PCI Express Gen3 Enables Smaller, Faster Server Storage

[December 04, 2012]

PCI Express Gen3 Enables Smaller, Faster Server Storage

(ENP Newswire Via Acquire Media NewsEdge) ENP Newswire - 04 December 2012 Release date- 04122012 - Taking advantage of PCIe Gen3 requires disk adapter silicon that's up to the task.

Storage vendors are focusing on small form-factor solutions that will fit into smaller chassis while providing the high performance that data centers require. Storage components, such as hard disk drives, are getting physically smaller (though increasing in capacity). 2.5-inch drives hold more capacity per space occupied than the 3.5-inch drives they are replacing, and SFF HDDs now boast storage capacities of 1TB or more. Choosing the right SFF configuration is important consideration for data centers because finding the right combination of form, fit and function, allows them to deploy one SKU for almost any configuration and simplify everything from the purchase decision to installation to maintenance.

To match the system's smaller footprint requirements, low-profile storage adapters are becoming more common as well, but delivering top I/O performance and low latency in such a compact form factor requires changes in connection methodology. Fortunately, PCI Express Gen 3 doubles per-lane bandwidth to low profile SAS and SATA drives, but requires a minimum of 16 native 6 Gb/s ports to realize maximum performance. This article examines the performance of PCIe storage adapters and quantifies bandwidth as a function of various adapter and driver parameters.

The New Generation of PCIe PCI Express (PCIe) is a motherboard-mounted expansion bus that, through a connected device such as a RAID adapter, connects the host system processor to add-on peripherals, such as storage systems. Introduced into servers and workstations in 2012, the third generation of PCIe (PCIe Gen3) doubles bandwidth to the host compared to its PCIe Gen2 predecessor, increasing per-lane throughput from 250 MB/s to 500 MB/s.

With PCIe Gen2, eight 6Gb/s SAS/SATA ports are sufficient to achieve maximum performance. However, PCIe Gen3 requires a minimum of 16 native 6Gb/s SAS/SATA ports to double the bandwidth through the storage connections.

A select group of storage adapters that claim to be designed for PCIe Gen3 has appeared the market, but most of them max out at eight ports and cannot take full advantage of PCIe Gen3's superior performance. As we will see, some SAS/SATA RAID adapters, available with 16 or 24 native SAS/SATA ports, are designed to fully exploit the high-performance characteristics of PCIe Gen3 (Figure 1).

Significance of High Native Port Count In recent years, the storage industry has been transitioning from 3.5-inch storage drives to 2.5-inch Small Form Factor (SSF) drives as advancements in technology allow storage vendors to address the aforementioned physical space challenges faced by data centers. Not only do SFF drives offer the obvious advantage of allowing more drives to fit into the same server rack space, but 2.5-inch drives hold more capacity per space occupied than the 3.5-inch drives they are replacing. Indeed, SFF HDDs now boast storage capacities of 1TB or more.

Additionally, the cost of 2.5-inch flash-based solid state drives (SSDs) is finally coming more in line with HDDs in terms of the traditional 'cost per GB of capacity' metric. That, combined with a higher read bandwidth, higher input/output operations per second (IOPs), better mechanical reliability, and higher resistance to shock and vibrations compared to HDDs, is driving an industry-wide transition to SSDs. As the quantity of drives in a server chassis increases, the storage adapter card's port count requirements also increase.

Expander is no option The traditional method for increasing the storage adapter's port count has been through the use of an expander - a board that enables the connection of additional attached SAS or SATA devices when the adapter does not have enough ports to accommodate them. However, expanders have a number of limitations: not only do they add complexity, they also occasionally face compatibility issues with other components in the storage solution.

On top of that, expanders are notorious for causing latency and limiting data transfer bandwidth. Both of these issues have long been tolerated by data centers using HDDs, as they did not cause a huge impact on the already slow read and write speeds of HDDs. But as higher-performance SSDs gain traction in storage solutions, the latency and bandwidth issues of expanders have become more noticeable, and therefore less acceptable.

In a RAID 5 configuration using 24 SATA SSDs (Figure 2), the use of expanders causes a roughly 60% performance drop on random read IOPs, and a roughly 20% performance drop on OLTP read/write IOPs, compared to a direct connection through native ports.

Similarly, in a RAID 5 configuration with SATA SSDs (Figure 3), the use of expanders cause a roughly 70% performance drop on sequential read MB/s, and a roughly 40% performance drop on sequential write MB/s, compared to a direct connection through native ports.

This problem can be partially overcome if SAS devices are used, since they are dual-ported and allow all eight SAS port connections to be leveraged through the expander. However, as illustrated in Figures 4 and 5 below, performance of the 8 6Gb/s SAS ports flattens out at the peak data rate and competing products cannot match the performance of adapters with 16 or more ports, such as the Adaptec Series 7.

Another drawback of expanders is the additional cost they add to a storage solution - about $200 for the expander itself plus the cost of cables plus installation, increased power consumption, and maintenance costs. An ideal solution for data centers would be a 6Gb/s storage controller with a high native port count that can take advantage of PCIe Gen3's performance.

However, as mentioned, most 6Gb/s storage adapters max out at only eight ports.

Multiport Silicon Solution As shown above, higher port adapter silicon takes advantage of the performance offered by PCIe Gen3. The Adaptec Series 7 SAS/SATA RAID adapter family uses PMC's 24 port PM8015 RAID-on-Chip (ROC), which combines an x8 PCIe Gen3 interface with 6Gb/s SAS ports to enable a new generation of high performance, high native port count RAID adapters that are unmatched by any other ROC in the industry.

Traditionally, RAID adapter performance has centered around read and write throughput, measured in megabytes per second (MB/s). Using this metric, our company's adapters perform up to 83% better than competing RAID adapters - 6.6GB/s on sequential reads and up to 2.6GB/s on sequential writes on parity RAID 5.

Moreover, with the popularity and growth of SSDs, input/output operations per second (IOPs) is emerging as the new 'lead horse' in performance metrics, with the most common configuration being the 4K random-read number. Using 4K I/O size in random scenarios is driven by the fact that most operating systems use 4K cache sizes in the server DRAM and, with that, 4K is typically the smallest I/O size for random workloads. In a RAID 5 configuration with 16 direct-connected SSDs, these 16-port, PCIe Gen3 adapters benchmark at 450K IOPs - nearly 10x the performance of previous-generation RAID adapters, and more than double that of the competition.

As noted earlier, RAID adapters with only eight native ports cannot pass PCIe Gen3's performance gains through from the bus to the storage connections. These 16- and 24-port adapters are the first on the market to take full advantage of PCIe Gen3 performance gains by using HD mini-SAS connectors to offer options with 16 or 24 native SAS/SATA ports Conclusion In order to continue meeting customer demand for fast and reliable access to data and content, data centers must employ efficient and physically smaller storage solutions that maximize I/O capability while fitting within budgetary and physical space requirements. A new generation of PCIe Gen3 storage adapters seek to enhance storage I/O performance by offering 16 ports required to maximize PCIe Gen3 performance.

Series 7 adapters perform up to 83% better than competing RAID adapters in read and write throughput - 6.6 GB/s on sequential reads and up to 2.6 GB/s on sequential writes on parity RAID 5 - and lead the field with 450K IOPs - nearly 10x the performance of previous-generation RAID adapters, and more than double that of the competition.

Juergen Frick has served as a Product Marketing Manager of PMC-Sierra's Channel Storage Division since June 2010. In June 2011 he was promoted to Senior Product Manager. In this role, Mr. Frick is responsible for Adaptec by PMC board level products and the EMEA market. Prior to this appointment he was the EMEA Product Marketing Manager for Adaptec's RAIDController- products for Serial ATA (SATA), Serial Attached SCSI (SAS) and SCSI. There he was responsible for the Channel business with RAID components and for the OEM-Business in Europe, Middle East and Africa (EMEA). Prior to Adaptec, Mr. Frick held various positions in technical marketing at ICP vortex and Intel.

About PMC PMC (Nasdaq:PMCS) is the semiconductor innovator transforming networks that connect, move and store digital content. Building on a track record of technology leadership, we are driving innovation across storage, optical and mobile networks. Our highly integrated solutions increase performance and enable next generation services to accelerate the network transformation. For more information visit

[Editorial queries for this story should be sent to] ((Comments on this story may be sent to (c) 2012 Electronic News Publishing -

[ Back To SIP Trunking Home's Homepage ]

Subscribe here for your FREE
SIP TRUNKING enewslettter.

Featured Partner

Featured Whitepapers

SIP Security for the Enterprise
Voice over IP (VoIP) is incorporated into a variety of computer networks, both public and private, and used for everyday transactions and communications among carriers, businesses, government agencies...

Making A Broadband Purchase Decision
Businesses today have many options for broadband connectivity. Clarifying your particular business needs prior to selecting a broadband provider will ensure an optimal match of broadband service to your requirements.

Voice-Optimized Network Delivers Premier Call Experience
Customers equate call quality with business quality. Real-time communication, interpersonal interaction, and the cordial tone of a call center representative can create a positive impression of your business that no email can match.

Featured Case Studies

Business Telecom Expenses Reduced 50%
A small to medium sized company in the midwest was interested in migrating to IP Communications, but in today's economy, they were hesitant to upgrade their communication system due to their perception that the cost would outweigh the benefits.

Multi-State Company Cuts Telecom Costs 50%
A multi-site, multi-state company with extensive monthly long distance fees and toll-free charges did not have adequate broadband for Broadvox SIP Trunking requirements, nor did they have a SIP enabled telephone system.

Discover Leisure Connects Remote Users to its IP-PBX
Discover Leisure is one of the largest resellers of caravans and motor homes in the UK. With 15 branch of?ces all over the country, the company spent a great deal of money every month just on internal phone calls.

Featured eBOOKS

Internet+: The Way Toward Global Unified Communication
Connecting the telephony of the enterprise PBX or Unified Communications (UC) system using SIP trunks instead of conventional telephone lines has been very successful in recent years.

What is SIP Trunking? Edition 2
SIP trunking is becoming more of a focus for service providers. One key issue many service providers face when deploying SIP trunks is NAT, or Network Address Translation, traversal.

What is SIP Trunking? Edition 1
A vast resource for information about all things SIP - including SIP, security, VoIP, SIP trunking and Unified Communications.

Featured Videos

Broadvox VAR Testimonial VAR 1:
Part 1 of the VAR (Value Added Reseller) Partner Program Testimonials for Broadvox...

E-SBCs AS The Demarcation Point:
Ingate's Steve Johnson talks to Erik Linask about the role session border controller plays as the demarcation point at...

Demystifying DPI
How can deep packet inspection protect your SIP traffic as well as your entire network?

Featured Resources

Partner Program Overview:
Over 4,000 VARs, Master Agents, Solution Providers, and Independent IT Professionals trust Broadvox. We offer customized services and solutions to fit seamlessly into any company's business model. And when you partner with Broadvox, every member of our team stands behind you and your customers 100%...

SIP Trunk UC Summit

What's New

Presenting the New Ingate/Intertex Website:
Internet+ is an extended Internet access allowing high quality SIP (Session Initiation Protocol) based real-time person-to-person communication, everywhere and for any application. It applies to both fixed and mobile networks ...

Featured Blogs

Featured Webinars

Secure SIP Trunking:
What You Need to Know

Successfully Deploying Enterprise SIP Trunking:
Tools and Techniques for Overcoming Common Roadblocks

Featured Podcasts

Getting the Most Out of Your SIP Trunks:
Ingate's Steve Johnson and TMC's Erik Linask discuss how best practices forgetting the most out of SIP Trunking services and common pitfalls to avoid.

Featured Datasheets

Ingate SIParator E-SBCs
Adopting SIP is a simple process with the Ingate SIParator, the secure enterprise session border controller (E-SBC). The SIParator makes secure SIP communications - including VoIP,SIP trunking and more - possible while working seamlessly with your existing network firewall.

Ingate Firewalls
Everyone is talking about enterprise usage of VoIP, instant messaging and other types of realtime communications including presence and conferencing.

SIP Trunk Solutions for Service Providers
The award-winning Ingate Firewall and Ingate SIParator deliver a high quality, reliable SIP trunk connection between the customer's IP-PBX and the service provider network, and solve interoperability issues to simplify deployments and support for remote diagnosis of reported issues.