Connectivity: High time for 10 G?

Emerging, high bandwidth services and the adoption of higher speed Ethernet standards, like Gigabit Ethernet and now 10 Gigabit Ethernet, are creating an environment where Internet Protocol (IP) and the delivery of advanced IP services like Voice over IP, IP videoconferencing and IP-based security are becoming common applications.

10 G will raise computing and networking performance to new highs that will support many new applications in the next decade. The rapidly growing Internet will profit from faster response times from service providers. Video on demand that makes intensive use of fast networked storage will become widespread. Health care will benefit from better remote diagnoses made possible by the transfer of very high resolution medical images.

Increasingly, voice, data and video networks are being converged onto a single infrastructure and the demand for reliability and Quality of Service (QoS) has never been greater. A reliable high-performance structured cabling system, built to handle these bandwidth intensive applications, is the fundamental asset required to enable today's leading organizations to increase productivity and sales while decreasing operational costs.

The pace of technology is not slowing. Today, customers are installing local area networks(LAN) that are capable of delivering services that are 1000 times faster than their networks could handle less than ten-years ago. The appetite for increased services and the need for speed are insatiable.

The latest iteration of the Ethernet standard, 10 Gigabit Ethernet moves data over single- and multi-mode fiber 10 times faster than the current Gigabit Ethernet standard, with a range between 65 meters and 40 kilometers.

From its inception, 10 G Ethernet was intended to retain backward compatibility and full interoperability with 10/100/1000M bit/sec Ethernet, while adding a tenfold increase in performance.

In 10/100 and Gigabit Ethernet, the MAC layer works in a linear manner - data moves serially in and out of the MAC layer with all the starting and ending control messages (including clocking and synchronization) embedded inside the datastream. With 10 G Ethernet, it is much more complex.

Worth the investment?

Research by Dell'Oro Group estimates that the average Gigabit Ethernet switch costs about $280 per port, factoring in modular and fixed-configuration prices. With 10G still costing about three times as much as Gigabit (on a per-Gigabit basis), observers say we're still a way off from hitting the pricing sweet spot.

"There hasn't been much widespread interest in migrating to <10G> in the LAN core, other than in some very large (college) campuses and research," says Lawrence Orans, principal analyst with Gartner.

Besides pricing, the need for 10G bit/sec of bandwidth simply isn't there in many enterprise backbones and data centers, he says. "A gigabit is still a lot of bandwidth for many companies, and it is still easy to trunk together two or four for larger connections."

The development of 10G Ethernet over copper cabling could drive further adoption of 10G, analysts say. The IEEE 803.3ak standard for running 10G over InfiniBand-style CX4 cabling was ratified earlier this year.

The 10 Gigabit Ethernet is already standardized - in the form of 10GBase-LX4 - for multimode fiber, which is the most commonly deployed fiber-optic technology within buildings. This was supposed to be a relatively cheap way of implementing 10 Gigabit, and to do so over 300 meters.

According to documents presented at an IEEE call for interest in November, there has been a "lack of broad market availability from multiple vendors of 10GBASE-LX4" and many companies are "privately investigating nonstandards-based, proprietary 10 GbE MM fiber solutions" as a result.

To head that off, the IEEE has now established the study group to take another swipe, and to create a standard that might bring 10 Gigabit Ethernet over multimode fiber to the masses.

If 10 Gigabit Ethernet is going to work for enterprise companies, multimode fiber is key. While there are copper standards in the works, many Gigabit Ethernet installations already use this fiber - and, as one can imagine, users don't want to have to pull new fiber to upgrade those connections to 10 Gigabit.

The idea is that a lower-cost fiber option for 10 Gigabit is necessary for the technology to take root in enterprise companies. Plus, if more Gigabit Ethernet connections are used, and if the number of devices connecting to enterprise networks continues to increase, eventually you need 10 Gigabit to aggregate those connections.

Applications

There are two different physical layer (PHY) specifications. The LAN PHY uses simple encoding mechanisms to transmit data on dark fiber and dark wavelengths.

The WAN PHY adds a Synchronous Optical Network / Synchronous Digital Hierarchy (SONET/SDH) framing sublayer to utilize SONET/SDH as the Layer 1 transport.

In terms of the physical media, 10GbE is specified only for fiber cabling.

Applications in the LAN

The development and standardization of 10GbE technology has been driven by a number of applications in the enterprise LAN, as well as applications in the carrier WAN and the MAN.

• Uplink for a Server Cluster

Business applications such as enterprise resource planning (ERP), customer relationship management (CRM), and supply chain management (SCM), as well as Web-based e-business applications continue to grow in importance within the enterprise environment. The enterprise typically starts with a small number of servers running multiple applications, directly connected to the core switch via 10/100 Ethernet links.

To meet the growing needs for performance and availability, the network manager adds additional servers, runs each application on a separate server or across multiple servers, and increases server performance with multiple processors and gigabit links. As the network scales, network managers may add a server aggregation layer between the servers and the core switch.

To increase resilience, they often use dual redundant links to connect each server to a pair of server aggregation switches, which are then connected to the core switch via redundant trunked gigabit links.

Providing increased performance in the uplink from the server aggregation layer to the core switch is a key application for 10GbE. High-performance enterprise servers can now saturate a gigabit link. As the number and performance requirements of the servers connected to the server aggregation layer increases, so does the need for a high-performance uplink.

In the majority of deployments, the server aggregation switches are located in the data center, that is, within 100 meters of the core switch. The network manager can choose to create the link using a 10GBASE-LX4 transceiver and multimode cable or 10GBASE-LR and single mode cable.

• Uplink for a Workgroup Aggregator

With the dramatic drop in the price of Gigabit network interface cards (NICs), over time they are expected to become the de facto standard connection for desktop PCs in the larger enterprise.

Similarly, as the price per port drops for Gigabit Ethernet stackable switches, enterprises will upgrade wiring closet switches to Gigabit. Initially the focus will be on users who need high performance (for example, CAD and graphics workstations), but as prices fall and data throughput needs increase, Gigabit deployment will become more widespread.

As the number of desktop Gigabit users increases, so will performance requirements for the uplink. This demand will initially be satisfied by trunked Gigabit, but when 10GbE uplinks reach an acceptable price (approximately equal to the price of a four-port Gigabit trunk, or $4,000 at today's prices), they will become the natural successor to gigabit links. In addition to the performance benefit of 10GbE, enterprises will also be attracted to the installation cost savings they can realize by running a single 10GbE link.

The majority of enterprises have installed multimode fiber in the risers that link the data center to the wiring closets. Therefore the ideal technology for this application will be 10GBASE-LX4, which provides a 10GbE link over FDDI-grade multimode fiber over distances up to 300m.

• Uplink for a Campus Switch

In campus environments, there is typically a distribution switch layer within each building. This layer aggregates the wiring closet switches and any local servers within the building. The distribution switch is then connected over a long-distance fiber link to the building in which the data center is located.

Single mode fiber is usually installed between the buildings, as the buildings are typically many hundreds of meters apart. Obviously, this link is a potential performance bottleneck as Gigabit begins to grow at the desktop. Again, performance and availability can be improved by using a trunked Gigabit link. Typically there are multiple fiber strands between building, so trunking does not require the laying of additional fibers-but the cost of terminating dark fiber is significant.

Therefore, when a 10GbE uplink based on the 10GBASE-LR standard (which allows a 10km link over single mode fiber) becomes available at an acceptable price, this technology is expected to become an attractive alternative to Gigabit trunking.

• 10GbE in a Core Switch Solution

Offering 10GbE in a core switch solution is an obvious complement to a 10GbE uplink for a workgroup switch.

Various 10GbE module options are required to support the different applications :



- A 10GbE connection to a local building distribution switch or server farm. The most obvious solution is 10GBASE-LX4, which will support up to a 300m link over FDDI-grade multimode fiber
- A 10GbE connection to a distribution switch in a different building on the campus. 10GBASE-LR will support up to 10km over single mode fiber.

- A local core-switch-to-core-switch connection, to provide a high-speed connection between dual redundant core switches. Any transceiver type can be used, although if the link is less than 28m, 10GBASE-SR may provide the most cost-effective solution.

*The next article on 10 G technology will focus on the application of 10 G in the areas of MAN(metropolitan Area Network) and WAN(Wide Area Network).