Evolution of Networks: Understanding the 10 Gigabit Ethernet

Introduction

With the release of 1000BASE-T standard in 1999, members of the Working Group 802.3 were free to start work on the next standard. In keeping with tradition, decided to develop a standard capable of transfer rates 10 times higher than the previous one, giving rise to the 10 Gigabit Ethernet (10GbE), which carries a whopping 10 gigabits per second.

Increase by 10 the throughput for each new network standard may seem like overkill, but as the migration to new networking standards is much slower than for new processors and new memory technologies, for example, end up being bigger steps necessary otherwise few would bother to upgrade equipment.
As predicted in the famous Moore's Law, processing power of processors and controllers generally doubles on average every 18 months, and the cost remains more or less constant. Thus, in a period of 54 months we have 8 times faster controllers, and so on, which makes the task of developing new standards for network relatively simple.

The biggest problem is that the cabling does not evolve at the same speed controllers, which forced the committee to bring the cables commonly used up to the limit before throwing in the towel and move to a standard cable more expensive and of better quality.

One example is the twisted-pair category 5, which were originally developed for use in networks of 100 megabits, but who ended up taking his life extended with standard 1000BASE-T with the adoption of a more sophisticated modulation and the use of four cable pairs.

As with Gigabit Ethernet, 10 Gigabit Ethernet development began in the fiber optic cables, which offer lower technically challenging, with the standard cables with copper being finalized last. Many thought it would be impossible to create a standard for 10GbE twisted-pair (after all, we're talking about a transmission rate of 1000 times higher than the original standard 10BASE-T), but ultimately ended up miraculously. Thanks to this, we have again the coexistence of standards for fiber and copper cable standards.

The 10GbE standard for fiber optic cables are divided into two categories: standards, long distance, using single-mode fiber and short patterns, using multimode fiber transmitters and cheaper.
The goal of the patterns of long distance is complementary patterns of 100 and 1000 megabits, offering a solution that can connect remote networks with a speed comparable to or greater than the DWDM and SONET backbones, more expensive technologies, currently used in Internet backbones.

Suppose, for example, you need to connect PCs 5000, divided between the university and the industrial city of a great city. You could use a 10 Gigabit Ethernet backbone for the major backbones, connecting the servers within three blocks and linking them to the Internet, using a network of managed Gigabit Ethernet switches bring the network up to the classrooms and departments, and finally using switches flights to bring the network to students and staff, complete with 802.11b/g/n access points also offer an option for wireless networking.

This establishes a pyramid, where individual users have relatively slow connections, which are interconnected and the connections between servers faster and more expensive, forming a system capable of absorbing several simultaneous video conference calls, for example.

Another application is in focus in the proper use of internet backbones. Using 10GbE, a single fiber optic cable transmits the equivalent of over 600 T1 lines (1.5 megabits each), or with a single 10GbE link we have enough bandwidth to meet a mid-sized city.

Among the patterns of long distance have the 10GBASE-LR (Long Range) that uses lasers of 1310 nm and offers a range of up to 10 km (with the possibility of reaching greater distances using high quality cables), the 10GBASE-ER (Extended Range), which uses lasers of 1550 nm and is able to cover distances of up to 40 km and the new 10GBASE-ZR, independently developed by Cisco and other manufacturers, which extends the range of up to incredible 80 km.

In all three cases, the maximum distance can be extended using repeaters and signal amplifiers, so that the link can be extended to very large distances, creating and interconnecting backbone networks.

Then we have the standard short-range, intended for use in data centers and local networks. As I mentioned, they are based on multimode fiber, which unlike the standard single-mode fibers used in long distance, are quite affordable.

Currently there are only two standards: 10GBASE-SR (Short Rage) technology uses short-wave laser, similar to that used in 1000BASE-SX and is capable of reaching up to 300 meters depending on the quality of the cable used, while the 10GBASE-LRM allows the use of fibers with a core of 62.5 microns, a fiber type of low quality, typically used in 100BASE-FX networks. When used in 10GBASE-SR, these fibers support very short distances (up to 26 meters), but in the 10GBASE-LRM they support up to 220 meters, hence the acronym LRM, "Long Reach Multimode".

Traditionally, the most common is that the patterns of short-distance optical fiber are used to create backbones, connecting the switches and routers in different network segments, while standards for copper cables, are used in individual points.

As he did in Gigabit Ethernet, the working group began developing a standard for copper-short for use in data centers. Then came the 10GBASE-CX4, which uses four pairs of twinax cables to transmit data at up to 15 meters. 10GBASE-CX4 cables use a special connector, similar to that used in InfiniBand, a technology used in network clusters and SANs. It is possible manually crimping CX4 cables, they are already bought the desired length. Here is a PCI-Express x8 connector and detail:

The 10GBASE-CX4 standard is cheaper than those based on fiber, since it is not necessary to use the transceiver (a very expensive component, which contains the optical transmitters and receivers). But, as might be expected, he fell into disuse with the popularization of 10GBase-T (or 802.3an), which is based on the standard twisted-pair cables.

Initially, spoke on the use of cables combined with Category 7 TERA connectors and can support the fifth-grade cables on 10GBase-T, but both ideas were eventually discarded in favor of cable Category 6 and Category 6A.
Use category 5e cables in 10GbE not impossible but would require a very complex modulation system, which increases excessively boards and switches.In addition, the distance would be very short (possibly something close to 15 meters of the 10GBASE-CX4), which would eliminate the usefulness of the standard.

Although Cat 6A cables are already very affordable, we have a huge installed base of points with cables cat 5 or cat 5e, which represent a major barrier to the popularization of 10 gigabit networks. Unlike a home network, where you can replace the cable with a certain facility, enterprise networks require a considerable investment in planning, cabling and certification of points, so any technology that requires the replacement of wiring face great resistance.Many are still making the migration from 100 to 1000 megabits, so the mass adoption of 10 Gigabit networks can still take half a decade or more.

To understand the difficulty in creating a standard for 10GbE cat 5e cables, nothing better than to understand a little better as the 10GBASE-T works.

In 1000BASE-T system is used PAM-5 modulation, where five separate signals are used to transmit 2 bits per baud (combined with control information).Thus, the 1000 megabits are transmitted in only 500 megabauds, ie 125 megabauds in each of the four pairs of wires.

The 10GBASE-T modulation system adopts a much more complex, PAM-16 which, as the name suggests, is based on using 16 different signs in each pair, each represented by a different voltage level. For comparison, the 100BASE-TX there is a difference between each level of 1V, the 1000BASE-T the difference drops to 0.5V and 10GBASE-T drops to 0.13V, which makes the issue of cabling increasingly critical :

Originally, 16 states allow the sending of 4 bits per baud, per pair. But, as usual, you must also send control information is transmitted so that the equivalent of 3,125 bits per baud (3 bits plus one additional bit to each baud 8), which allows the 10,000 received in megabits only megabauds 3200. As the four pairs of wires are used simultaneously, then we have megabauds 800 per pair cables.

Just as in 1000BASE-T, each baud takes only half a cycle to be transmitted, which reduces the transmission frequency. Still, the 800 megabauds result in a frequency of 400 MHz, far beyond the 100 MHz supported by Cat 5 cable.

The following are the list of category 6 cables, supporting speeds of up to 250 MHz and are built in much more stringent standards with respect to signal attenuation and crosstalk. Despite the frequency is lower than required, it was possible to include support for them within the standard, but only for short distances, only 55 meters.

This is because the frequency cable is not supported by an exact value, but the frequency for which he is certified for transmission to 100 meters. A Cat 5 cable can carry signals over 100 MHz, but the ease with which they would not reach the final of the 100 meters with an acceptable quality.

Reducing the length of the cable, reduce the attenuation, which allows the cables supporting the transmission of signals of higher frequency, but only over short distances. In the case of Cat 6 cable, it was proved that they are capable of transmitting signals from 400 MHz to 10GBASE-T, but only up to 55 meters, hence the specification.

In practice, some cat 5e cables that exceed the specification also supports the 400 MHz frequency over shorter distances. If you're lucky, you may have success using a cable 10 or 20 meters, for example. However, standards need to work "forever" and not "sometimes" and rightfully so the cat 5e were removed from the final specification.

For it was possible to use cables up to 100 meters, as in previous standards, was created Cat 6A standard, which supports frequencies up to 500 MHz and is based on even stricter standards.

Although still more expensive cables Cat 6A tend to drop in price as technology is becoming popular, allowing a gradual migration. From the moment the price difference is not an impediment, it is worthwhile to switch to category 6A cabling in all new installations, even in networks to 100 megabits and 1000, as the default standard cat5e and superexcede cat6, meeting all the requirements and they offer the advantage of allowing migration to gigabit 10 when desired.

Although the gigabit ethernet is already standard on new motherboards for some time, not to mention the large supply of loose boards, there is still a huge installed base of 100 megabits points, which will continue to be used for a long time.

As today most of the points is basically just used for web access and / or basic share files and printers, even a 100 megabit port ends up being more than enough, since about 8 MB / s effective rate transmission are sufficient to copy small files and large web access bottleneck is the link speed of access and the local network. With that in mind, it is expected that the 10GbE very quickly became popular in local networks, even though the cost of 10GBASE-T adapters and cables Cat 6A dropping.

In 2008, a 10GBASE-T card cost more than $ 1,000 over and consumed 25 watts. In 2011, there is already a large supply of cards for PCIe servers below the house of $ 500 and the newer chipsets, produced using the technique of 40 nm already offer an in home consumption of 4 watts. Typically, 10GBASE-T cards use a PCI Express x8 slot, since anything below that would serve as an interface to the bottleneck, as in this model of Solarflare:

Nevertheless, it will take until 10GbE controllers are developed compact and cheap enough to be integrated into motherboards for home PCs, as in the case of Gigabit Ethernet chips, and it is safe to say that adoption among home users will only start when these plates with integrated 10GbE controllers are available in volume.

In the medium term, local networks will continue to be based on interfaces 100 and 1000 megabits and 10GbE will be used to interconnect the switches of the network, avoiding the bottleneck caused by using a single gigabit link to interconnect switches with 24 or 48 customers each one. Only much later is that we must attend to the popularization of 10GbE on desktops, possibly well after the standard 100 gigabits is already under implementation in the backbones and servers.

The 10GbE also represents the end of the hubs, repeaters and half-duplex links, which were replaced by the exclusive use of links full-duplex point to point, between the stations, switches and network routers. With that, let also be used for CSMA / CD, the collision detection system that is used from the first Ethernet standards.

Although you can still connect multiple switches in cascade, with Cat 6a cables of 100 meters each for longer distances, the idea is that you use one of the patterns of fiber optic cables when you need to achieve greater distances. With 10 km offered by the 10GBASE-LR and 40 km offered by 10GBASE-ER, cover large distances is no longer a problem.