I occasionally have VMware customers ask about jumbo frames, most often in reference to their iSCSI SANs. I attribute the curiosity mostly to VMware now officially supporting jumbo frames for iSCSI and NFS in vSphere. So with vSphere supporting it, and if your NICs, switches and SAN support it, should you use it?
Maybe. It depends on your goal, your equipment, and your workloads.
At the basic byte-counting level, you get 1460 bytes (1500 byte payload minus 20 bytes of IP and 20 bytes of [minimal] TCP) of iSCSI payload out of a 1538 byte iSCSI frame (1518 byte frame plus 8 bytes of preamble/SOF and 12 bytes of interframe gap), which is 94.93% efficient. With 9000 byte jumbo frames, your iSCSI payload is 8960 bytes out of a 9038 byte frame, or 99.14% efficient, a 4.21% gain. Not huge, but not insignificant.
However, the biggest potential gains of jumbo frames is not on the wire efficiency side, but on the host processing side. Each packet requires checksum calculations and protocol computation overheads. If your main CPU is doing these calculations, the conversion to jumbo frames can provide some CPU relief. TOE (TCP Offload Engine) functionality in your NICs (with O/S support) shifts some or all of those calculations to the NIC hardware, and TSO (TCP Segmentation Offloading) and LRO (Large Receive Offload) support can also bring CPU relief. An important note is that VMware does not currently support most TOE functionality, but does provide support for TSO, and support for LRO for a very limited set of NICs (which are conveniently the Intel VT and Broadcom 5708/5709 cards commonly found in Dell servers). See KB 1006143.
But the single most important question to ask yourself before going down the jumbo frames road is this: Am I intending to push my servers and storage so hard that I need the gains from jumbo frames? I support a fairly large number of setups, and I’m rarely designing anywhere near the edge of equipment performance. I like comfortable performance buffers, and so do my customers. I also like having excess performance capacity for growth. If you’re at the performance edge, perhaps the more appropriate question is whether your core design and core equipment are the right choices. Perhaps you need to step them up a notch. If you’re deeply concerned about the CPU load from software iSCSI, buy hardware iSCSI HBAs (but be sure to verify them against the VMware hardware compatibility list!). Also note that some new NICs are being listed as “NIC w/TOE iSCSI”, but these are not the same as a full iSCSI HBA, and you will not get the hardware offloading of iSCSI that you desire.
The best community posting I’ve found on iSCSI with vSphere:
I have a habit of saving and forwarding many of the ridiculous voicemails that salespeople leave in my inbox, but it isn’t too often I receive a letter of this quality, pointing to the problems that salespeople face when introduced to social networking.
The beauty of social networking is that I can typically gain value from the people I am connected to, before I am asked to buy something.. The danger comes when we decide to sell before providing any real value. Gary Vanerchuk said at Big Omaha, “we are like 18 year old boys trying to close too fast.” –This is why salespeople ruin social media.
In sales, people are told that they can reach a broad audience through social media and think they can reach this audience without working at it. To provide real value to the people you connect with takes real work. Social media doesn’t eliminate working hard, in magnifies it. We now have more messages and more people, but each person seeks information. Our messages are read much quicker and distributed to a larger audience, making it more important to respond.
I’m waiting for the realtor who wants to answer my questions online and tell me about market conditions, helping me to learn their expertise. Teach me about how I can use my home and rental property better. If I met you at a social gathering, you probably wouldn’t ask me to list my house immediately, you would probably find out my needs and I would describe my situation.
Finally, to this realtor’s credit, I do admire his personal touch in a handwritten letter but I think his effectiveness will improve if he provides content to his network and improves his penmanship.
After 9 seasons of the Biggest Loser, NBC continues to attract a steady audience of couch-dwellers intrigued by the success of obese contestants training as though they were professional athletes. I will admit to being a fan of this phenomenon, but find myself wondering why?
I look into the IT industry and realize we wage a similar war to each of these contestants. in order to take control of the health of IT networks we must understand the affect of small methodical issues. It doesn’t take eating one greasy meal to become morbidly obese, but eating a little extra at every meal has huge implications. The worst networks I see were overlooked daily over time; patches were missed, upgrades were delayed, hardware sits in boxes and software is left un-deployed.
Very few of us operate like Jillian Michaels to motivate ourselves proactively when we are already in good health. But we know that’s what we want.
So what changes?
What we know is that each network environment we manage requires a level of discipline. If you are a manager of IT professionals, you know this all too well. Each component needs to be prioritized and scheduled for ongoing maintenance and checks and balances need to be in place to insure the reliability of these actions, even when automated.
If you are a single IT professional, you face the most difficult of self-disciplined tasks; ensuring the health of your network where there are no professionals around. That is, there are no doctors observing the missteps or checking cholesterol, the only observers are looking for dramatic symptoms, not habits.
Now is the time to get off the couch and weigh-in on your environment and be honest with yourself about what needs to be done each day. And if you want to hear about massive transformations, I have a few stories that would keep Jillian awake at night.
Thursday’s landline outage for much of Southeast Nebraska included the city of Lincoln’s 911 service. After 935 minutes, from 7:45 AM to 11:20 PM, media outlets published alternative emergency numbers. As an IT guy for a high performance comapany, I was shocked to see this happen in several of the communities that I serve. Landline service is usually regarded as the gold standard for Five Nines service.
Due to this recent outage it will take 181 years of perfect service to reach 99.999% uptime on this system. Ironically Windstream’s website promotes Four Nines of reliability for home service with the Your Landline is your Lifeline campaign. But surely 911 service is not Four Nines, with almost an hour a year of downtime?
Our goal isn’t to be hard on a local phone company, but since many of our boardroom ideas come from enterprise server rooms, we want to see what businesses can learn.
First, it appeared as though both the primary and redundant switch were modified and then upgraded. Whether or not this was the ultimate issue, only 1 technician can tell us that, but we do know that whenever upgrading a system, we need to verify a backup (yes good switches can have backup configurations) and verify the primary before modifying the secondary.
Secondly, whenever your system demands a high level of reliability, you want to have options, whether that is rolling to an alternative site or a different telecommunications company, many small businesses are implementing backup Internet connections to keep communications a priority.
Lastly, I’m sure that in this case these were enterprise switches, but often even in larger deployments, business decision-makers overlook the impact that switches play in a business environment. Whether you are looking to prioritize traffic for a VOIP deployment or just have a more stable network, switches and other network infrastructure are key to keeping your environment stable.
