Traffic shaping based on a port has limited use. Traffic shaping based on a traffic type in conjunction with guaranteed minimum bandwidth allocations is where we need to get so all traffic types can take advantage of a large pipe. This will allow traffic to burst when no one else is using it while allowing higher priority traffic to get bandwidth when it needs it in a congested network.

I can see using a port group based traffic shaping model for a backup connection in VMs or in a DMZ to match other connections are slower.

One quick way to see your traffic is look at esxtop or resxtop’s networking screen during vMotion and under high traffic. I don’t think you will see any where near the 18Gb+ bandwidth that dual 10GbE ports are capable of…

I still say, keep the network model as simple as possible, apply good monitoring and management tools and only implement traffic shaping and QoS if needed.

Morning! Couldn’t you do something similar using 1 distributive virtual switch and using traffic shaping. I was under the impression that in utilization of Dvs that you can do both egress and ingress traffic shaping.

I appreciate your comments.

Disclaimer: Not a network guy

Keep up the good work!
Cheers
Sri

Say for instance you need to have a group of VMs on a PVLAN and they need to live in blades which expand multiple chassis (say more than 4, which is the limit on VC stacking today). Leave aside the fact that PVLAN may or may not be supported in all three mentioned layers, and if they are supported, will certainly work in different ways. Still, you need to configure the network on all three layers, think about it. If you are on vSphere Ent, no distributed switches, means also you need to touch every vSwitch on every host, plus every Flex-10 module, plus network. Let’s add vlan limitations in Flex-10 to make it more appealing … In large scale environments it can become a nightmare. In small scale, a burden.

If you go for a solution like the one outlined above by Brad (N1K + Passthru + N5K) you can etherchannel every host to the N5K with a trunk. PVLAN required? It works the same everywhere (NX-OS) and you configure on a single point for up to 256 host: the N1Kv VSM. A lot simpler. Simple = reliable. Not to mention the savings in Ops costs.

My 0.2 cents.

Juan

Looks like you totally forgot the data center reality! That they have existing infrastructure and management tools. That they have Dell servers (or other vendor servers, really sorry to say no Cisco servers!)… and relevant management tools in place. That they have trusted their business with all these years. So get this man, to your surprise! The reality is by adding UCS, they are adding more management points, even if it is just one

So you need to get real first, and understand that with the management infrastructure that is already inplace in every data centers with Dell Servers, it is just adding Server HW. That is one step…pretty simple and provides one major thing customer really cares “Investment Protection”. In addition, ease of deployment, etc, etc….

So, don’t you agree, in fact with UCS you are just adding more complexity to existing data centers, instead of simplifying..Now customer has to worry about managing two different server platforms…

I will definitely agree that you have some value proposition, if in case you build ground up new data centers where that companies just started up…otherwise it’s just “Nuke and Pave” proposition…and I bet customer don’t agree with it…even if Cisco is giving away to UCS chassis free..

Don’t make me post this to my blog, by not posting it to yours Be fair…can u?
Cheers
Sri

All this can be done with VMware and their native vSwitches. In addition, VMware supports traffic shaping with rate limiting feature, just in case you still want to limit bandwidth per traffic/port group..So I see no need for customer to spend money on Nexus 1000V soft switch. No value proposition, similar to Flex-10.

Again, I agree that pass thru is the best option today, leaving path to FCoE support which comes with industry standards DCB based Converged Enhanced Ethernet, later part of 2010. So I would say customer should wait till on buying any convergence gear till the standard based products hit the market in by end of 2010. So that in fact means wait till Nexus 5K and UCS 6100 FI all these move to DCB standards based CEE..These products need to move on from Cisco proprietary DCE…and that will happen later in the years along with rest of the industry.

There is no point in rushing in to all these products that are based on pre-standard technologies….

Don’t you agree? If not, they may end up with expensive paper weights….
Cheers
Sri

We have been looking at network bandwidth and QoS on 10G Ethernet in our labs. While we can use benchmarking and load generation tools to get 9.5+G of throughput on a 10G port, the question that we keep asking ourselves is, will we see this in the real world?

I am sitting in a VMware vSphere Fast Track course with 20+ Admins that still run their network connections on 1G. They are running less then 10 VMs per host with 8 1G ports of connectivity. With that few of VMs and since each 1G port is limited to 1G of traffic the likelihood of a host actually being able to max out a pair of 10G ports seems pretty low.

So in most situation why would we want to spend the additional time and overhead of setting up QoS policies. I would say setup monitoring of the network and watch for bandwidth issues and only setup QoS if needed.

I put more of these thoughts in a white paper…
http://download.intel.com/support/network/sb/10gbe_vsphere_wp_final.pdf

With all that said, I believe in two 10G ports that are wide open for all traffic types to share the bandwidth, UNLESS, you have a real need to manage your bandwidth of a specific traffic type. Having an Admin manually carve a 10G in to 4 segments that don’t share unused bandwidth doesn’t seem to match the new paradigm of dynamically changing virtualized data centers.

I look forward to reading about the areas / use cases that QoS is really needed and is more efficient then just adding another 10G port to the host. The reality is 10G is not that expensive especially when compared to paying for the additional operational expenses to implement and manage QoS on every host in a data center.

If you need storage over ethernet and don’t yet have support for fcoe in your storage or server products take a look at a recent VMware document on Exchange performance using iSCSI and NFS – http://blogs.vmware.com/performance/2009/07/exchange-performs-well-using-fibre-channel-iscsi-and-nfs-on-vsphere.html

The testing used generic Intel nics vs dedicated fiber HBAs and showed iSCSI and NFS ( our interest is iSCSI which unlike FC CAN be port channeled and run over distance greater than FCOE distance limit of 300m) to be very close in performance. If the testing had used hardware iSCSI adapters it likely would have beaten native Fiber Channel.

From your postings … the N5K & FEXes don’t seem to be candidates your environment – a couple of 3750Gs would probably be ideal and they could handle limited ethernet over IP in whatever fashion you needed now.

What happens if a company needs more than 320 physical server, say 321 do they have to run another instance of UCSM and manage that separately…

Let’s say in the case of HP or Dell a blade chassis holds 16 servers. So for 320 servers that would be 20 chassis. Each of the 20 chassis has Ethernet, Fibre Channel, and iLO management, so that’s 3 points of management per chassis. 20 chassis with 3 points of management is 60 management points for 320 servers.

With Cisco UCS there is no individual per chassis management points, you only need to manage the single Fabric Interconnect. Furthermore, the Ethernet, FC, and iLO is all managed in a single pane of glass … 1 management point. So for 320 servers thats 1 point of management in Cisco UCS, and 60 points of management for HP/IBM/Dell.

Ok, 321 servers? That would be 2 points of management in Cisco UCS, and 63 for HP/IBM/Dell.

Why stop at 321? How about 640 servers? 2 points of management in Cisco UCS, 120 with HP or DELL.

You get the idea.

Cheers,
Brad

Thanks for joining the conversation. You said:

… we’ll ignore the fact that the only thing you can run at the moment is virtualised workloads (and only in VMware)

Well, I won’t ignore the fact that you need to do your homework on Cisco UCS because you are dead wrong here. Our customers can run VMware, Citrix, Hyper-V, Windows, Solaris x86, and Linux workloads on UCS.

the problem is not as simple as “how many IP’s are consumed”

If you think the management capabilities of UCS are simply reducing the IP addresses to keep track of, again, you need to do some homework. You mentioned turning up new services quickly, lets work with that scenario…

Here’s one simple example of many: Imagine a new application to be deployed that requires a new VLAN and Firmware update pushed to all of the adapters on the perhaps hundreds of servers that will be supporting that application. The Cisco UCS customer simply logs into the Fabric Interconnect and with a few clicks of the mouse updates a single Service Profile template with the new VLAN settings and Firmware bundle. -DONE- The system goes out to the hundreds of servers driven by that template and updates the Firmware and VLAN settings. Now that’s time to market.

This time it was VLANs and Firmware, the next time it might be QoS or BIOS settings. A process that would otherwise take weeks, several different management platforms, and team of people is all done in a few minutes after a few clicks of the mouse. All of this with the out-of-the-box capabilities in UCS Manager. This process can also be driven via the XML API integration with 3rd party automation players such as BMC BladeLogic or EMC Ionix, for example.

I could go on with 10 other similar examples, but you get the idea, right?

Do you still think UCS is just solving “pain points” only in the network?

Cheers,
Brad