Welcome!

@CloudExpo Authors: Yeshim Deniz, Elizabeth White, Pat Romanski, Liz McMillan, Aruna Ravichandran

Related Topics: Microsoft Cloud, Open Source Cloud, Machine Learning , Silverlight, Release Management , CRM

Microsoft Cloud: Blog Feed Post

Performance Tuning Windows 2012: Network Subsystem | Part 1

NDIS, the protocol stack, and user mode applications

Offload Capabilities
Offloading tasks can reduce CPU usage on the server, which improves the overall system performance. The network stack in Windows 2012 (and prior versions of the OS) can offload one or more tasks to a network adapter permitted you have an adapter with offload capabilities. The table below lists the details about offload capabilities:

Receive-side scaling (RSS) is a network driver technology that enables the efficient distribution of network receive processing across multiple CPUs in multiprocessor systems.

Checksum calculation

The network stack can offload the calculation and validation of Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) checksums on send and receive code paths. It can also offload the calculation and validation of IPv4 and IPv6 checksums on send and receive code paths.

IP security authentication and encryption

The TCP/IP transport layer can offload the calculation and validation of encrypted checksums for authentication headers and Encapsulating Security Payloads (ESPs). The TCP/IP transport layer can also offload the encryption and decryption of ESPs.

Segmentation of large TCP packets

The TCP/IP transport layer supports Large Send Offload v2 (LSOv2). With LSOv2, the TCP/IP transport layer can offload the segmentation of large TCP packets to the hardware.

Receive Segment Coalescing (RSC)

RSC is the ability to group packets together to minimize the header processing that is necessary for the host to perform. A maximum of 64 KB of received payload can be coalesced into a single larger packet for processing.

Receive-Side Scaling (RSS)

Receive-Side Scaling (RSS)
Windows Server 2012 (as well as Windows Server 2008 R2, and Windows Server 2008) supports Receive Side Scaling (RSS). RSS directs network processing to up to one logical processor per core. For example, given a server with Intel Hyper-Threading and 4 cores (8 logical processors), RSS will use no more than 4 logical processors for network processing.
RSS distributes incoming network I/O packets among logical processors so that packets that belong to the same TCP connection are processed on the same logical processor. RSS also load balances UDP unicast and multicast traffic from Windows Server 2012, and it routes related flows (as determined by hashing the source and destination addresses) to the same logical processor, preserving the order of related arrivals. Windows Server 2012 provides the following methods to tune RSS behavior:

· Windows PowerShell: Get-NetAdapterRSS, Set-NetAdapterRSS, Enable-NetAdapterRss, Disable-NetAdapterRss. These cmdlets allow you to view and modify RSS parameters.

· RSS Profiles: Used to determine which logical processors are assigned to which network adapter. Possible profiles are:

o Closest. Logical processor numbers near the network adapter’s base RSS processor are preferred. Windows may rebalance logical processors dynamically based on load.

o ClosestStatic. Logical processor numbers near the network adapter’s base RSS processor are preferred. Windows will not rebalance logical processors dynamically based on load.

o NUMA. Logical processor numbers will tend to be selected on different NUMA nodes to distribute the load. Windows may rebalance logical processors dynamically based on load.

o NUMAStatic. This is the default profile. Logical processor numbers will tend to be selected on different NUMA nodes to distribute the load. Windows will not rebalance logical processors dynamically based on load.

o Conservative: RSS uses as few processors as possible to sustain the load. This option helps reduce the number of interrupts.

You can use the set-netadapterRSS cmdlet to choose how many logical processors can be used for RSS on a per-network adapter basis, the starting offset for the range of logical processors, and which node the network adapter allocates memory from:

· MaxProcessors: Sets the maximum number of RSS processors to be used, ensuring application traffic is bound to a maximum number of processors on an interface.

set-netadapterRSS –Name “Ethernet” –MaxProcessors <value>

· BaseProcessorGroup: Sets the base processor group of a NUMA node, affecting the processor array used by RSS.

set-netadapterRSS –Name “Ethernet” –BaseProcessorGroup <value>

· MaxProcessorGroup: Sets the Max processor group of a NUMA node, affecting the processor array used by RSS.

set-netadapterRSS –Name “Ethernet” –MaxProcessorGroup <value>

· BaseProcessorNumber: Sets the base processor number of a NUMA node, allowing partitioning processors across network adapters and specifying the first logical processor in the range of RSS processors that is assigned to each adapter.

set-netadapterRSS –Name “Ethernet” –BaseProcessorNumber <Byte Value>

· NumaNode: The NUMA node that each network adapter can allocate memory from.

set-netadapterRSS –Name “Ethernet” –NumaNodeID <value>

· NumberofReceiveQueues: If your logical processors seem to be underutilized for receive traffic, you can try increasing the number of RSS queues from the default of 2 to the maximum number supported.

set-netadapterRSS –Name “Ethernet” –NumberOfReceiveQueues <value>

RSS does not provide any interaction with virtual machines, instead you can configure VMQ. RSS can be enabled for virtual machines in the case of SR-IOV because the virtual function driver supports RSS capability. In this case, the guest and the host will have the benefit of RSS. The host however, does not get RSS capability because the virtual switch is enabled with SR-IOV.

Receive-Segment Coalescing (RSC)
Receive Segment Coalescing can improve performance by reducing the number of IP headers that are processed for a given amount of received data.  You should use RSC to tune performance of received data by grouping (or coalescing) smaller packets into larger units. This can reduce latency and increase throughput for received heavy workloads. On network adapters supporting RSC, make sure that it is enabled, unless you have low latency, low throughput networking needs that benefit from RSC being turned off.

In Windows Server 2012 you can use the following PowerShell cmdlets to configure RSC capable adapters: Enable-NetAdapterRsc, Disable-NetRsc, Get-NetAdapterAdvancedProperty, and Set-NetAdapterAdvancedProperty. RSC can be examined using the cmdlets Get-NetAdapterRSC and Get-NetAdapterStatistics. The Get cmdlet shows if RSC is enabled and if TCP enables RSC to be in operational state. In the example above, IPv4 RSC is enabled. To understand failures, you can view the coalesced bytes or exceptions caused by entering the following command:

PS C:\Users\Administrator> $x = Get-NetAdapterStatistics “myAdapter”

PS C:\Users\Administrator> $x.rscstatistics

CoalescedBytes : 0

CoalescedPackets : 0

CoalescingEvents : 0

CoalescingExceptions : 0

RSC and virtualization
If the host adapter is not bound to the virtual switch, RSC is supported on the physical host. If the adapter is bound to the virtual switch, Windows 2012 will disable RSC on the physical host.
RSC can be enabled for a virtual machine when SR-IOV is enabled. In this case, virtual functions will support RSC capability; hence, virtual machines will also get the benefit of RSC.

Network Adapter Resources

A few network adapters actively manage their resources to achieve optimum performance. Several network adapters let the administrator manually configure resources by using the Advanced Networking tab for the adapter. For such adapters, you can set the values of a number of parameters including the number of receive buffers and send buffers.  In Windows Server 2012, you can configure advanced network settings using the following PowerShell cmdlets:

  • Get-NetAdapterAdvancedProperty
  • SetNetAdapterAdvancedProperty
  • Enable-NetAdapter
  • Enable-NetAdapterBinding
  • Enable-NetAdapterChecksumOffload
  • Enable-NetAdapterLso
  • Enable-NetAdapterIPSecOffload
  • Enable-NetAdapterPowerManagemetn
  • Enable-NetAdapterQos
  • Enable-NetAdapterRDMA
  • Enable-NetAdapter
  • Enable-NetAdapterSriov

Message-Signaled Interrupts (MSI/MSI-X)
Network adapters that support MSI/MSI-X can target specific logical processors. If your network adapter also support RSS, then a logical processor can be dedicated to servicing interrupts and deferred procedure calls (DPCs) for a given TCP connection. This will greatly improve performance, by preserving the TCP cache.

Interrupt Moderation
Lastly, we’ll discuss interrupt moderation. Some network adapters expose different interrupt moderation levels, or buffer coalescing parameters, or both. You definitely should consider buffer coalescing when the network adapter does not perform interrupt moderation. Interrupt moderation will reduce CPU utilization because it minimizes the per-buffer processing cost, but you should consider that interrupt-moderation  and buffer coalescing can have a negative impact on latency-sensitive situations. The table below lists the suggested adapter features for various server roles.

Role

Checksum offload

Large Send Offload (LSO)

Receive-side scaling (RSS)

Receive Segment Coalescing (RSC)

File server

X

X

X

X

Web server

X

X

X

Mail server (short-lived connections)

X

X

Database server

X

X

X

FTP server

X

X

X

Media server

X

X

X

These settings serve as guidelines only . Depending on the workload, your network adapter(s), and your specific situation, your experience can be different. In our next article we’ll go deeper into tuning the network adapter and utilizing some of the features we discussed.

Read the original blog entry...

More Stories By Hovhannes Avoyan

Hovhannes Avoyan is the CEO of PicsArt, Inc.,

@CloudExpo Stories
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
SYS-CON Events announced today that TidalScale, a leading provider of systems and services, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale has been involved in shaping the computing landscape. They've designed, developed and deployed some of the most important and successful systems and services in the history of the computing industry - internet, Ethernet, operating s...
Infoblox delivers Actionable Network Intelligence to enterprise, government, and service provider customers around the world. They are the industry leader in DNS, DHCP, and IP address management, the category known as DDI. We empower thousands of organizations to control and secure their networks from the core-enabling them to increase efficiency and visibility, improve customer service, and meet compliance requirements.
In his session at 21st Cloud Expo, Michael Burley, a Senior Business Development Executive in IT Services at NetApp, will describe how NetApp designed a three-year program of work to migrate 25PB of a major telco's enterprise data to a new STaaS platform, and then secured a long-term contract to manage and operate the platform. This significant program blended the best of NetApp’s solutions and services capabilities to enable this telco’s successful adoption of private cloud storage and launchi...
Data scientists must access high-performance computing resources across a wide-area network. To achieve cloud-based HPC visualization, researchers must transfer datasets and visualization results efficiently. HPC clusters now compute GPU-accelerated visualization in the cloud cluster. To efficiently display results remotely, a high-performance, low-latency protocol transfers the display from the cluster to a remote desktop. Further, tools to easily mount remote datasets and efficiently transfer...
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant tha...
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It’s clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Tha...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, will lead you through the exciting evolution of the cloud. He'll look at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering ...
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere delivers a more modern architectural approach to storage that doesn't require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbui...
In his general session at 21st Cloud Expo, Greg Dumas, Calligo’s Vice President and G.M. of US operations, will go over the new Global Data Protection Regulation and how Calligo can help business stay compliant in digitally globalized world. Greg Dumas is Calligo's Vice President and G.M. of US operations. Calligo is an established service provider that provides an innovative platform for trusted cloud solutions. Calligo’s customers are typically most concerned about GDPR compliance, applicatio...
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their ass...
As hybrid cloud becomes the de-facto standard mode of operation for most enterprises, new challenges arise on how to efficiently and economically share data across environments. In his session at 21st Cloud Expo, Dr. Allon Cohen, VP of Product at Elastifile, will explore new techniques and best practices that help enterprise IT benefit from the advantages of hybrid cloud environments by enabling data availability for both legacy enterprise and cloud-native mission critical applications. By rev...
Companies are harnessing data in ways we once associated with science fiction. Analysts have access to a plethora of visualization and reporting tools, but considering the vast amount of data businesses collect and limitations of CPUs, end users are forced to design their structures and systems with limitations. Until now. As the cloud toolkit to analyze data has evolved, GPUs have stepped in to massively parallel SQL, visualization and machine learning.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant th...
Digital transformation is changing the face of business. The IDC predicts that enterprises will commit to a massive new scale of digital transformation, to stake out leadership positions in the "digital transformation economy." Accordingly, attendees at the upcoming Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA, Oct 31-Nov 2, will find fresh new content in a new track called Enterprise Cloud & Digital Transformation.
The dynamic nature of the cloud means that change is a constant when it comes to modern cloud-based infrastructure. Delivering modern applications to end users, therefore, is a constantly shifting challenge. Delivery automation helps IT Ops teams ensure that apps are providing an optimal end user experience over hybrid-cloud and multi-cloud environments, no matter what the current state of the infrastructure is. To employ a delivery automation strategy that reflects your business rules, making r...