One advantage of being a "consultant" is getting to peek into a variety of environments, cultures, methodologies, and whatnot. One thing I see fairly often that seems to warrant some mention is slow performance over network pathways. A cursory search on Google reveals that there are plenty of articles, blog posts and tweets regarding slow "deployment creation", it's tough to find anything really focusing on the execution side: deploying the installation from the share to the clients.
The potential points of trouble or failure in a typical LAN or WAN environment are numerous. From client hardware and software, to client configuration settings, to physical wiring, switches, hubs, routers, wireless equipment, wireless configuration settings, server hardware, server software, server configuration settings, scheduled backups, scheduled anti-virus scans, unpredicted end-user activity, power faults, wireless interference, and layer on top of all this the issues incurred in virtual data center environments. These are some of the aspects and any one of which (or combination of several) that can cause performance drag or outright failure. The more you dig into the world of network engineering, the more you can appreciate what a network engineer deals with.
So, if you're Autodesk product network deployments are taking longer than they should to deploy, here are some things to look at:
Network Site Link Speeds
If your employer is large enough to have a dedicated "network admin", consult him/her for link speeds between points of deployment (server shares) and end users (desktop and laptop computers). Find out how many hops are involved. What the switches and routers are like. What limitations they're aware of. I recommend doing this BEFORE you even attempt to build network deployment shares (referenced in my book. shameless plug, I know) Basically, you always want to keep as little time, distance and latency as possible between the installation source and the installation target. This is irrespective of doing it manually, via scripts, group policy or management products like Configuration Manager.
NIC Configuration Settings
Verify the Network Adapter configuration settings with your network administrator. I've seen plenty of situations where everyone assumed it was automatically set to the correct configuration, when it was anything but. Sometimes atypical network equipment configurations require atypical client configurations to suit. Even if you are absolutely, positively convinced that all of your clients are using identical and correct network settings, check them again, just to be sure.
Concurrent Utilitization: Server
If your deployment share resides on a file server, or a print server, or an application server (web, database, etc.), I would STRONGLY recommend you consult your server administrators to setup some performance monitoring to measure and verify the loads placed on each server. If one role or function is hogging the resources (CPU time, memory, disk I/O, NIC throughput, etc.) consider moving it to another server.
If concurrent roles or functions are not an issue, check to see when system and file backups are scheduled to run. Do your best to avoid scheduling your network deployments during system backups, anti-virus scans, or scheduled maintenance. If your servers are virtualized, and your server admins ocassionally move them from one physical host to another (or SAN storage attachments, etc.) it's important to coordinate your activities to avoid problems.
Talk to your department coordinators or "power users" also. Make sure they don't have any tasks they run during certain hours to back-up their project files, or perform operations that directly tax the server or its resources. Copying a ton of large files can greatly impact network performance if the server is not configured to address that usage ahead of time.
Always always always check the event logs for any sign of potential issues.
Concurrent Utilitization: Network
If the server is not being taxed, you should take a careful look at what's going over the network segments. Between the clients and the switch or gateway. Between the switch or gateway and other routers, and between the other routers back to the servers. There are lots of ways to do this and it's important to consult and coordinate with your network admins to do this effectively. In many cases they can help identify bottlenecks caused by two major traffic activities that can be separated to avoid conflict.
Client Integrity
Check for available/free disk space on the target hard drive volume. Check for errors and warnings in the client event logs. Is the client getting frequent anti-virus quarantine events? Are installed applications causing problems? Be careful of applications that want to continue running in the background, especially when the client is tight on physical memory. If the computer is being used heavily, consider upgrading the hardware or installing a separate physical hard drive to isolate I/O tasking. The list here goes on and on, of course.
Client Activity
Are users performing CPU or disk-intensive activities? Rendering large models, video processing, audio editing, etc.
More
Oh yes. There's more. MUCH more. I haven't (and won't) go into the impacts of other key networking services like DNS, Kerberos, WINS, DHCP, encryption overhead, certificates, dual network interfaces, teaming, virtual switches, virtual NICs, or any of that stuff. It's more than I feel up to blabbering about, but it's out there, waiting for your brain to absorb it all.
And Finally...
Before you jump into building your network deployment, especially if you're aiming for multiple servers across a WAN, it's important to gather as much information as possible about your network so you can plan your strategy correctly. If you spent hours convincing your coworkers, management and customers/users that network deployments are the "way to go", don't you want to make sure it works as great as it possibly can?
Consult your network and server administrators. Run tests to verify file copy speeds across all links you plan on using. Run your tests over a month or at least a few weeks, at all times of day, all days of the week. Record details of performance and identify patterns in best and worst results. Pointing a finger at one of your network admins and blaming them for slow speeds is only going to piss them off and get you moved to the bottom of their list of things to take care of. Having measurement data and engaging in a cooperative discussion will get you where you want to go with the least amount of pain and effort. Be sure to bring donuts and tell lots of jokes too. It never hurts.
Tools and Links
WindowsNetworking.com - Troubleshooting Network Issues
Advanced Network Adapter Troubleshooting for Windows Workstations
WireShark (the de facto diagnostic tool)
Network Monitor 3.4 for Windows 7, Vista, Server 2003, 2008, 2008 R2
Sysinternals Networking Utilities
Sysinternals Process Utilities
Books by Mark Minasi at Amazon
No comments:
Post a Comment