Painful Product Packaging Points

Four “P”s I wish were less painful, doh! – that makes five now.  Oh well.

So my day job involves packaging software for mass deployment.  We use Wise Scripting to tie everything together and compile the goo into an EXE which is shot out of an Altiris canon to several thousand computers.  We do this for very small applications, and very large applications.  One of them, which I specialize in is bundling multiple Autodesk products.  Usually, this consists of AutoCAD, Raster Design, Inventor Series (including Mechanical and Mechanical Desktop), but also the little leech products like DWG TrueView, Design Review, Vault Client, as well as numerous third-party products that ride on top of, or along side of, these Autodesk products.

The joy comes when moving to Windows XP Pro 64-bit clients.  This is when things get downright stupid.

With the Autodesk 2008 suite of products, x64 was a complete afterthought.  Yes, it works, sort of, except for Raster Design, but there are a lot of places where I went bald pulling hair out dealing with registry stupidity, folder path stupidity, services and processes stupidity and so on.  You might think: Well, why not just use their “wonderful” Deployment tool to do the heavy lifting?

Well, the Deployment tool is a heroic effort and does wonders for producing a packaged installation that almost gets you to the goal line.  There are some issues it does not handle well however.  There are some issues it doesn’t even touch, which is where we have to step in and pour our own concrete to make this bridge work.

Warning: Deeply Stupid Geek Jargon Follows

For starters, you cannot push the setup.exe and the .MST and .INI parameters via SMS, System Center Configuration Manager, or Altiris because the setup.exe spawns child processes and exits before the children are done playing.  The result is that you get NO result at all.  Imagine this:  You build a package and push it out with your favorite deployment tool (SMS, ConfigMgr, Altiris, etc.) to 10,000 computers.  When you come in the next day, you look at the status results and find that all 10,000 report “Failed”, even though most (not all) will have installed just fine.  The problem is that no matter what you do to force the calling process to wait for the entire tree of sub-processes spawned by setup.exe, the setup.exe still won’t wait and once it has given birth to all it’s children, it closes the door, and goes home.  The net result is nothing.  Meanwhile, the individual installation processes (for each product and component) continue running. 

Let’s say you added a request line to force a reboot at the end of the installation, which is a VERY common situation.  But you need to capture the result codes from all of the installation goodies to ensure they all returned a zero (0) indicating successful completion.  Or also a 3010 code, indicating success, but an in-service swap needs to occur following a reboot.  What happens is setup.exe launches, then it spawns another msiexec process and maybe some other processes and then setup.exe closes, leaving those processes to finish on their own.  Age-old parameters like /SMS and START /WAIT have zero impact on this.  So your 2nd line of instruction, the reboot, is initiated and guess what?  Yep.  The client gets kicked in the face and reboots in the middle of installing one of the products.  The net result is an incomplete installation which is a complete waste of time to diagnose and repair.  You end up manually removing each one, cleaning up leftovers and doing another installation manually.

Now, the correct way to do this:

You dive inside the .INI file which the deployment tool creates and you extract from this the list, and (very important) the sequence, of each component and product it installs.  This usually breaks down into the following:

• Microsoft DirectX
• Microsoft .NET Framework
• Microsoft Visual C++ 2005 Runtime
• Microsoft Visual C++ 2008 Runtime
• Microsoft WSE 3.0
• Microsoft MSXML 6.0 SP2
• Microsoft VBA6 Runtime
• Various KB hotfixes (optionally, depending upon conditions)

In addition to these prerequisite items, you need to also be careful to stop certain processes and services (to allow DLL’s to be removed and folders and files to be deleted cleanly), and so on.

After the carpet bombing, the ground forces can finally move in.  This is when you install the base products, the various language packs, the service packs and hotfixes (and OH BOY - I will get to that in a moment), and finally, following behind to mop up things like registry keys and values, modifying shortcuts, and so on.

• Design Review + service packs + hotfixes
• Inventor Suite + service packs + hotfixes
• Mechanical Desktop / Mechanical + service packs + hotfixes
• AutoCAD + service packs + hotfixes
• Raster Design + service packs + hotfixes
• DWG TrueView + service packs + hotfixes
• Third-Party apps + service packs + hotfixes
• Replace shortcuts to add /p profile arguments, etc.

Sound like fun?  Nah.

So, what was the comment about hotfixes above?  (hold on, I’m laughing pretty hard right now and might have to change my underwear…. hold on… um.. uhhhh…. ok) I’m back.  To be fair, Autodesk is not that much different from other software vendors when it comes to releasing updates to customers.  Microsoft has made impressive strides to standardize on the Windows Installer framework and MSIEXEC services.  This usually consists of a .MSI package for installing a product, and optional .MST (aka “Transform”) files for encapsulating custom instructions and settings, and finally .MSP (aka “Patch”) files for installing hotfixes, service packs and various updates.  The problem with many vendors is that they respect the .MSI and .MST guidelines but fall off the wagon and down a hill when it comes to standardizing on an update packaging scheme.  Autodesk (like many others) releases updates as .ZIP files, .EXE files, .MSP files and even individual data files (e.g. DLL, XML, and so on).  PAINFUL to package, test and deploy consistently.  Each of those form factors incurs its own unique methodology for installing on each client.  Long story short: I really wish Autodesk would make *all* updates as .MSP or .MSI packages, nothing else.

Back to the roadshow…

So, aside from this rather linear set of challenges is another aspect: x86 or x64.  You might think that when you whip out a disk that says “64-bit” or “x64” that it follows the appropriate guidelines cleanly.  Nope.  And again, to be fair, neither does Microsoft.  But it is improving overall.  The problem with the Autodesk 2008 suite is that half the applications dump their goods under %PROGRAM_FILES% and the other half under %PROGRAM_FILES(X86)%.  The same holds true for the registry, where some go under HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall, and the rest under HKLM\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall.  The real fun comes when Wise scripting environment translates %PROGRAM_FILES% into “C:\Program Files (x86)”, but has no counterpart to “C:\Program Files” on a 64-bit client. You have to derive that explicitly using “%SYSDRIVE%:\Program Files”

I need a beer…

So, with Autodesk 2008 suite, it was a painful mess, but we sorted it out and got both the installation and uninstall packages working very well and consistently.  Now we’re on to building the 2010 suite.  At first things looked like they were almost completely cleaned up, but then comes Design Review 2010.  It still writes everything in 32-bit mode, so it’s files get placed under “C:\Program Files (x86)” and its registry keys under HKLM\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall.  But that’s not all.  Next comes a new set of “Language Packs” for each product (or most products, not all of them), even if you’re on an English-US platform (0409 codeset, en-us). 

The weirdness comes when you try to hotwire this mess in Wise Script and you call the AcadLP.msi installer (for example) and think you can use the TRANSFORMS=acad.mst (because that acad.mst) happens to be sitting in the same folder with the AcadLP.msi), but that doesn’t work.  You have to invoke the .MST in the parent folder, which is the same one used for the main acad.msi installation (which has to come first by the way).  So, because the .MSI and .MST are in different paths, you have to specify the full path in the TRANSFORMS= statement.  Be careful of the 256-character limit on execution statements.  Doh!

Case in point: UNC server path:

\ \ Server1\Packages\ADSK2010\ACAD2010\AdminImage\x64\acad\acad.msi TRANSFORMS=acad2010.mst /qb! /l*v %TEMP%\Acad2010.log

Ho ho ho!  AhhhhhH!!!  (sip…  Hoegaarden is very good beer.  And it’s raining outside).

Conclusion

I really don’t have a conclusion.  I don’t know where I was going with this as much as it felt like a cathartic dumping of frustration on a rainy day.

Holy crap one of my daughters is grilling bar-b-que chicken and it gets me more pumped up than a case of Red Bulls.  I’m serious.  My leg won’t stop twitching.  The smell….  where was I?  Oh yeah…

I don’t want to pick on Autodesk unilaterally here.  They have made enormous strides in the area of packaging and deployment automation.  Compared to what we had to do to “push out” AutoCAD R12 years ago, this is a dream.  Somewhat like bitching about how long it takes to microwave a can of soup, when it used to take 30 minutes on the stove years earlier.  But still, they need to know these pains from customers in order to plan ahead.  I hope they get this in the context it was intended, which isn’t that they make a bad product, or even a bad deployment tool, just that it needs a few improvements and it could be state of the art.  For example, if it could bundle all products into a single installation and do a proper job of encapsulating the entire collection of sub-processes, threads and forks, then it could effectively return a comprehensive result to an infrastructure management product like ConfigMgr or Altiris, etc.  That would be HUGE.  Almost as huge as the hunger I have right now for bbq chicken.  Gotta go.

Thanks!