The Never-Ending War: Centralized IT vs Department Developers

I doubt any of you will read this one to the end, and I can't blame you.  But I have to say that this is probably one of my best articles so far.  It's one that I'm fairly passionate about and it shows in my verbosity.  I apologize for torturing your eyeballs in advance, but I must go on.

For many of you, this scenario should sound pretty familiar:

A large corporate business evolves over decades to the point where they decide it makes sense to centralize their IT operations.  They create a new department and appoint a CTO or CIO and staff the various functions.  Prior to this, it was common to find each department building and maintaining their own unique software applications to solve their own internal business needs, aka "line of business" applications ("LOB" for short).  In decades past, it was dBase or FoxPro.  In more recent times it's been Microsoft Access or Excel.  Quite often, it involves VBA code.  Lots and lots of VBA code.

The centralized IT department strives to gather all of the LOB applications and apply some centralized procedures to maintaining it.  Maybe the company is trying to be ISO or CMMI compliant, or maybe ITIL, or whatever.  Who knows.  In any case, the MBA mindset comes to the logical conclusion that redudant job roles are expensive and inefficient, especially when one person in Finance is building roughly the same application as the person in Sales.  These isolated, parallel and typically redundant efforts have no incentive or desire to coordinate efforts with other departments due to politics and funding, so they march on unrestricted for years.

When IT finally gains control of a few of the key LOB applications, they apply common ITIL or CMMI procedures on change control, which invariably slows down the process of implementing updates.  The LOB folks quickly grow tired of this, and eventually pull away from IT to continue building their own applications.  The cycle continues.

This is often referred to as a "NO-WIN SITUATION", because that's exactly what it is.

The top management folks are left with two options:

• Ignore it and leave the fighting to the lower level managers
• Stress out trying to make a decision over which is the lesser evil: inefficiency or inefficiency.

The two greatest risks facing them are:

1. Allowing the redundant waste of time and resources to grow unchecked (because it's NEVER really checked) and allow inconsistent results to spread.  The practice increasingly melds the company core business processes to hundreds of tiny little tools built by people that have worked themselves into a position of necessity (cant' fire them or you risk being stuck with a broken tool and a busted business process)
2. Forcing the LOB apps under the control of the IT department, becoming painfully slow at responding to surge-capacity, and frequent requirements changes.  Everything must now be vetted, developed, tested and implemented under the new procedures.  No more "walk-up" requests.  No more Summer intern kid-turned-full-time-developer to handle your needs.  LOB requests now get routed through a load-sharing process to "resources" which share their precious time with other (possibly competing) LOB requests.

Ugliness.  Bad feelings.  Animosity.  All that kind of stuff.

This is one of the reasons I F-ING HATE MICROSOFT ACCESS

The product itself is not bad or evil.  Fire isn't bad, when used properly.  Each serves a noble purpose.  But fires created by a small group of trained professionals is one thing.  Handing every person in a 10 mile radius a can of gasoline and a box of matches is a little different.  Server backended applications are like the trained professionals.  Microsoft Access is like a truckload of gasoline cans and matches being handed out like rice bags from a UN truck in Rwanda.

The most common scenario today involves this vicious cycle of stupidity:

• The Sales department asks a promising Summer intern to build them an uber sales tracking app.  Uber-brainy intern cracks open a "Access VBA Unleashed" book, and builds the new application in MS Access 2000 using lots of VBA.
• Years later, the company decides to take advantage of volume licensing and buys into a SELECT or ENTERPRISE AGREEMENT contract to get upgrades more cheaply. The Company recieves licensing for Office 2010 and wants to upgrade all computers ASAP.
• The Sales department cries that Access 2010 will break their LOB app.  They cry to upper management. IT is ordered to put the upgrade plans on hold.
• The IT department now decides on one of two courses of action:
  • Dedicate several developers to re-developing the application to work in Access 2010, but they don't understand the LOB requirements or logic, so they have to work closely with the Sales department to gain better understanding.  They also have to decide how to address deprecated functionality (from Access/VBA 2000 to 2010), and incorporate new features, or if there is even enough time to do that, or just patch it up and make it work.
  • Dedicate several developers to shoehorn the old application to work via the Access 2000 runtime engine, if that even works (sometimes it does, sometimes it doesn't).  This often becomes a mess, with file associations getting corrupted and interaction issues with other third-party applications.
• The plan to deploy Office 2010 is now sidelined until they can adequately resolve this dilemma.
• By the time a solution is reached, Office 2012 is released and the process repeats

Either way, it ends up being a long, drawn-out waste of time, money and effort.  Sure, it can be argued that it's not a waste because it's provides (hopefully) a workable solution.  But if the entire problem COULD have been avoided by adopting proven practices, isn't that ultimately a waste?

A database is a database.  It should be used as a database, not as an end-user application platform.  Client/Server or web-based interfaces would easily decouple the functionality from the data store.  Then if the only thing that changes are the database connection parameters, you don't have to rewrite the entire application.  In many mature development environments, this is considered programming 101.

You probably won't believe me, but I actually do get sick of hearing myself talk and get queazy reading my own drivel.  But painful as it may be, I must go here...

Over-Caffeinated Rant (deep inhale, and....)

I haven't really delved into the virtues of decoupled business logic in n-tier applications architecture.  There's an enormously powerful incentive to following this time-tested and well-proven approach, which is along the same lines of logic as multi-stage code compilation, assembly line production, and zone defense in team sports.  That is, that compartmentalizing certain logical blocks of functionality make those blocks portable and flexible.  It makes them more easily (and more affordably) adaptable to future changes.  You know, those future changes that are quite often unforeseeable and unpredictable?  It also tends to make those newly defined blocks more efficient, both in construction and execution performance.

But things like n-tier business logic and decoupling are rarely familiar terms to the Summer intern who happens to dig tinkering with VBA (or VSTO) within Office applications like Excel and Access.  To them, it's just fun to code, and who can blame them?  How do you effectively convince this wunderkind teenager that "fun" isn't as "cool" as decoupling your code and applying standard design prinicipals to it before ever punching a key or clicking a mouse?   It's a serious conundrum for businesses of all sizes, but particularly insidiuos and debilitating for larger corporate behemoths due to their scales of inefficiency at all levels.  Who is the best equipped, and motivated to insert themselves into this vicious chain reaction and disrupt it entirely: IT, LOB stake-owners, or senior management?  IT rarely has the global authority to stomp out such wildfires alone.  LOB stake-owners rarely have the global incentive.  Senior management doesn't give a shit because that's why they hired mid-level management to insulate them from.  When LOB stake-owners are coupled with mid-level management (often one and the same), it's a done deal: nothing is going to improve.  This is the most common scenario in modern corporations today.  A Catch-22.

My coffee just ran out.