Choosing the Right Data Repository

All you mental freaks out there likely saw "repository" and thought "suppository", but that's ok.  Settle down.  It's not the weekend yet.

This article cooked up in my thimble-sized brain after a string of events at my office.  The issues all circled around having islands of data sitting around in various places, inside of various "containers" and in various formats and structures.  The data in one particular case involves inventory.  But inventory is a big, fat, drippy, soggy bag of goo that has different meanings and different values to a lot of people circled around it like kids at a campfire.  Some want numbers.  Some want names and labels.  Some want locations.  Some want state values (indicators of current status or changes of state).  Some want derivatives, like contract data, support metrics, and other secondary and tertiary cost drivers.

If you put a microphone up to each of these people and ask them where all this data should reside, you will very likely hear a wide variety of answers:

• Spreadsheets
• Database Tables
• XML files
• Visio Drawings
• AutoCAD Drawings
• Compound Documents (Word documents with embedded spreadsheets, etc.)
• Web-enabled reports (web services providers)
• Paper print-outs

And the list goes on and on.

So... who is correct?

As it turns out, when you're in the midst of sifting through what bucket fits the most needs of the most people, it's difficult to see the end of the path in the fog of new requirements.  Until you've met with everyone involved, and discussed what they (a) think they need and (b) what they REALLY need, it's near impossible to avoid choosing the wrong container.  I've been in this business for over twenty-five years and I've seen the smartest MBA, PhD, and senior brains screw it up.  It's hard not to.  The reasons are a-plenty...

1. You can't predict future needs
2. You can't predict future project priorties
3. Sizing the storage requirements is challenging
4. Placing the storage provider is challenging
5. Avoiding proprietary "lock-in" can be challenging

In general, here's what I usually try to follow...

1. Strive for version independence.  Avoid storing in a product where version changes often cause upgrade issues for consumer applications and services.  ADO and ADO.NET (ODBC, OLEDB) are fairly version-agnostic.  Meaning that if you move the tables from SQL 2008 R2 to SQL 2012 there's much less chance of having to recode applications to change the connection endpoint compared with storing in Microsoft Access, Excel or even AutoCAD drawings.
2. Find a "platform-neutral" provider medium.  Client applications are always off the table unless there isn't a choice.  It has to be a "server" application or service.  It has to provide robust access to everyone that needs it, with the least overhead, and the least performance drag.  If every consumer of the data has to have a proprietary application installed in order to access it, you're taking a risky, and often costly direction.  This is why databases are most often preferred, since they offer up access in the most neutral ways and the widest range of applications and uses.
3. Take your time!  Don't let anxious users pressure you into making a quick decision rather than a correct decision.  Just because the floor plan folks want everything in AutoCAD in time to meet their looming deadline doesn't mean that a month later, five times as many users will scream for access to the data via a web services provider medium, or an XML table.
4. Try to leverage what you already have.  If you have Oracle, use it.  If you have SQL Server, use it.  If you have an intranet server with web services enabled, use it.  Try to not build new bridges when others exist unless you can't possibly make do with what you have. Reinventing the wheel is dumb.
5. Enlist an objective team.   Gather folks from your database group, your applications development group, management group, finance group, any group that might even be remotely impacted by what you're trying to accomplish.  You'd be amazed what you can learn from people you didn't think had any interest or involvement in what you'r doing.  Many times they've been down the same road before and can offer valuable insight and advice.  Take it!  Learn from it.  It's ok to meet with the direct consumers, but if you surround yourself with only them, you will be going forward semi-blind.
6. Get metrics!  Don't ever base a purchase decision or a project direction on input from test users that don't back it up with hard numbers.  It's not that you can't trust them (sometimes you can't), but it's that without numbers, there's no way to objectively compare and analysis what's going on.  For example:  I was involved with a project to establish a central data repository for a variety of applications over assorted quality WAN links.  Three of the locations complained about performance problems and pushed HARD to go with another alternative.  It turned out to be a network link configuration problem at the router, and once it was addressed, the performance was perfectly fine.  Had this not been investigated, the alternative project direction would have delayed the entire project for four more months and added 50% more cost.  Metrics are king.

Even if the majority of the consumer-end users are clamoring for going with a platform that fits squarely within their operational environment, try to avoid a knee-jerk response.  I've seen too many projects that followed that direction and everyone realized later that they should have started with something more generic or "external" to their own needs.  The reasons are often tied to extending logical links to other data sources to leverage more powerful automation uses.  

For example, the sales department chose Excel spreadsheets to store all their static tables of rates and tiers for various services and products.  The engineering team would like to link to that in order to improve material and cost estimation, but the spreadsheets are on a server that doesn't allow access to the engineering department.  Furthermore, when they decide to move their spreadsheets to a SharePoint intranet, they realize the format and layout of the data is difficult to use for external linking.  Had they chosen an XML file posted on an intranet server, or used a robust Database host, the data would have been format and layout agnostic and more easily accessible to users without distributed and varied layers of security (NTFS, shares, groups, application-specific formatting, etc.)

All I'm saying is that you need to be cautious and take your time making a decision on where to store data that will be used by a lot of different roles throughout your organization.  It's kind of like placing a watering hole in a vast desert.  Those that are too far will suffer before they can reach it.  Those that show up with buckets that won't fit into the Well are going to suffer too.  Choose the location and configuration wisely.

What is “Scalable”?

You hear it all the time.  Vendors blabber about it.  IT engineers drop the word like sugar packets in a coffee cup.  Business pundits toss it around like a tennis ball on a court.  Scalable.

What is “Scalable”?

In the most basic form it means something can be scaled or grown larger and maintain some proportional characteristic.  For example, growing a business operation from 10 to 1,000 and the output metric grows by the same (or some proportional ratio) factor.

But that’s not entirely accurate.

The problem is that the assumption is always monolithic.  It assumes a system, process or entity is scaled as a whole, in one unit or grouping.  But you can scale in many ways.  It’s impossible to compare one scalar method with another as a whole.  There is no axiom that applies consistently across all scalar methods.

For example, do you grow the business operation by hiring and filling up an office under the existing management framework, or do you split into multiple operations in multiple offices with multiple leadership channels?

Do you add servers to your data center, or do you add more data centers and do you add physical or virtual servers?

Maybe you see the picture here: Monolithic or Hierarchical structural expansion.

Remember, NOTHING scales equally in all ways.  Most things scale best in one way only.

So the next time someone says something is “scalable” or asks “is it scalable”, ask them “in what way?”

Dave’s Rule: Problems scale infinitely, Solution never do.

Windows 7, Windows Server 2008 and R2

Anyone who has poked their eyes out reading my crap knows what I think about Windows 7 and Windows Server 2008.  But, just in case you still have your eyes (translation: you haven't gouged them out to avoid the excruciating agony of reading my rants):

Windows 7 is, in a word:  FANTASTIC

DISCLAIMER:

I am not a direct, indirect or contract employee of Microsoft, nor for any subsidiaries under their influence or control.  I was not approached by anyone to voice the following views.  Nor was I compensated in any way for this work by anyone from anywhere at anytime whatsoever.  These views are purely my OWN and I take full responsibility for what I say herein.  If you do not agree with these views, you are free to kiss both cheeks of my ass in the midst of a serious stomach flu or the day after a major Mexican dinner.  The choice is yours.

WARNING:

The following is a verbose, bloated, gut-wrenching, eye-socket-gouging, sphincter-tightening, fist-clenching, jaw-clenching, brow-furrowing rant which may offend those of non-Microsoft faiths.  It may border on blasphemous.  It may result in you falling asleep at your keyboard and waking up with small square imprints on your forehead.  You have been warned...

I consider Windows 7 to be the best operating system Microsoft has ever produced.  I've been a regular user of Microsoft operating systems since Windows 3.1.  I went through 3.11 (WFWG), Windows 95, Windows 98, Windows NT (workstation and server) 3.5, 3.51, 4.0, Windows 2000 (professional and server), Windows XP, Windows Server 2003, Windows Vista, and now Windows 7 and Windows Server 2008.  Taking the ultimate risk of alienating an entire race of beings:

Windows 7 is (to me) the best client operating system available today.

For years, I have worked extensively with UNIX variants such as Sun, Computervision, Intergraph, and many Linux variants including Mandrake/Mandriva, Suse, RedHat, Fedora, Slackware, and Ubuntu.  As recent as Windows Vista, I would still regularly split my time at home on Vista and some flavor of Linux within a virtual environment.  There were features in Linux I still preferred in both the UI and command shell.  I am still comfortable on multiple platforms, including Apple OSX Snow Leopard, which is very nice indeed.

I started into the Windows 7 evaluation cycle early on.  I jumped in during the alpha builds and progressed into the beta builds.  Like many others, I was doing a wipe-and-reload of my home computer with each new leaked build.  When the Release Candidate was posted I knew it was "it".  With the RTM release it was only fit-and-finish and even more impressive.  I have not even wanted to fire up any of my Linux virtual machines, nor bother upgrading them to newer versions (Ubuntu 9.10 was my latest).  Windows 7 is absolutely perfect for my needs.

All of the nay-sayer blabbering about the beta builds being faster because of "debug check code" and "disabled features", were completely and utterly full of shit!   By "full", I mean they were filled to capacity, including the vacuous cavity of skull volume that should have contained some sort of mammalian cerebral matter.  I will never again listen to those in-factual pseudo-pundits for anything, let alone technology advice.  If you read the tech mags, tech blogs and listen to the tech podcasts, you know who I'm talking about.

My computer, my wife's computer and my kids computers are all running Windows 7.  I love it.  They love it. And they RARELY love anything I love (except for our cat and dog of course).  It wasn't an overnight changeover by any means.  Let me elaborate here a bit...

Case in point:  I used my youngest kids for IT experiments to see if they really cared about using Windows over Linux or OSX.  I'd take them with me to the Apple store at MacArthur Mall in downtown Norfolk, and let them play with everything and watch their reactions from afar.  I'd ask them later what they liked and didn't like.  I have four kids, ages 10, 14, 17 and 18.  The 10 and 14 had their views. The 17 and 18 have theirs as well (and boy do they ever!).  All four like the animated interface of OSX and the pretty icons, but mostly they loved the large screen displays.  Once I got them using Windows 7 on a large screen they all seemed to discount the value of the animated interface goodies.

Then I replaced their desktop with a Linux desktop running Ubuntu with some Windows Vista desktop themes and also OSX themes and all the bells and whistles I could put on them that they tend to like.  The boot time was noticably slower than our Windows boxes (on the exact same hardware, by the way), and in the end, while they were fine playing online Flash games, none of their Windows games or apps would work and most importantly: iTunes was not available.  My older two use iTunes to shop and buy music and movies, which is (or was at the time) impossible with any of the iTunes clones for Linux (Floola included).  Synchronizing libraries was fine but they wanted more, and Linux couldn't satisfy their needs.

I put them all on Vista and they were borderline on that as well.  At times they would ask me to put them back on Windows XP, but I resisted, mainly to see if they'd "come around" eventually.  They did, but they were still not excited about Vista.

Then I put Windows 7 on their computers and it was like the room went from monochrome to color and someone turned the lights on.  Life is good.  Asteroids may come crashing into our planet, but we're just fine on the computer front for now.  All I heard for days was "wow!  this is cool! when did this get put in here?"

Windows Server 2008
Given that this came out in tandem (or near-by) with Vista, this was still a huge improvement over both Windows Server 2003 R2 and Windows Vista.  It's a rock-solid, easy to setup, easy to use, and extremely stable and reliable server operating system.  That's about the best way I can say it.  As with Windows 7, I jumped into the beta program early and never left.  People blabber on about how UNIX and Linux servers never need a reboot (which is utter bullshit unless you ignore patches entirely, and OH YES, they have their patches), and how "fast" they are.  A total crock of goat shit.  If you configure them to support an apples-to-apples comparison, Windows Server 2008 blows them away.  Pissed off yet?  Go back and re-read the sentence in purple again.

And Server "Core" is simply wonderful.  Sure, it stole what it does from what the Linux circus has been doing.  But the Linux circus has been stealing from the OSX and Windows world for years as well.  They all steal from each other.  Let's face it, the GUI was stolen from Xerox, so STFU everyone.

Windows Server 2008 R2 extends the vast array of improvements even further.  When most people who are familiar with Windows Server 2003 really dig into what's new in 2008 and 2008 R2 their jaws fall off their hinges.  When I used it for the first month I looked like Bob the Enzyte guy.

The list of features is like a late night infomercial wet dream.  Ok, a wet dream for IT geeks, but still a dream, and a wet one at that.  Reading about them doesn’t do it justice.  You have to SEE them and USE them yourself and you will walk away like you’ve been hit with a bolt of lightning.  If you don’t work with Windows or Windows Server it may be lost on you since your brain hasn’t been upgraded to handle this sort of incredible orgasmic synapsial activity boosting awesomeness.  Ok, maybe that’s a little over the top, but whatever.  I don’t care.  I passed another exam and I’m feeling pretty good.  And the weather outside is fantastic today.  What’s not to feel good about?

2008: The year of server flipping

We've all heard too much about "flipping houses" on various TV shows, magazines, web sites, etc. The practice of buying a "fixer-upper", fixing it upper, and then selling it for a profit, is a respectable effort (even a profession in some cases). But how does this fit with the IT world?

Simple: It fits with virtualization and existing hardware. A lot of the buzz being regurgitated on the web today is about how much virtualization will play in data center plans for 2008. There are all sorts of reasons and rationale provided. The interesting thing is how the various offerings will "fit" with respect to budgets and refresh cycles in most medium/large data center environments.

Many big spenders just finished a round of big spending, so it may be a tough thing for many IT managers to gently pry open the boardroom door again and kindly ask for more funding to chase after yet another enticing goal. The benefits are easy enough to justify and bulletize:

• Run more roles on less hardware
• Reduce electricity costs (hardware, cooling, etc.)
• Portability and failover simplification (aka Disaster Recovery)
• blah blah blah

Then comes the clincher: How much?

Well, the answer is, as always: "It depends".

On what? Well, you may be able to leverage your existing hardware, but that depends on which virtualization platform product line you intend to pursue. EMC's VMware ESX Server, Microsoft's upcoming Hyper-V and Hyper-V Server products, and open source (sort of) products by Xen and others, each have their own hardware limitations with respect to the physical host machine. That said, in a perfect world, many will be more than happy to buy new hardware with the latest and greatest goodies built-in, and begin consolidating their older machines into VM guest instances on the newer box. But in reality, many will be forced (by the evil budget folks) to make-do with existing hardware. In many cases that will mean hardware that won't run 64-bit products.

What to do?

Many will roll up their sleeves, drink some coffee, tell a few bad jokes, and while everyone is trying hard to laugh (like it was really funny), the IT tech will be switfly wiping a box and reloading with something like Linux and VMware Server 2.0, both of which are FREE and run just fine on 32-bit hardware. Ideal? Not really. But then again, define "Ideal". It is an entirely subjective term, like my other favorite terms: good, great, ok, fine, not too bad, and my favorite of favorites - so-so.

I will call this "flipping servers", since it's not moving things (servers) from older hardware to virtual instances on newer hardware. It will be ripping the walls and floors out and rebuilding the insides of the house and moving all new appliances and furniture (and tenants) in. Slick-n-reload as many refer to it.

So, for 2008, I see a considerable amount of server flipping in the pipeline. Everyone wants to virtualize and consolidate and gain from the benefits. But many don't have the budget or the cojones to ask for a budget increase to do it the way the vendors recommend. And is this really a failure? Hell no. Not by a long shot. It's getting the job done within imposed constraints. And in most cases it will work just fine. Maybe not for the likes of eBay, Amazon.com, or WSJ, but for most small-medium business data center environments (ok, "server rooms" for most of us), it will be what works and be affordable and attainable.