Monday, December 12, 2011
Database Digressions and Developmental Digestion
Put on your caffeine hat and come with me on a voyage into the ethereal world of nerds versus geeks. That's the party peel-off sticker I'm slapping our foreheads, written with a blunt Crayola crayon. With enough alcohol and caffeine it will all make sense soon enough. Let's go...
I would venture to say that 99.99 percent of software development projects, probably even software development organizations, are arranged into de facto groups that place the application "coders" in a separate function from the database folks. The problem this has created has been legendary. It's not only led to human behavioral issues and philosphical approaches to address such structural idiosyncracies, but it has also led to mechanical approaches, such as LINQ and XPATH, and so on. Trying to fit automation processes to human processes is always a challenge, on a good day. When the human process is broken, it's a disaster that unfolds slowly and often at an incredibly high cost (both in terms of time and money).
Did i say INCREDIBLY high cost? Yes. Incredibly high. The problem is that it's almost always a secondary, or tertiary cost. The kind that don't stand out in distinction on a balance sheet, so they often hide in the margins. Those are what often lead to bad feelings between IT groups and the Financial groups. The main reason is that the Financial folks are left to ponder where these strange, nebulous cost creeps are coming from, while the tech-minded IT (Dev) folks are rarely prepared to articulate and quantify them adequately. They may indeed "know" where they come from, but cannot communicate it in the language of Finance, so they shy away from it, making the problem worse over time. This is analogous to a couple that don't discuss sensitive issues because they know they will always turn into an argument.
I'm digressing into the human/financial aspects. But what about the structural and mechanical aspects?
I have mixed feelings about things like LINQ. It's technically a very clever, impressively creative approach to translating mental processes from one world to another. But at the same time, I've rarely seen die-hard DBA's embrace it over traditional SQL/T-SQL, so the divergence is not only mitigated, but elevated with yet another wedge driven into place. A classic example is when a Dev guy walks over to ask for help with a complex SQL query that was coded in LINQ (or anything other than T-SQL) and the DBA looks at it and says "Sorry. I work with SQL. Can't help you." I am aware there are exceptions to this scenario, but they are "exceptions", not the general, majority rule, at least from what I've seen and heard. I admit that I haven't seen or experienced every environment, so I'm obviously speaking anecdotally.
I'm picking on LINQ unfairly though. It's not really a fault of LINQ. It's a respectable concept and the incarnation has evolved respectably as well. It's somewhat of a situation of "blaming the messenger" when the message is the problem. The message is unavoidable though. The message is a symptom of an age-old broken human condition in the IT environment: divisional politics. Not departmental, but "divisional" in the same sense as "functional", whereby the DBA and Dev folks are functionally divided. They may be best of friends. They may be mortal enemies. It doesn't really matter, since they still focus on, and operate within their own distinct worlds, with their own unique languages and customs.
Bridges of Translation
If you are in a large enough organization to afford the luxury of a dedicated API group, they may be your bridge. Having to convey the wishes of the AppDev folks with the plumbing capabilities of the server and DBA folks, they become the liaisons of different cultures and dialects. They may even provide the abstraction that spares the AppDev team from the horrors of learning a different culture in order to cook their programmatic banquet meals. If you're not that fortunate, it sucks to be you, maybe. Just kidding. Ok, I'm really not kidding, it really does suck to be in that situation. I've been there so I'm not trying to condescend as much as sympathize.
The end result of these cultural divides is that databases go in one direction and code goes off in another direction. Efficiency suffers. Progress is stammered. If the groups are in different physical locations (different rooms, buildings, cities, countries) it only exacerbates this further by making it too easy to cultivate an "us versus them" environment. If you have any poison pill personalities in either group it can be gasoline on a smoldering fire, so be careful of those.
Light at the End of the Tunnel
If you can bridge the divide, it is always worth the effort. Always. I cannot stress that enough. Even if pride is a boulder to swallow, break it into smaller pieces and work on it. Offer some concessions, some goodwill, something to prove any ney-sayers and poison pills on the other side incorrect about their assumptions about your group. Conversation is key. Get the people talking. Learn what the other side has to deal with. Maybe there are pains your group causes them that you're not even aware of. Just learning about such things can help you refocus and make adjustments that may seem minor on your end but could be HUGE on the other end.
Making exchanges of conciliatory effort can go a long way towards building a stronger team, improving communication, raising productivity and positive outlook, and ultimately making a better product. Quality can't happen without a cohesive group of people, and you can't bridge any divides by waiting for the "other side" to make the effort. You often have to make the first move and meet them halfway. Ultimately, until people issues are resolved, you can't achieve an efficient operation or efficient processes. This is where most of the bullshit forms-bloated corporate environments grew out of. People issues give rise to barriers of mistrust and push-back over perceived unfairness of responsibility and effort. You can throw billions of dollars at trying to make those processes automated but if they attempt to lift the human process into a computerized process model, it will be horribly broken, and inefficient at best.
Before you ever consider purchasing or developing a system to model a business process, do some deep analysis of the process itself. Never start with the current process as the assumption of efficiency. In most cases it's nowhere near what it should be. Fix the process. Build cohesion. Push forward, but don't drag the baggage along. It may look frightening at first, and people will scream and complain and flip out, but if the process is re-modeled properly, nobody can argue with a better model. Once that's done, you can translate that into software and hardware and move on to the next challenge.