Why basic IT services must be commoditized before we move to the next level

I recently got an e-mail from Gord, a former colleague who just came back from a job interview. Gord was lamenting the fact that in spite of all the talk surrounding data stewardship, metadata, and business intelligence. The reality is that most companies are still interviewing people based on product specific technical expertise.

While on one hand it is becoming more and more apparent that “IT failures” are less issues of technology working, and more issues of poor business alignment, companies when hiring are not specifically requesting these skills, or looking into this track record. So, while an interviewer interviewing for a DBA position could ask the questions:
“How do you ensure that the data modeling policy is being followed? How do you deal with non-compliance?” or;
“Have you ever worked in an environment with a focus on metadata management? Can you describe the challenges, and how you dealt with them?”

Instead, the main questions are these:

“Have you ever completed a major database upgrade project?”
“How do you configure Real Application Clusters in Oracle 10g?”
“Describe a robust back-up regime.”

Reading back the questions, it’s clear that the former set of questions are mushy and don’t have clear-cut right and wrong answers. The latter set of questions are point-blank, and while there may be different ways to answer them, the answers can be easily validated.

But there should be other observations from looking at these questions. The answer to the first set of questions should give you an idea of how business minded the DBA is. While the answers to the second set of questions will give you no such insight; but they will tell you how competent the DBA is at physically managing the database.

For the time being, DBAs that can, say, perform a rolling upgrade with zero downtime is quite the hero indeed. But on the same note, why must that DBA be confined to a single company? Isn’t that ability applicable to EVERY company, regardless of their line-of-business. I mean, if you can ensure that behind the scenes your DB is running flawlessly, why are these trades being haphazardly being reproduced in every IT department. And by the way, I’m not just singling out DBAs here, I would say that at least half of all IT roles have little or no direct linkage to business activity.

Of course, we hear about how these roles are becoming less relevant due to increasing automation, but I don’t buy this. Any system that is automated still requires people to monitor them, as well as people to fix them. Simply put, the DBA, the network technician, the Java/VB/C# developer, and all of those other roles which are not in themselves expressions of the business, are still necessary.
Where I believe the reluctance for change is in outsourcing and in sharing of resources across companies.

Ironically, the companies that have a better understanding and a greater need for things like data governance and business rules management are also the least likely to let go of control over these basic services. The main reason for this is stability, security, and privacy, and an overall greater dependency on IT to automate their business.

On the flipside, smaller businesses, especially those that are growing quickly, are more likely to take a risk with a hosted solution. An IT debacle at a small transport company is far less likely to get pasted all over the following day’s headlines than a screw-up at a major bank. Another difference is that smaller businesses that go with a hosted solution may also see it as more risk averse. By sailing on the same ship as dozens or even hundreds or thousands of other businesses, there is the comfort in that there is “safety in numbers”. As sound as this argument is, large companies are simply not as trusting and often see themselves as more important than any of the hosting vendor’s clients, and therefore more important than the even the sum of its clients. In my own opinion I think there is some validity to this argument, so I’ll leave it at that.

With that said, there is still a gaping hole left to fill. Namely, the vast majority of hosting vendors are what are known as ASPs or Application Service Providers. ASPs typically offer hosted versions of popular workgroup applications, such as accounting applications, reservation systems, and so on. While these applications are all configurable, by comparison to a custom built application, they are extremely rigid and can quickly calcify even a small business’ operations by forcing the business to operate in a predetermined fashion.

At the other end of the extreme are what I would describe as “raw” hosted severs. These are companies which will host an Oracle DB for you, and possibly provide some basic DBA assistance. While these services are definitely a step in the right direction there is little preventing you from shooting yourself in the foot.

So I say we are in need of higher level services that are easily configurable, but are abstracted to a business level.

The solution of course is more generic web services. We are beginning to see these pop-up, but from what I can tell, they are still in their infancy. I must admit, I have not done a recent survey, so I can’t tell you if the services I describe below exist yet, but if they do, I urge you first and foremost to bring them to my attention (free plugs), and secondly to review them yourself. Thus, the services I have in mind are:
1. A hosted business rules management (BRM) solution
2. A hosted business process management (BPM) solution, with business activity montoring (BAM), workflow management, transaction management, and global scheduling
3. A hosted relational database solution, with ETL functionality, and metadata management
4. A hosted forms solution (I know a few of these already exist, and I use them)
5. A hosted reporting solution (although I can’t think of any, I’m sure there must be some of these out there)
6. A hosted user directory and identity management solution (I know that these also exist, but am not sure if they had these types of services in mind)

Now, as I just mentioned above, with the exception of hosted forms, and possibly hosted reporting solution, I haven’t seen any of the other hosted solutions on the market. I would say the main reasons for this are:
1. The transactional through-put of web based applications has yet to match traditional solutions.
2. It is not clear how these services would integrate with each other, and more importantly there are no standards for doing so.

For the first problem, of computational resources I’m not too concerned. If the above mentioned solutions are deployed in a grid-computing fashion, and there are multiple applications running on them, then there are tremendous opportunities for optimization. This is simply a problem that will work itself out over time.

The second problem is much thornier. Standards take time to work out, and can often limit the flexibility of what can be done. A more likely outcome is that a large web services hosting company such as a Google or an Amazon.com may release a single packaged suite than encompasses all of these tools. We can see that both of these companies are already positioning themselves in this way with Google taking a more desktop approach and Amazon.com taking a more back-office approach. However, I don’t see either of them taking this space head-on.

I myself look forward to the day where I can architect, build, deploy, and provide high-level support for a full-blown system for a client in Africa or Asia, Europe, or wherever the business may be, without ever having to worry about anything other than the business details to do my job.

One of the great blessings about a career in IT is that it gives you a window into practically every other business out there. The more we can get away from commoditized details and move towards the technical essence of the business, the more varied and interesting our jobs become. Let’s hope these solutions happen sooner than later.

BEGIN SHAMELESS PLUG

I have recently got involved with a pretty cool internet radio project, that I’m proud to unofficially announce. It’s called TUN3R.com and to put it into a nutshell, it’s a next-generation internet radio portal. What makes it different from other radio/music portals is that all stations are laid out in an expansive grid which you can whiz around with a cross-hair to “tune in” immediately to any station.

The technology itself is very impressive, and what I like most is that it reminds me of the good ole’ days where you could just play around with a station tuner and randomly find things you were never expecting to hear. For now though, the product is not fully baked, and we’re going to be radically improving the searching, including the addition of a new type of search which to my knowledge has not been done yet, so that will really push the site to “11”.

Anyway, please check it out at: http://tun3r.com/

END SHAMELESS PLUG