The DBA is Dead? No, the DBA as Gatekeeper is Dead!

I still remember this meeting when I had been a DBA for less than a year at IBM. It was IBM, so I was the lone primary DBA for this internal client. With 3 whole months of experience and a shiny college degree. The system administrator told the client “No”. What a revelation that was to me, that the answer could just be “No”.

Gatekeeping and Where it has Gotten Us

One of the roles of a DBA is to keep people from doing stupid stuff. Always has been, and always will be. Here’s the problem: particularly in certain companies, the DBA team has a reputation as being obstructionist. And honestly this is more at the root of a move away from relational databases than anything else.

When I started as a DBA, the RDBMS was chosen by an architect or other high level technician. It was chosen on factors like what vendors supported and what DBAs at the company had experience with. Sometimes cost was a determining factor. Sometimes this was directed by a CTO who had a particular preference. But the data storage answer for an app was always RDBMS.

Over the course of my career, I’ve seen a fundamental shift. While all of those factors still play a role, more and more developers are the ones choosing the RDBMS. And the DBA gatekeepers have taught them (wrongly) that RDBMS is not the solution because the developer has so little control and must constantly meet the DBA’s “No”.

Our role as DBAs, then should not be to be the brick wall that developers beat their heads against, but to be the enablers and the ones to express how easy and efficient using a relational database is. Because if you don’t have unchanging and well-defined access patterns for your data, a Relational Database can still outperform most other data storage schemes for most things. Yes, there are cases where big data requires something else. The problem with storing unstructured data in an unstructured or schema-less format is that while that makes ingesting the data faster, it often makes querying the data and especially aggregating and returning subsets of the attributes in the data slower.

With Docker the old hassle of helping developers have their own personal development environment is so much easier – if we can get relational DBAs to embrace Docker and if the RDBMS offers a free version for small environments (like Db2’s Developer-C ).

There are so many new innovations and features available today in IT, and so few of them filter into the DBA space, or they seem slow to filter into the DBA space. Database object versioning and deploying new versions seamlessly without outage? That should be baked into every RDBMS, and it just isn’t. Plugins and sample code for configuration management and infrastructure as code tools like Puppet and Chef? Those should be everywhere. Tutorials to really understand concurrency control and the differences between snapshot-based and serialized, and even optimistic locking? They should be a dime a dozen.

If DBAs don’t make the effort to modernize, then we really are dinosaurs and it will be our own fault when we are made obsolete by cloud options, by storage options that developers can control (for better or worse), and by the general feeling that we obstruct without adding value.

There is still a place for “No!”, but the DBA should instead try:

1. “No, but you know what would work? …”
2. “That’s problematic, what are you trying to accomplish?”
3. “How can I help you automate that?”
4. “What data/database related problems are you facing? How can I help?
5. “Yes, that would work, but you know what would work better? …”
6. “Have you considered …”

DBAs in the Cloud

There’s an illusion at the moment that moving data and databases to the cloud eliminates the need for DBAs. Perhaps it changes our role and the amount of DBA resources needed, but it does not eliminate the need for DBAs. Here are a few ways DBAs are still needed:

Understanding what “Hosted” or “Managed” Means

When entering in to any cloud scenario, there are very specific lines drawn on what is your job and what is the cloud provider’s job. It is important to understand this at a very technical level so you don’t miss out on very important tasks that must be done to keep a database healthy.

Asking the tough questions

I have actually worked with a couple of smaller cloud providers, and you would be shocked at how they manage their databases. One small cloud provider came to a company I worked for when a database they were supporting went down and they could not recover it. They literally had NO backups over 2 years of providing database services behind the software they offered as a service. We unfortunately could not help them. Loosing two disks in a raid 5 simply requires having some kind of backup available. Your DBA can get beyond SLAs and what is promised to how the provider is actually fulfilling things to see if they really can provide what they promise.

Performance

Do you think your cloud provider is going to come out and admit that your performance problems are caused by over-subscription of CPU or memory, or even storage? In most scenarios, the reaction is going to be to point to the application you’ve written or the database structure or the SQL. All of these are things they don’t cover. You still need a DBA to help with database structure. You still need a DBA to help optimize SQL. These are things that are not strengths of many developers. Your DBA can see where bottlenecks are and point you to very specific questions to ask your provider to get to the heart of problems that can be caused by bad cloud architecture or support.

Security

Every RDBMS offers a wealth of security features. Do your developers know and use these features? The vast majority of scenarios that I see include the application using one very highly-privileged ID for all applications to access a database. This has all kinds of problems that lead to data loss and exposure for hacking at so many levels. DBAs are likely to be familiar with the various security schemes and how to implement them to secure your data without making developers’ lives harder than they have to be.

New Features

All of the major RDBMSes are adding new features constantly. It’s not just the open-source or current hot NoSQL provider that do this. Oracle, IBM, Microsoft, all of them have been putting millions of dollars a year into developing new features for their RDBMS clients. They’re still putting millions of dollars a year into these products. They broadcast the new features, but it sometimes takes a deep technical understanding of the product to identify (for example) that automatic column group statistics in Db2 11.5 could get a major performance boost with little downside. DBAs keep their ears to the ground for this sort of thing. We revel in the details and delight in the things that may sound very minor, but could make a big difference for your databases. Developers understandably work with multiple databases and data storage methodologies. They should. They should not be expected to understand the use cases of IBM Db2 BLU and where and how it can absolutely destroy or skyrocket your query performance. This is for the DBA to know and to advise on and codify into shorter and more digestible ideas.

Monitoring

My cloud provider is monitoring database performance and will fix it if there’s a problem, right? Well, they might be monitoring for the large-scale performance problems at the system level, but they’re not going to notice you’re suddenly seeing a spike in lock-wait time. They probably won’t even notice if there’s an extra few milliseconds in disk read or write time. These are things that a DBA can proactively identify and either work with the developers to fix if it is a logical problem, or take the details to the cloud provider if it is a physical one. Using the same monitoring tools we have today, we can often monitor cloud databases. Or we can learn the cloud providers’ tools and use them. Things like query execution time or the delay that the app might be seeing are things that are noticed when it is too late. DBAs can often identify things before they become noticeable. Let me watch a database during a load test – even if the load test is a drastic success, I can probably find improvements that wouldn’t have showed up until you hit double that much volume.

Summary

If you’re the “NO” DBA, it is likely that at least one person in your organization can’t wait to find the way to get rid of you – whether that’s switching to MongoDB, moving to the cloud, or just justifying the hiring of consultants who will say “Yes” more often. Change or become obsolete. Embrace new methodologies like DevOps and figure out how you can make them better instead of blocking them. Don’t fear the cloud, work with it, embrace it, learn to monitor and analyze it and add more value to your organization than the scripts that take the backups. If you can be replaced by scripts, you eventually will be.