Databases Eight Years from Today, 2018 Edition #TSQL2sday

Almost exactly five years ago today, back in March of 2013, I wrote a post called Databases Five Years from Today. In it, I predicted:

• You’d still be supporting 2005 and 2008 – while the number of 7% of servers are still 2005 and 2008 might seem small, it means that on average, every shop with 14 servers still has one of these boat anchors. (The folks at Quest recently told me the number of 2000 servers is still big too, but their monitoring app just stopped supporting 2000, so that was that.) I give myself a point on that.
• There’d be no widespread adoption of Hekaton, columnstore indexes, or AGs – and I’d concur here. I don’t have solid numbers to back these up, but I bet less than 1 in 5 production servers has any of these 3 features. I give myself a point here.
• We wouldn’t see massive widespread migrations to Azure SQL DB – but I was careful to point out that I was only talking about existing apps, whereas new development would likely start in non-SQL-Server places. I know Azure SQL DB gets a lot of press, but as of March 2018, the prediction date, the widespread migrations aren’t happening. The painful implementation of cross-database queries alone made it a non-starter. I give myself a point here too.

3 for 3, that’s pretty good – especially given how far out on the lonely island I felt when making those predictions. But you know what’s interesting? Fast forward just one or two more years, and my accuracy would probably drop by a lot. Predicting more than five years out is really hard. Surely nobody could get that right.

The 100th T-SQL Tuesday says, “Predict 100 months out.”

Adam Machanic asked us to look out 100 months, or about 8 years. To give you some idea, here’s a sampling of my blog posts from 8 years ago:

• MCM Prep Week: Interview with Joe Sack – I was just starting to go through the MCM program, and Joe was running it. Today, I run a consulting company, and Joe designs adaptive query processing, and we’ve also both changed employers at least twice since that post.
• SQL Azure FAQ – the product was just launching. Since the post, it’s gone through more name changes than Joe & I have had jobs. It grew in ways you would expect (max database size is up to 4TB), and had some odd head fakes along the way (remember Federations?)
• SQL Server Magazine Bankruptcy – I’m old enough to remember a time when this was a physical print magazine that you could hold in your hands.

So with that in mind, some of these predictions are going to seem a little wacko, but I’m aiming way far out. Let’s go from safest to riskiest bets.

In 2023, DBAs will still be a thing.

The safest bet in this post – this seems so incredibly obvious to me, but there’s still “thought leaders” out there saying the opposite, so I have to put this down in writing.

There’s a chance we’ll be called Database Reliability Engineers, but the core parts of the job – designing, building, securing, and scaling data storage – are still going to be a lucrative career in 2023. In fact, it’s going to be even bigger because…

The data safety business will look like the car safety business.

Car manufacturers struggle with safety regulations: every country insists on having their own slightly different standards. For example, over in Europe, you can get adaptive headlights that automatically point out things you should be seeing, and dim specific areas of the light so oncoming drivers aren’t blinded.

In the US? Nope – a 1968 law prohibits them.

Turn signals in the US: amber. In the EU: clear. Fog lights in the back of the car? Required in the EU, never seen in the US.

But car makers have to build vehicles that are sold everywhere, and countries refuse to agree on what’s “safe.” Manufacturers end up with fleets of lawyers, designers, and engineers to build all kinds of differences to meet where a particular car is sold.

We don’t have that luxury in the database business: the same web site code base has to service customers everywhere, and meet different regulations based not on where the customer is surfing from – but a combination of their location and what country issued their passport.

This is gonna suck, bad, and governments simply aren’t going to suddenly start cooperating on a single data standard that works for everyone. That’s not how governments work.

Update 2018/09/23: California passed their own act, which doesn’t line up with the GDPR. There’s no United States standard yet.

Update 2021/11/15: there’s still no US standard, and Brexit has made the EU GDPR situation even more intricate.

<= 5% of SQL Servers will run on Linux.

(To be clear, I’m talking about Windows as their primary OS. Yes, a lot of SQL Servers run in virtual environments inside VMware, so it’s SQL-in-Windows-in-Linux, but that doesn’t count as running SQL Server on Linux. I’m also not talking about Azure SQL DB – I wouldn’t be surprised at all if Microsoft switched that over to Linux in the 8-year time span.)

Where’d I get the number from? Well, today in 2018, 72% of installs are from the last 8 years, SQL 2012/2014/2016/2017. That means 8 years from now, maybe 72% of installs will be SQL Server 2017 or newer – and 2017 is required for Linux support. Realistically, the only Linux install range would be 0-72%.

Even if 1 in 10 new SQL Server 2017s were installed on Linux, that’d still only be 7% of the install base. This prediction is way safer than it looks. (I almost said 1%, but I think there’s a decent chance that truly large shops – like shops with over 1,000 instances – will use Linux, and even small adoptions there have a big difference in the numbers.)

Update 2021/11/15: based on the data I’m seeing in SQL ConstantCare®, Linux adoptions are still well, well under 5%, and I see similar metrics from talking to other folks out in the field.

Your developers will have several projects built with serverless architectures.

Right now, when I talk to data professionals about serverless architecture, I can almost hear them tuning out. I understand – until you’ve used it, it just seems so farfetched. But going from our experience on PasteThePlan and SQL ConstantCare®, it’s utterly phenomenal.

But serverless is going to mean way more to you in 8 years than Docker containers, SQL Server on Linux, graph databases, or Hadoop. Your developers are going to be all about building apps in function-as-a-service platforms, and they’re going to wonder why databases are so far behind.

Your default new database will be in a cloud PaaS.

Five years ago, when someone asked for a new SQL Server database, you might have created it either on a shared physical server, or a shared (or rarely dedicated) VM.

Today, you probably default to creating a new database on an existing virtual machine. Most of those virtual machines live on-premises, but there’s a significant percentage that live as VMs in Azure VMs, Amazon EC2, and Google Compute Engine. You wouldn’t dream of deploying a new physical server by default.

Today, you likely wouldn’t respond with, “Sure, I’ve created you an Azure SQL DB. Here’s how you connect.” For SQL Server, your only two PaaS options today are Microsoft Azure SQL DB and Amazon RDS SQL Server. Microsoft’s on the cusp of releasing a 3rd option, Azure SQL DB Managed Instances. Their marketing site says it’s in public preview, but it’s not – you can sign up, but they don’t have enough staff to support new users.

By 2026, I think the next shift will already be over and done – just as we switched from physical boxes to VMs, we’re going to shift – but not to VMs in the cloud. In 2026, I bet your default new database will be in a Platform-as-a-Service option. It might be Azure SQL DB Managed Instances, or something else entirely.

Which brings me to the next prediction…

2 big clouds will offer an MSSQL-compatible serverless database.

Amazon’s got a head start on this in preview now: Amazon Aurora Serverless is an on-demand, auto-on, auto-scale, auto-off database server with MySQL compatibility. You don’t pay for instances, availability zones, or regions – you just pay for the queries you run, per hour that you’re making database queries.

If you haven’t seen Aurora yet, the intro video does a great job of explaining why businesses hate databases:

https://www.youtube.com/watch?v=_oWa4fWAqTY

I’m predicting a couple of very big leaps here:

• Google and/or Microsoft are going to follow suit on Aurora Serverless’s pricing, and
• Amazon and/or Google are going to offer their own Platform as a Service implementation of Microsoft SQL Server (like Azure SQL DB, but different) – or Microsoft is going to license it to them, or who knows, even open source some part of MSSQL that enables this to happen

Either of these bets is risky on the 8-year horizon, but I’m going out on a leap and making both. I’m going to hedge my bets a little though:

• They may not be compatible with the latest version of SQL Server – for example, if it came out today, I think it’d get serious adoption even with just SQL 2012 compatibility. (That’d be 61% of the market, remember, and old apps are often on autopilot with low performance requirements, great fit for a serverless database.)
• They may not get any quick adoption – it takes years for a service like this to catch on. (Azure SQL DB is a great example – it’s 8 years old now.)

Update 2019/05/06: Azure SQL DB Serverless is out.

Update 2021/11/15: Amazon Babelfish for Aurora PostgreSQL is out.

Microsoft will fix the “String or binary data would be truncated” error.

This is the riskiest prediction out of all of ’em.

Oh I know what you’re thinking: it’s been the top-voted user request for over a decade, and as soon as Microsoft dumped Connect and switched to a new user feedback system, this request immediately bulleted to the top of the list again, getting almost 3x more votes than the #2 issue!

And yes, someone from Microsoft recently commented on it:

All I can say is:

Update 2019/03/20: this error is fixed in SQL Server 2016 SP2 and SQL Server 2017 with trace flag 460.