Fact-Checking the 24 Hours of PASS

I woke up this morning, excited for the 24 Hours of PASS, and the slide decks are already online! Let’s check out the slides for the first session, Field Testing Buffer Pool Extension and In-Memory OLTP Features in SQL Server 2014.

They start out by talking about how the SQL Server world has changed:

Things definitely have changed – 4-8GB used to be normal for SQL Server, and now 32GB is pretty much the bare minimum for physical servers as the slide notes. I see smaller VMs a lot still though.

Then they start talking about the first new feature, Buffer Pool Extensions. Here’s the hardware they used for testing:

Awesome! 24 cores means we’re talking about the Big Daddy of SQL Server – Enterprise Edition. The sticker price on this solution is around $168k USD, so it’s great to see they used the appropriate amount of memory here, 384GB of RAM. No playing around with this system.

So how’s the test set up?

What the – what happened to our 384GB of RAM? Why did we get SQL Server drunk and tie its hands behind its back? Someone, please, help, there’s a SQL Server being abused here!

And why are we testing with 2004 hardware? Didn’t they just get done telling us how hardware has changed? What the heck is going on here?

The “test” results are useful only in the sense that they’re good for a laugh:

So transactional throughput goes up by 37%, but CPU … TRIPLES?!? How about we let this SQL Server use all of its memory and see how the test fares? Why wouldn’t this scenario be tested?

This is where I have to keep asking – is this an educational presentation, or a marketing presentation?

I don’t have the time before this session to really analyze the deck, although I’ve written about the wild inaccuracies of Fusion-io’s SQL 2014 tests before. Let’s just look at the takeaways from the first feature tests:

The “sweet spot” is less memory than SQL Server 2014 Standard Edition supports? Putting a Fusion-io drive in rather than buying memory? What, are Fusion-io drives free now? In what world does this make any economical sense at all?

So what references did they draw from for this “educational” session?

Oh, right.

Everyone involved in this session should be ashamed. This isn’t community education. It’s Fusion-io marketing, pure and simple.

I am deeply disappointed that PASS would allow this to masquerade as educational material.

Update After the Presentation

The presentation was just what it looked like – a Fusion-io marketing pitch, not educational material for SQL Server community members facing real pain points. Huge, major props to Bradley Ball, the session moderator, who asked the attendee questions and held the presenters to the fire.

During the Q&A, we learned that the testing didn’t actually unearth any real use cases for Buffer Pool Extensions. The presenters admitted that whenever you can add memory to a server, that’s a better answer than buying expensive solid state storage. Their testing also showed limited performance gains over 16GB of memory – thereby meaning that it’s always better to add memory.

The presenters kept suggesting that this technique would work for virtual machines, though, demonstrating a fundamental misunderstanding of virtualization. Don’t ever use local SSDs in the host for SQL Server’s Buffer Pool Extensions – this stops you from using vMotion or LiveMigration to recover automatically from host failure. This is just simply the wrong answer, period.

The good news is that Bradley pushed past the vendor spam and got to the truth.

The bad news is that this session was shown to PASS members who didn’t have the benefit of knowing that it was pure marketing spam, plain and simple. This is why I’ve been beating this drum so hard leading up to PASS.

If PASS is going to show spam and call it training, it’s up to us – community volunteers – to get the truth out.