Some Questions I’ve Answered Recently On Database Administrators Stack Exchange

Good Citizen


Here are some interesting questions that I’ve answered on dba.stackexchange.com:

I should probably get a real job.

Thanks for reading!

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

Free SQL Server Troubleshooting Stored Procedures

Giveaway


I’ve been doing a lot of work on all of my free SQL Server troubleshooting stored procedures lately.

If you haven’t used them, or haven’t even heard of them, now’s a good time to talk about what they are, what they do, and some of the newer features and functionality.

I hate to be one of those “bug fixes and performance improvements” people, but I am really happy with the way a lot of these are working.

It’s not that I’m not constantly looking for ways to improve, it’s just that for a lot of them there aren’t really “new” things I can go find and show you, or new ways to look at old things that I think would be helpful.

New things tend to come with new versions of SQL Server. If you don’t like it, tell Microsoft to ship more often. Ha ha ha.

For a full overview of what all of them do and all that, head to the README file.

For everyone else, feel free to follow along.

Fresh Install


I used to get really annoyed with myself, because it took a lot of clicking to install or update all the scripts.

Now, thanks to GitHub magic (actions), and a world-renowned super genius (Drew Furgiuele b|t, who also has an awesome podcast that you should listen to), you can get all updates in a single script.

Because I’m forgetful, I wanted this to happen every time I merged into the main branch without me having to remember to rebuild the main file.

That’s what this whole thing does.

What a gem, that Drew.

sp_HealthParser


I’m going to start by reintroducing a couple of the newer procedures that I released leading up to my PASS Precon.

The first one is sp_HealthParser, which digs deep into the system health extended event for all sorts of gruesome details about what’s going on in your SQL Server.

I’d been meaning to do this for a long time, and finally had the time and patience to write all the necessary XQuery to do it. Once I started writing it, I started kicking myself for not doing it sooner.

There’s so much great information in there:

  • Queries with significant waits
  • Waits by count
  • Waits by duration
  • Potential I/O issues
  • CPU issues
  • Memory issues
  • System health (duh) issues
  • Blocking
  • Deadlocking

The really cool thing about the wait stats is that they get logged in a way that you can see them over time, and see when your system was busy.

The most common way I execute it is like so:

EXEC dbo.sp_HealthParser
    @warnings_only = 1,
    @skip_locks 1;

There are a number of sections in there that have a column which will flip to WARNING when something happens that your SQL Server doesn’t like, and a lot of the time I wanna focus in on just that.

I also usually skip the blocking and deadlocking sections, in favor of setting up specific extended events to monitor them.

Also because the blocked process report that it captures is missing a lot of details that the real blocked process report has in it.

That being said, if you’re looking at a server for the first time, you may want to see what’s in there.

sp_LogHunter


Another newer procedure is sp_LogHunter. The idea here is that SQL Server’s error log is a horrible window into your server’s soul, but searching through it is a nightmare.

  • You can only search for a single thing at a time
  • The results usually exclude a lot of useful information around any hits for your search
  • Often using the GUI is incredibly slow and involves a lot of clicking

What I wanted was a stored procedure that looked for a whole bunch of the awful stuff I’ve seen in error logs over the years, and the results to get returned in chronological order, all together.

That’s what this does. There are a bunch of configuration options, but I usually just fire it off to look through the last 7 or so days of log data. But you can choose specific time windows, look for custom message entries, and more.

EXEC dbo.sp_LogHunter;

sp_HumanEvents


Extended Events are great, but for all the XQuery you have to write to get data back in something that isn’t the cursed-by-old-and-new-gods GUI.

One of the hardest stored procedures I’ve ever written, sp_HumanEvents, does all that for you. Not for every extended event, but for the ones I found myself using most often with clients.

  • Query performance
  • Wait stats
  • Compiles
  • Recompiles
  • Blocking

This thing can do a ton of different stuff, but the most common thing I do with it is to troubleshoot slow stored procedures that clients have me working on.

In SSMS, there’s no way to only return some query plans, and a lot of stored procedures I end up working on have a lot of queries in them (some very fast, some very slow), and I want to focus on the slower ones.

To get set up to do that, this is what I run:

EXEC sp_HumanEvents
    @event_type = 'query',                   
    @query_duration_ms = 1000,               
    @session_id = N'58',                    
    @keep_alive = 1;

That will create an extended event that monitors a single session for any queries that run over 1 second.

From there, all you have to do is watch live data, and then run the procedure to get a ton of detail about them.

sp_HumanEventsBlockViewer


For years, I’ve been working on sp_BlitzLock to troubleshoot deadlocks in SQL Server. One day, after going through a lot of yuck to do the same thing with the blocked process report, I decided to write sp_HumanEventsBlockViewer.

It was originally intended as a companion script to look at the blocked process report extended event that sp_HumanEvents sets up, but it quickly became its very own island unto itself.

Now, you do need to enable the blocked process report, and you do need to set up an extended event target to capture it, but once you do the amount of detail you can get about blocking problems on your server is incredible.

To see how to do that, check out the README file. Once that’s done, just point this baby at it and let’er rip. Not RIP 🪦

EXEC dbo.sp_HumanEventsBlockViewer
    @session_name = N'blocked_process_report';

sp_PressureDetector


sp_PressureDetector, at least according to Google’s search reports, is my most popular script.

This one got a lot of work before my PASS Precon. Not to add stuff to it really, but to make the results as clear as possible.

The idea behind this one is to show you resource pressure currently happening on your SQL Server, and when possible, queries that are causing it.

  • CPU
  • Memory
  • Disk
  • tempdb

It will also give you a lot of good information about the server itself, like how much data is stored on it, and configuration details that most often matter for hardware.

You don’t have to do much else with it aside from hit F5.

EXEC dbo.sp_PressureDetector;

sp_QuickieStore


An ode to the dismal shortcomings of the Query Store GUI, sp_QuickieStore comes to the rescue.

By default, it gives you the 10 worst queries from the last 7 days by average CPU. There are about 100 ways to search for, include, and exclude certain queries from the results.

Seriously. Just look at all the parameters for that. You can find what hurts, research a stored procedure, search across all databases with Query Store enabled, and way more.

The thing that I kept running into, though, is that the default results would often include a lot of results that weren’t pertinent. Overnight problems and whatnot.

For years I’ve been saying “anyone can find a slow query”, because that’s really easy. But you need to find the queries that matter to you.

A little before Christmas, I decided I should live by my own rules, and added a few new parameters.

Here’s how they work:

EXEC dbo.sp_QuickieStore
    @workdays = 1,
    @work_start = '9am',
    @work_end = '5pm';

When you use these, it will only show you queries that happen on weekdays (respecting local DATEFIRST settings), with configurable time frames.

By default, it look from 9am to 5pm, and those hours will automagickally convert from your local time zone to UTC (because that’s how Query Store stores dates). But you can adjust that to your needs.

If you want to find everything that happens every weekday morning, afternoon, or all day, you can do that now.

Time Flies


The first commit I made to sp_PressureDetector was back in December of 2019. Looking at the code now is a touch sentimental, and more than a touch embarrassing.

My formatting game, along with the small amount of details returned, aren’t really anything I’m psyched about. But I’m very proud of what these have all turned into.

I get decent traffic to my repo. There’s a dbatools cmdlet to install my scripts. I get nice emails from folks out there who have found and solved problems with them, and I’ve solved hundreds of client problems with them.

It may all sound a little braggy, and I’m totally okay with that. I’m super proud of them these days.

Thanks for reading!

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

Join me at DataTune in Nashville, March 8-9 2024

Spring Training


This March, I’ll be presenting my full day training session The Foundations Of SQL Server Performance Tuning.

All attendees will get free access for life to my SQL Server performance tuning training. That’s about 25 hours of great content.

Get your tickets here for this event, taking place Friday, March 8th-9th 2024 at Belmont University – Massey Center 1900 Belmont Blvd, Nashville, TN 37212

Here’s what I’ll be presenting:

The Foundations Of SQL Server Performance Tuning

Session Abstract:

Whether you want to be the next great query tuning wizard, or you just need to learn how to start solving tough business problems at work, you need a solid understanding of not only what makes things fast, but also what makes them slow.

I work with consulting clients worldwide fixing complex SQL Server performance problems. I want to teach you how to do the same thing using the same troubleshooting tools and techniques I do.

I’m going to crack open my bag of tricks and show you exactly how I find which queries to tune, indexes to add, and changes to make. In this day long session, you’re going to learn about hardware, query rewrites that work, effective index design patterns, and more.

Before you get to the cutting edge, you need to have a good foundation. I’m going to teach you how to find and fix performance problems with confidence.

Event Details:

Get your tickets here for this event!

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

The Art Of The SQL Server Stored Procedure: Wrap Up And Final Thoughts

Thanks For Reading!


Here’s the full list of posts from the series:

I will eventually revisit these to record some material on them, and perhaps expand on it.

But this is good enough for now.

Follow On


By far, the most commented on post in the series is the one about formatting. It’s mostly people continuing to be wrong about it, but it was interesting to note.

The general idea of the series was to teach developers about the types of things I always seem to be fixing and adjusting, so that I can hopefully fix really interesting problems in the future.

Of course, that all depends on folks finding these and reading them. If that were the general sway of the world, I’d probably never had been in business in the first place.

So if you’re here and reading this, you’re special.

Thanks for reading!

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

The Art Of The SQL Server Stored Procedure: Cursors And Loops

I Don’t See Nothin’ Wrong


You will, for better or worse, run into occasions in your database career that necessitate the use of loops and cursors.

While I do spend a goodly amount of time reworking this sort of code to not use loops and cursors, there are plenty of reasonable uses for them.

In most cases, I would prefer to see developers write a loop manually, because there are many difficult to understand cursor options that change their behavior, and can really mess with query performance.

Those are just a couple examples from my site, without getting into all the many other variations in cursor types.

One very important distinction in cursor types is what set of data the queries in them will see.

As an example, static cursors make a copy of the results of the select, and then use that copy (like a temporary table) to work off of. Dynamic cursors read the underlying data on each iteration.

Especially in cases where your cursors may modify data, and you expect changes to the data to be reflected in future iterations of the cursor, static cursors are likely not a good choice.

Likewise, if you’re already putting data into a temporary table for the cursor to work off of, there is likely no benefit to a static cursor type because you’re just making another copy of the temporary table’s contents.

As I’ve said before, it’s up to you to test different options to see which works correctly and performs the best for your needs.

Also as I’ve said before, never trust the defaults. When you just write DECLARE CURSOR with no options specified, SQL Server may make a lot of bad decisions.

Appropriate Uses


In general, I’d expect many readers to understand when a loop or cursor is the only (or best) way to accomplish what you need.

I use them in several of my stored procedures because they’re the only sensible way to get things done.

Notably, in sp_LogHunter:

DECLARE
    c
CURSOR
    LOCAL /*Nothing outside of this needs to reference it*/
    SCROLL /*I want the FETCH FIRST command available to me*/
    DYNAMIC /*I don't want to make another copy of my temp table*/
    READ_ONLY /*Okay, perhaps this is over communicating*/
FOR
SELECT
    command
FROM #search;

The only real additional point I’d make here is that much like with dynamic SQL, if you’re going to use these options, you’re doing yourself (and anyone else who many need to run or manage your code) a great service by building verbose debugging options and error handling into it.

Knowing things like:

  • Starting row counts
  • Iteration count/progress
  • Current parameter/variable values
  • Rows in play per iteration
  • Iteration timing

Are all quite useful landmarks in your running code, help you find logical errors, and which parts you may be getting stuck on from a performance point of view.

Also from sp_LogHunter:

IF @debug = 1
BEGIN
    RAISERROR('log %i of %i', 0, 1, @l_log, @h_log) WITH NOWAIT; /*Which log file I'm working on, out of how many I need to process*/
    RAISERROR('search %i of %i', 0, 1, @l_count, @t_searches) WITH NOWAIT; /*Which search I'm working on, out of how many I need to run*/
    RAISERROR('@c: %s', 0, 1, @c) WITH NOWAIT; /*The current search command syntax*/
END;

These are all things that helped me while writing it for reasons listed above.

You, like me, may be surprised where your loops take you when they finally sally forth.

Thanks for reading!

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

The Art Of The SQL Server Stored Procedure: Pagination

Video Star


A common-enough practice to limit search results is to write paginated queries. You may give users the ability to specify how many rows per page they want to see, or you may have a set number of rows per page.

But the end goal is to allow users to very quickly get a smaller number of rows returned to them. Almost no one needs to get many thousands of rows back, unless they’re planning on exporting the data.

In these four videos, which are part of my paid training, I’ll teach you how to write reliably fast paging queries in SQL Server.

If you like this sort of stuff, head to the very end of my blog post for a discount code on all of my training.

 

 

 

 

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

The Art Of The SQL Server Stored Procedure: Dynamic Searches

Convenience


Like having a built-in type to make dynamic SQL more easily managed, it would also be nice to have some mechanism to manage dynamic searches.

Of course, what I mean by dynamic searches is when you have a variety of parameters that users can potentially search on, with none or few of them being required.

Right now, of course, your first best option is writing parameterized dynamic SQL that generates a where clause specific to the set of non-null parameters being sent to the procedure.

Your second best option is to just slap a statement-level recompile hint on things.

There are of course pros and cons to each.

Dynamic SQL vs Option Recompile


For dynamic SQL, the main cons are:

  • Relying on developers to write it safely
  • Detachment from stored procedure (harder to identify in query store/plan cache)
  • Potentially managing permissions
  • Handling increasingly complex scenarios (lists of columns, custom ordering, optional joins, etc.)

The main pros are:

  • Execution plan reusability
  • Transparent optimizer predicates
  • Consistent performance for indexed searches
  • Incredible flexibility

For OPTION(RECOMPILE), there’s really not a whole lot of up and down.

  • Pro: Really simple to slap on the end of a query
  • Con: Query constantly compiles a new plan (no plan reuse, but no cache bloat either)

As long as you have query store enabled, tracking the plans over time is available to you.

I don’t see queries that take a long time to compile all too frequently, but it’s worth keeping an eye on compilation time if you’re trying to balance these options locally.

Trust me on this one.

Managing Performance


The most difficult part, once you’ve made a choice, is figuring out indexing. Assuming you want searches to be fast, this is important.

Assuming you have just one table, and just 6 optional parameters, there are over 700 potential index key column combinations to consider.

Once you consider includes, and throwing columns into the mix for sorting, the mind begins to boggle.

In many cases, nonclustered column store indexes make this a lot easier. Row store indexes are built in a way that may make having many similar ones necessary.

Key column order makes a lot of difference when speeding up searches generally. An index on columns (a, b, c) will certainly make a variety of searches fast, so long as column a is part of the predicates.

Searches that do not filter on column a will have to scan the full index. Now you’re stuck trying to figure out what the most common or important searches are.

In this video, I talk about solving indexing problems with this type of query.

Thanks for reading (and watching)!

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

Happy New Year, From Darling Data

Up All Night


I don’t wanna. Neither do you. Have a good one.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

The Art Of The SQL Server Stored Procedure: Parameter Sniffing

Responsible Reading


One of the most fun parts of my job is telling people that parameter sniffing is a good thing. They’re often shocked, because the phrase has such ominous connotations to developers.

The parameters! They get sniffed! Performance! Something! Stuff!

Parameter sniffing does not mean the sky is falling. Like I’ve said before, if it were such a bad idea, it would have been abandoned.

What you have to be mindful of is how data gets distributed over time, and doing a few things to make sure the optimizer has as few reasonable plan choices as possible.

  • Indexes keys that cover all where clause predicates at minimum
  • Indexes includes that cover all select list columns in special circumstances
  • Queries written to best take advantage of those indexes (SARGability, etc.)
  • Using temporary objects to isolate critical parts of queries into manageable chunks

That will help cut down on how disastrous plan sharing is between queries that need to process very different amounts of data.

Under really crazy circumstances (and I know, I’ve linked to this before), I use dynamic SQL to fix parameter sensitivity issues.

 

SQL Server 2022 Not To The Rescue


SQL Server 2022 has a feature that’s suppose to help with parameter sensitivity issues. It’s okay sometimes, but the current set of limitations are:

  • Only equality predicate parameters are evaluated for sensitivity issues
  • Only one parameter is selected for managing sensitivity
  • Only three query plan choices are generated for different estimation buckets

I have seen it be helpful in a few cases, but there are times when it should obviously kick in, but doesn’t.

There is currently no query hint to force the feature to kick in when it doesn’t (and it should).

Just so you know how I set up for this:

ALTER DATABASE StackOverflow2013 
SET COMPATIBILITY_LEVEL = 160;

Here are my indexes:

CREATE INDEX
    beavis
ON dbo.Posts
    (OwnerUserId, PostTypeId)
WHERE
    (PostTypeId = 1)
WITH
    (MAXDOP = 8, SORT_IN_TEMPDB = ON, DATA_COMPRESSION = PAGE);

CREATE INDEX
    butthead
ON dbo.Votes
    (VoteTypeId, UserId, PostId)
INCLUDE
    (BountyAmount, CreationDate)
WITH
    (MAXDOP = 8, SORT_IN_TEMPDB = ON, DATA_COMPRESSION = PAGE);

CREATE INDEX
    stewart
ON dbo.Badges
    (UserId)
WITH
    (MAXDOP = 8, SORT_IN_TEMPDB = ON, DATA_COMPRESSION = PAGE);
GO

Here are the checks that validate options, etc.

SELECT
    d.name,
    d.compatibility_level,
    d.is_query_store_on
FROM sys.databases AS d
WHERE d.database_id = DB_ID();

SELECT
    dsc.configuration_id,
    dsc.name,
    dsc.value,
    dsc.is_value_default
FROM sys.database_scoped_configurations AS dsc
WHERE dsc.name = N'PARAMETER_SENSITIVE_PLAN_OPTIMIZATION';
sql server
way to be

With That Out Of The Way


Here’s the stored procedure that causes us grief.

CREATE OR ALTER PROCEDURE
   dbo.VoteSniffing
(
    @VoteTypeId integer
)
AS
BEGIN
SET XACT_ABORT, NOCOUNT ON;

    SELECT
        UserId  = 
            ISNULL(v.UserId, 0),
        Votes2013  = 
            SUM
            (
                CASE
                    WHEN 
                    (
                            v.CreationDate >= '20130101'
                        AND v.CreationDate <  '20140101'
                    )
                    THEN 1
                    ELSE 0
                END
            ),
        TotalBounty  = 
            SUM
            (
                CASE
                    WHEN v.BountyAmount IS NULL
                    THEN 0
                    ELSE 1
                END
            ),
        PostCount  = 
            COUNT(DISTINCT v.PostId),
        VoteTypeId  = 
            @VoteTypeId
    FROM dbo.Votes AS v
    WHERE v.VoteTypeId = @VoteTypeId
    AND   NOT EXISTS
    (
        SELECT
            1/0
        FROM dbo.Posts AS p
        JOIN dbo.Badges AS b
          ON b.UserId = p.OwnerUserId
        WHERE  p.OwnerUserId = v.UserId
        AND    p.PostTypeId = 1
    )
    GROUP BY 
        v.UserId
    ORDER BY
        PostCount DESC;
END;
GO

The good news is that all formatting checks pass here. Thank you, thank you, thank you. You’re far too kind.

Execution Problems


There’s only one parameter. It’s an equality predicate. And the VoteTypeId Column is highly volative.

sql server
oh dear me

You would think that with data distributions that look this way, there would be some attempt to remedy the situation by the Parameter Sensitive Plan Optimization feature.

Unfortunately, there is not.

Little Big Plan


When executed first with 4 as the VoteTypeId, the plan is very fast. Of course it is. We’re finding 733 rows to work with.

EXEC dbo.VoteSniffing
    @VoteTypeId = 4;
sql server execution plan
simple as

I have no qualms with this plan, other than the erroneous No Join Predicate warning. There is a join predicate, and you can see it in the Seek operators.

Heck, you can even see it in the query text. It’s pretty stupid.

When the plan is repeated for VoteTypeId 2, it is a disaster.

EXEC dbo.VoteSniffing
    @VoteTypeId = 2;
sql server query plan
not so hot

A Seek takes 9.5 seconds, a Nested Loops join occurs for 5 seconds, and a spill occurs for 22 seconds.

After the spill, another ~22 seconds is spent in various operators until it’s complete.

In other words, we still have our work cut out for us to tune this thing. If we go back to our list:

  • Indexes keys that cover all where clause predicates at minimum ✅
  • Indexes includes that cover all select list columns in special circumstances ✅
  • Queries written to best take advantage of those indexes (SARGability, etc.) ✅
  • Using temporary objects to isolate critical parts of queries into manageable chunks ❌

In this case, if we use a #temp table and isolate the portion of the query that evaluates a parameter (just getting data from the Votes table), we can solve 99% of the parameter sniffing problems between the majority of the VoteTypeIds.

However, VoteTypeId 2 is still a special flower. This is a good place to pause and ask ourselves if repeatedly filling a temporary object with 37 million rows is a good use of time.

This is when some techniques we’ve discussed before would come in handy:

  • Dynamic SQL to treat VoteTypeId 2 differently from the others
  • A separate stored procedure to intercept VoteTypeId 2 executions

Game Change


The reason why this is a more sensible arrangement is because when the plan is compiled initially for VoteTypeId 2, it runs just fine.

It’s also true that when VoteTypeId 4 uses the plan for VoteTypeId 2, it goes from taking 0 seconds across to taking a couple seconds. Sharing is not caring.

This is the fast plan for VoteTypeId 2. It has a lot of operators with additional startup costs that make grabbing small amounts of data slow.

sql server query plan
good for some

Parameter sensitivity is a tough thing at times. Managing multiple plans for the same query can end up being quite complicated.

There are absolutely times when you’re going to have to throw in the towel and use a statement-level recompile hint to make things function appropriately.

Thanks for reading!

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

The Art Of The SQL Server Stored Procedure: Temporary Objects

And Then You Were Gone


I probably overuse temp tables. I’m the first one to admit it. But often when I choose to use one, it’s a choice between:

  • Spending a long time trying to business understand logic for a company I don’t work for
  • Tinkering with query syntax and using really weird tricks and hints to get different plan shapes
  • Waiting a long time to create or tweak indexes on huge tables (usually on “dev” hardware)

And you know what? If I had to work anywhere full time, and not balance performance tuning requests from a bunch of people, I just might spend more time on those things.

Don’t tell anyone, but sometimes if I’m really busy, I have no problem waiting a long time for an index change to go through, so I can look at something else for a while.

In general, temp tables offer a convenient way to focus on one problem part of the query, without the noise of a bazillion other joins and applys and subqueries and everything else going on.

They also offer a familiar paradigm to anyone looking at the work later, that doesn’t involve SELECT TOP (2147483647) UNION ALL SELECT TOP (1) type syntax, which can really raise some eyebrows on unfamiliar faces.

A Recent Example


I’m using Stack Overflow in my query, but a recent client engagement had a problem query that looked about the same.

WITH
    top5 AS
(
    SELECT
        p.*,
        n = 
            ROW_NUMBER() OVER
            (
                PARTITION BY
                    p.OwnerUserId
                ORDER BY
                    p.Score DESC
            )
    FROM dbo.Posts AS p
    WHERE p.OwnerUserId = 22656
    AND   p.PostTypeId = 1
)
SELECT
    u.DisplayName,
    t1.Title,
    t2.Title,
    t3.Title,
    t4.Title,
    t5.Title
FROM dbo.Users AS u
LEFT JOIN top5 AS t1
  ON  t1.OwnerUserId = u.Id
  AND t1.n = 1
LEFT JOIN top5 AS t2
  ON  t2.OwnerUserId = u.Id
  AND t2.n = 2
LEFT JOIN top5 AS t3
  ON  t3.OwnerUserId = u.Id
  AND t3.n = 3
LEFT JOIN top5 AS t4
  ON  t4.OwnerUserId = u.Id
  AND t4.n = 4
LEFT JOIN top5 AS t5
  ON  t5.OwnerUserId = u.Id
  AND t5.n = 5
WHERE t1.OwnerUserId IS NOT NULL;

The problems with this contain multitudes, but let’s let the query plan do the talking:

sql server query plan
oh, that.

In the words of Drake: “Take Care”

If you’re the type of developer who insists that common table expressions are some sort of magic potion, I do hope that you this is your hangover.

Practical Magic


Temporary tables are the cure for common table expression hangovers, for the most part.

Let’s see how we do with one.

WITH
    top5 AS
(
    SELECT
        p.*,
        n = 
            ROW_NUMBER() OVER
            (
                PARTITION BY
                    p.OwnerUserId
                ORDER BY
                    p.Score DESC
            )
    FROM dbo.Posts AS p
    WHERE p.OwnerUserId = 22656
    AND   p.PostTypeId = 1
)
SELECT
    t.OwnerUserId,
    t.Title,
    n
INTO #top5
FROM top5 AS t
WHERE t.n <= 5;

SELECT
    u.DisplayName,
    t1.Title,
    t2.Title,
    t3.Title,
    t4.Title,
    t5.Title
FROM dbo.Users AS u
LEFT JOIN #top5 AS t1
  ON  t1.OwnerUserId = u.Id
  AND t1.n = 1
LEFT JOIN #top5 AS t2
  ON  t2.OwnerUserId = u.Id
  AND t2.n = 2
LEFT JOIN #top5 AS t3
  ON  t3.OwnerUserId = u.Id
  AND t3.n = 3
LEFT JOIN #top5 AS t4
  ON  t4.OwnerUserId = u.Id
  AND t4.n = 4
LEFT JOIN #top5 AS t5
  ON  t5.OwnerUserId = u.Id
  AND t5.n = 5
WHERE t1.OwnerUserId IS NOT NULL;

Here’s the new query plan:

sql server query plan
hi mom

Without changing indexes, we get our query from 26 seconds to 1.5 seconds.

Really puts the “common” in “common table expressions”, doesn’t it?

Argue


While the “should I use common table expressions or temp tables?” question is largely a settled one for me, one thing that always comes up in seemingly chaotic practice is temp tables vs. table variables.

Table variables certainly have their place in the world, though often not without compromises or alternatives.

Many developers will use table variables for the purpose of passing them directly to another stored procedure. That’s fine and all, but you can just use #temp tables and reference them directly in another stored procedure, too.

Other times, which on gets used seems to be a question of moods and fancies of the developer currently working on something.

Some time ago I talked about when table variables are most useful in SQL Server practice.

And I stand by the majority of what’s discussed in here still. Hopefully you find it enlightening, even if you’ve already seen it.

Thanks for reading!

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.