If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.
I am sometimes asked to write special bits of code by people to solve a specific problem they’re having.
A recent one was for quite an unreasonable situation brought on by a shamelessly written vendor application, where:
Implicit transactions were in use
Bizarre locking hints abounds
Absolutely zero attention paid to transaction handling
Which lead to scenarios where select queries would run, finish, and never close out their connection. Of course, this was bad, because loads of other queries would get blocked by these things that should have just ended their sessions and released their locks and been on their way.
And so I wrote this thing. A thing that I’d always sort of made fun of the concept of, because I’d seen so many bad implementations of it throughout the years.
Most of them would just look for lead blockers and kill them without any consideration as to how much work they’d done, which would lead to even more blocking during rollback.
This one specifically looks for things that have used zero transaction log space.
Here it is. I don’t love it, but I wanted to share it, because it might make you feel better about some code that you weren’t proud to write, either.
Thanks for reading!
/*
EXEC dbo.sleeper_killer
@debug = 'true';
SELECT
sk.*
FROM dbo.killed_sleepers AS sk;
*/
SET ANSI_NULLS ON;
SET ANSI_PADDING ON;
SET ANSI_WARNINGS ON;
SET ARITHABORT ON;
SET CONCAT_NULL_YIELDS_NULL ON;
SET QUOTED_IDENTIFIER ON;
SET NUMERIC_ROUNDABORT OFF;
SET IMPLICIT_TRANSACTIONS OFF;
SET STATISTICS TIME, IO OFF;
GO
CREATE OR ALTER PROCEDURE
dbo.sleeper_killer
(
@debug bit = 'false'
)
AS
BEGIN
SET NOCOUNT ON;
/*Make sure the logging table exists*/
IF OBJECT_ID('dbo.killed_sleepers') IS NULL
BEGIN
CREATE TABLE
dbo.killed_sleepers
(
run_id bigint IDENTITY PRIMARY KEY,
run_date datetime NOT NULL DEFAULT SYSDATETIME(),
session_id integer NULL,
host_name sysname NULL,
login_name sysname NULL,
program_name sysname NULL,
last_request_end_time datetime NULL,
duration_seconds integer NULL,
last_executed_query nvarchar(4000) NULL,
error_number integer NULL,
severity tinyint NULL,
state tinyint NULL,
error_message nvarchar(2048),
procedure_name sysname NULL,
error_line integer NULL
);
END;
/*Check for any work to do*/
IF EXISTS
(
SELECT
1/0
FROM sys.dm_exec_sessions AS s
JOIN sys.dm_tran_session_transactions AS tst
ON tst.session_id = s.session_id
JOIN sys.dm_tran_database_transactions AS tdt
ON tdt.transaction_id = tst.transaction_id
WHERE s.status = N'sleeping'
AND s.last_request_end_time <= DATEADD(SECOND, -5, SYSDATETIME())
AND tdt.database_transaction_log_bytes_used < 1
)
BEGIN
IF @debug = 'true' BEGIN RAISERROR('Declaring variables', 0, 1) WITH NOWAIT; END;
/*Declare variables for the cursor loop*/
DECLARE
@session_id integer,
@host_name sysname,
@login_name sysname,
@program_name sysname,
@last_request_end_time datetime,
@duration_seconds integer,
@last_executed_query nvarchar(4000),
@kill nvarchar(11);
IF @debug = 'true' BEGIN RAISERROR('Declaring cursor', 0, 1) WITH NOWAIT; END;
/*Declare a cursor that will work off live data*/
DECLARE
killer
CURSOR
LOCAL
SCROLL
READ_ONLY
FOR
SELECT
s.session_id,
s.host_name,
s.login_name,
s.program_name,
s.last_request_end_time,
duration_seconds =
DATEDIFF(SECOND, s.last_request_end_time, GETDATE()),
last_executed_query =
SUBSTRING(ib.event_info, 1, 4000),
kill_cmd =
N'KILL ' + RTRIM(s.session_id) + N';'
FROM sys.dm_exec_sessions AS s
JOIN sys.dm_tran_session_transactions AS tst
ON tst.session_id = s.session_id
JOIN sys.dm_tran_database_transactions AS tdt
ON tdt.transaction_id = tst.transaction_id
OUTER APPLY sys.dm_exec_input_buffer(s.session_id, NULL) AS ib
WHERE s.status = N'sleeping'
AND s.last_request_end_time <= DATEADD(SECOND, -5, SYSDATETIME())
AND tdt.database_transaction_log_bytes_used < 1
ORDER BY
duration_seconds DESC;
IF @debug = 'true' BEGIN RAISERROR('Opening cursor', 0, 1) WITH NOWAIT; END;
/*Open the cursor*/
OPEN killer;
IF @debug = 'true' BEGIN RAISERROR('Fetch first from cursor', 0, 1) WITH NOWAIT; END;
/*Fetch the initial row*/
FETCH FIRST
FROM killer
INTO
@session_id,
@host_name,
@login_name,
@program_name,
@last_request_end_time,
@duration_seconds,
@last_executed_query,
@kill;
/*Enter the cursor loop*/
WHILE @@FETCH_STATUS = 0
BEGIN
BEGIN TRY
IF @debug = 'true' BEGIN RAISERROR('Insert', 0, 1) WITH NOWAIT; END;
/*Insert session details to the logging table*/
INSERT
dbo.killed_sleepers
(
session_id,
host_name,
login_name,
program_name,
last_request_end_time,
duration_seconds,
last_executed_query
)
VALUES
(
@session_id,
@host_name,
@login_name,
@program_name,
@last_request_end_time,
@duration_seconds,
@last_executed_query
);
IF @debug = 'true' BEGIN RAISERROR('Killing...', 0, 1) WITH NOWAIT; END;
IF @debug = 'true' BEGIN RAISERROR(@kill, 0, 1) WITH NOWAIT; END;
/*Kill the session*/
EXEC sys.sp_executesql
@kill;
END TRY
BEGIN CATCH
IF @debug = 'true' BEGIN RAISERROR('Catch block', 0, 1) WITH NOWAIT; END;
/*Insert this in the event of an error*/
INSERT
dbo.killed_sleepers
(
session_id,
host_name,
login_name,
program_name,
last_request_end_time,
duration_seconds,
last_executed_query,
error_number,
severity,
state,
error_message,
procedure_name,
error_line
)
SELECT
@session_id,
@host_name,
@login_name,
@program_name,
@last_request_end_time,
@duration_seconds,
@last_executed_query,
ERROR_NUMBER(),
ERROR_SEVERITY(),
ERROR_STATE(),
ERROR_MESSAGE(),
ERROR_PROCEDURE(),
ERROR_LINE();
END CATCH;
IF @debug = 'true' BEGIN RAISERROR('Fetching next', 0, 1) WITH NOWAIT; END;
/*Grab the next session to kill*/
FETCH NEXT
FROM killer
INTO
@session_id,
@host_name,
@login_name,
@program_name,
@last_request_end_time,
@duration_seconds,
@last_executed_query,
@kill;
END;
IF @debug = 'true' BEGIN RAISERROR('Closedown time again', 0, 1) WITH NOWAIT; END;
/*Shut things down*/
CLOSE killer;
DEALLOCATE killer;
END;
END; --Final END
Going Further
If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.
Why You Should Avoid People Who Tell You To Avoid Subqueries In SQL Server
Thanks for watching!
Going Further
If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.
Cursors In Scalar UDFs, and Other Performance Pitfalls In SQL Server
Going Further
If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.
I’m going to be totally open and honest with you, dear reader: I’ve been experimenting with… AI.
See, I’m just a lonely independent consultant, and sometimes it’s just nice to have someone to talk to. It’s also kind of fun to take a query idea you have, and ask “someone” else to write it to see what they’d come up with.
ChatGPT (for reference, 4 and 4o) does a rather okay job sometimes. In fact, when I ask it to write a query, it usually comes up with a query that looks a lot like the ones that I have to fix when I’m working with clients.
If I poke and prod it enough about the things that it has done wrongly, it will agree with me and do things the right way, eventually. That is an improvement over your average T-SQL developer.
Your average T-SQL developer will spend a terrible amount of time trying to figure out ways to write queries incorrectly, even when you show them the right way to do something, often under the assumption that they’ve found the one time it’s okay to do it wrong.
For this post, I came up with a query idea, wrote a query that did what I wanted, and then asked the AI to write its own version.
It came pretty close in general, and even added in a little touch that I liked and hadn’t thought of.
Duplicate Post Finder
Here’s the query I wrote, combined with the nice touch that ChatGPT added.
WITH
DuplicateTitles AS
(
SELECT
Title,
EarliestPostId = MIN(p.Id),
FirstPostDate = MIN(p.CreationDate),
LastPostDate = MAX(p.CreationDate),
DuplicatePostIds =
STRING_AGG
(CONVERT(varchar(MAX), p.Id), ', ')
WITHIN GROUP
(ORDER BY p.Id),
TotalDupeScore = SUM(p.Score),
DuplicateCount = COUNT_BIG(*) - 1
FROM dbo.Posts AS p
WHERE p.PostTypeId = 1
GROUP BY
p.Title
HAVING
COUNT_BIG(*) > 1
)
SELECT
dt.Title,
dt.FirstPostDate,
dt.LastPostDate,
dt.DuplicatePostIds,
dt.DuplicateCount,
TotalDupeScore =
dt.TotalDupeScore - p.Score
FROM DuplicateTitles dt
JOIN dbo.Posts p
ON dt.EarliestPostId = p.Id
AND p.PostTypeId = 1
ORDER BY
dt.DuplicateCount DESC,
TotalDupeScore DESC;
If you’re wondering what the nice touch is, it’s the - 1 in DuplicateCount = COUNT_BIG(*) - 1, and I totally didn’t think of doing that, even though it makes total sense.
So, good job there.
Let’s Talk About Tuning
To start, I added this index. Some of these columns could definitely be moved to the includes, but I wanted to see how having as many of the aggregation columns in the key of the index would help with sorting that data.
Those datums? These datas? I think one of those is right, probably.
CREATE INDEX
p
ON dbo.Posts
(PostTypeId, Title, CreationDate, Score)
WITH
(SORT_IN_TEMPDB = ON, DATA_COMPRESSION = PAGE);
It leads with PostTypeId, since that’s the only column we’re filtering on to find questions, which are the only things that can have titles.
But SQL Server’s cost-based optimizer makes a very odd choice here. Let’s look at that there query plan.
two filters? in my query plan?
There’s one expected Filter in the query plan, for the COUNT_BIG(*) > 1 predicate, which makes absolute sense. We don’t know what the count will be ahead of time, so we have to calculate and filter it on the fly.
The one that is entirely unexpected is for PostTypeId = 1, because WE HAVE AN INDEX THAT LEADS WITH POSTTYPEID.
My first thought was that that, since we’re doing this: (CONVERT(varchar(MAX), p.Id), ', '), that the compute scalar right before the filter was preventing the predicate on PostTypeId from being pushed into an index seek.
Keep in mind that this is quite often necessary when using STRING_AGG, because the implementation is pretty half-assed even by Microsoft standards. And unfortunately, the summer intern who worked on it has since moved on to be a Senior Vice President elsewhere in the organization.
At first I experimented with using smaller byte lengths in the convert. And yeah, somewhere in the 500-600 range, the plan would change to an index seek. But this wasn’t reliable. Different stats samplings and compatibility levels would leave me with different plans (switching between a seek and a scan). The only thing that worked reliably is using a FORCESEEK hint to override the optimizer’s mishandling.
This changes the plan to something quite agreeable, that no longer takes 12 seconds.
STRING_AGG? more like STRING_GAG! 🥁 🤡
So why the decision to use the first plan, au naturale, instead of the plan that took me forcing things to seek?
12 second plan: 706 query bucks
4 second plan: 8,549 query bucks
The faster plan was estimated to cost nearly 10x the query bucks to execute. Go figure.
For anyone who needed a reminder:
High cost doesn’t mean slow
Low cost doesn’t mean fast
All costs are estimates, with no bearing on the reality of query execution
Thanks for reading!
Going Further
If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.
Profiling Query Performance In SQL Server With Extended Events The Easy Way
Thanks for watching!
Going Further
If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.
Plan Cache Pollution From Temporary Objects In SQL Server
Thanks for watching!
Going Further
If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.
If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.
Or at least at some point, back when I gave all my precious blog blood somewhere else, I used to quite enjoy writing the release notes for the First Responder Kit. It was fun, and there were a lot of contributors to credit.
This most recent release had a note in it that got me taking a stroll down memory lane.
Deprecating sp_BlitzInMemoryOLTP, sp_BlitzQueryStore, and sp_AllNightLog
sp_BlitzQueryStore was originally written by Erik Darling when he worked here. He’s moved on to start his own excellent company, plus his own sp_QuickieStore. You should be using that instead.
sp_BlitzInMemoryOLTP was always kinda distributed as a courtesy – the real home for it is in KTaranov’s Github repository, and you can still find it there. It hasn’t been updated in over 6 years, and I’ve never seen anyone using it, so I’m removing it to streamline support issues.
sp_AllNightLog was a ton of fun when we built it several years ago, but it’s consistently had a problem. Companies start using it, then decide they want to build something even more ambitious, typically a C# service with robust error handling and scheduling. sp_AllNightLog isn’t the kind of thing I want to encourage beginners to use – it’s complex.
I didn’t have much to do with sp_BlitzInMemoryOLTP. I’m still not entirely sure what it does. All I know is that In-Memory was the hottest frog in the pan for exactly 14.9 minutes.
But I have some quite fond memories of building sp_BlitzQueryStore, and sp_AllNightLog.
I didn’t write every bit of code in either one, but I definitely started work on and them pitched in quite a bit on both. Other contributors deserve whatever credit they’re publicly willing to take.
If you’ll permit a fella with a lot more grey on his face than there was when these two procedures were first F5 birthed into the world to reminisce a bit, I would like to eulogize them here.
sp_BlitzQueryStore
I had been working on sp_BlitzCache just about full time for a couple years, since Jeremiah (who wrote it originally) had decided to embark on a professorial career. When Query Store got announced, I knew I wanted to write something for it.
After all, this seemed like a no-brainer for folks on SQL Server 2016 to adopt. I just had no concept of what I wanted to do, until one day…
I had just gotten off the phone with the worst credit card company in the world, because someone had purchased ONE-HUNDRED $99 Play Station gift cards from a Russian IP address with a .ru email, and they told me that I would need to fill out 100 dispute PDFs to dispute each charge separately.
I forget where I was walking home from, but I was in the company Slack, and I had a message from BrentO asking how I wanted to approach it, and I felt like I had to make something good up on the spot. It turned out to be: I want to find all of the worst metric spikes, and grab the queries that ran during them. So it would look for the highest CPU, reads, writes, memory, tempdb, etc. and look for the queries responsible for them. And since we have all this groovy historical data, I wanted to show which queries were sensitive to parameter sensitivity by looking for wild swings in those metrics.
In theory, this was a great idea. In practice, those queries were god awful slow. It wasn’t all my fault, of course; I can’t take full credit. I see a lot of scripts (including queries from the SSMS GUI) that hit Query Store which are equally as God awful slow.
Perhaps ironically, some of the absolute slowest points in any Query Store query are the ones that hit the “in memory” tables.
that is seven minutes, thank you for asking
At any rate, SQL Server 2016 adoption was fairly slow, and Query Store adoption was even slower. It was even hard to recommend turning it on at first because of all the bugs and issues that were cropping up and getting fixed in CUs (and even then, Service Packs). SQL Server 2017 didn’t help things at all, and I was out on my own in the world by the time SQL Server 2019 got released.
So poor ol’ sp_BlitzQueryStore languished a bit. Of course, as I added checks and gizmos to sp_BlitzCache, I’d also add them to sp_BlitzQueryStore, but… It just wasn’t the same every day utility belt tool for me.
When I sort of lost faith in the whole thing was sometime in 2018 when I tweaked a query in sp_BlitzQueryStore to try to speed things up, and it made my local SQL instance stack dump, and I had to manually restart it. If that happened with a client, hoo boy!
But here’s to you, sp_BlitzQueryStore! It’s how I first started learning the Query Store DMVs, how they related, and what data was in them.
You’re like that long-term relationship that ends before you meet the person you end up marrying.
sp_AllNightLog
This made me feel cool, because:
I’d always loved Log Shipping (still hate AGs)
It was my first “programming” stored procedure
Let me clarify point 2 a bit, because I’m not one of those “SQL isn’t coding” people. Most things that I write only work with data. This worked with REAL LIVE FILES. Writing them. Copying them. Restoring them. Across two different servers. Where neither one knew the other existed.
Wild. WILD!
I’m not sure if I’m allowed to say the name of the company that wanted it, but they were based in Chicago, so I ended up flying out there a couple times to work on it along with Brent.
That kind of stuff had never happened to me before, and has only happened a couple times since.
I learned some hard lessons from this one:
If you fat-finger an invalid date value in an output parameter, you’ll end up with a REALLY HARD BUG TO FIND
If you don’t add a WAITFOR to looping code that’s constantly looking for new databases, new backups to take, and new backups to restore, you can really drive one CPU crazy
It was also when I learned that you can’t add triggers to “system” tables, like restorehistory, in the dbo schema, in msdb. If we could have done that, a few things would have been way easier.
Of course, my fondest memory of this one was when it finally worked. I remember hitting a bazillion errors and issues and debugging stuff for ages. And then one magical day, the Agent jobs started up, and it was all green.
I was far from alone in working on it; I don’t want it to sound like THIS ONE’S ALL ME. There were a group of 3-4 people who put in work writing and testing things.
That was kind of the nicest thing about it — real collaboration with real people in real life — not just pushing changes around the internet and asking people to test them.
The Departed
While it is a bit sad to see them go, I totally understand why they had to. It’s difficult to be responsible for large code repos that you don’t use a lot, and have become unfamiliar with because you don’t use them regularly.
I’m a bit surprised that sp_BlitzBackups didn’t also end up in the deprecation pile. It hasn’t had an issue opened since 2019, or any commits aside from version bumps. But maybe it’s just that well-written! If that ever does get deprecated, I have exactly one funny memory of the writing process, and it probably won’t get a blog post. I can spare you the drama of “it runs really slow when there are hundreds of databases” and “how slow?” and “like a minute” and “why do you need to run it every 15 seconds anyway?”.
Perhaps the most surprising thing about the years since 2016 is that not a single third party monitoring tool has embraced Query Store data in their performance diagnostics. Hopefully someday someone tells them about it, I guess?
But hey, that’s enough about that! Go get the most recent release of the First Responder Kit because it has a bunch of new and improved in it. Enjoy your shinies while they last.
Thanks for reading!
Going Further
If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.
The Difference Between Read Committed And Read Committed Snapshot Isolation In SQL Server
Thanks for watching!
Going Further
If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.
